# Method for combining derivative free optimization results of different data inputs

I am working on an algorithm that has multiple fixed parameters. The algorithm analyzes time series data and spits out a number. The fixed parameters need to be such that this number is as small as possible.

What I found, is that when optimizing the parameters for a specific time period, these parameters don't necessarily work well when used on another time period.

The way I see it, is that there are two possible solutions to this problem:

1. use a longer time period when optimizing the parameters
2. find a method of combining the optimal parameters for different time periods, such that these "averaged" parameters work well on all time periods

Option 1. would be incredibly expensive in terms of computational time. And although it makes intuitive sense that this should fix the problem, I am not sure that this would indeed be the case.

Option 2. reminds me of training neural networks, where one would feed in a large number of "data points" and somehow take a (weighted) average of the results to find a set of parameters that work well for all data points. Unfortunately, I know very little to nothing about the algorithms used for this kind of optimization/learning.

Any help or suggestions are greatly appreciated. Please let me know if there is anything you'd like me to expand upon.

Thanks!

• my loss function is not an actual mathematical function, so I don't know exactly what it looks like, this also means that I can't compute the gradient.
• The optimization method I currently use is differential evolution using the best1bin strategy.
• The reason the predictions fail is because of overfitting.
• There are ~10 parameters
• One iteration takes about 5-8 seconds
• This question might be more appropriate for Data Science. If you ask it over there, please remove it here, to prevent duplicates. Jun 3, 2020 at 9:43
• This might be hard to answer without knowing what your loss function looks like, how the predictions depend on the parameters, whether you can compute a gradient, what optimization method you're currently using, and why the predictions fail on future times. I suggest reading about SGD, overfitting, and regularization.
– D.W.
Jun 4, 2020 at 2:25
• @D.W. To answer your questions: my loss function is not an actual mathematical function, so I don't know exactly what it looks like, this also means that I can't compute the gradient. The optimization method I currently use is differential evolution using the best1bin strategy. The reason the predictions fail is because of overfitting. Thank you for your suggestions, I'll read into them! Jun 4, 2020 at 9:04
• Thanks. Can you edit your question to incorporate that information into the question, so it reads well for someone who encounters it for the first time and so that people don't have to read the comments to understand what you are asking?
– D.W.
Jun 4, 2020 at 17:54
• How many parameters are there? There are a number of gradient-free optimization methods.
– D.W.
Jun 4, 2020 at 17:56