Skip to main content
Computer Science

Return to Answer

added 329 characters in body
Source Link
D.W. ♦
D.W. ♦
  • 165.6k
  • 21
  • 230
  • 490

You say you are overfitting. The usual solution is to use more training data and/or regularization.

The easiest solution is probably going to be to train on all data, not just a limited time window. Perhaps you can find a better mathematical optimization algorithm. You don't give any details on the optimization algorithm you are using or how you compute the loss function; perhaps using a stochastic version (e.g., analogous to SGD instead of GD) would provide better efficiency. You could also consider Bayesian optimization, which constructs a differentiable model to estimate/predict the output of your loss function, and then uses SGD on it. Another option is to use gradient-based methods and estimate the gradient using finite differences; if you have 10 parameters, you can estimate the gradient by evaluating the function 11 times on 11 different inputs. Then you could combine that with gradient descent.

Or perhaps you can use regularization; for instance, if you have a prior on parameters, you might be able to choose a suitable regularization method. I suspect it's going to be hard to say without getting into the specifics.

You say you are overfitting. The usual solution is to use more training data and/or regularization.

The easiest solution is probably going to be to train on all data, not just a limited time window. Perhaps you can find a better mathematical optimization algorithm. You don't give any details on the optimization algorithm you are using or how you compute the loss function; perhaps using a stochastic version (e.g., analogous to SGD instead of GD) would provide better efficiency. You could also consider Bayesian optimization, which constructs a differentiable model to estimate/predict the output of your loss function, and then uses SGD on it.

Or perhaps you can use regularization; for instance, if you have a prior on parameters, you might be able to choose a suitable regularization method. I suspect it's going to be hard to say without getting into the specifics.

You say you are overfitting. The usual solution is to use more training data and/or regularization.

The easiest solution is probably going to be to train on all data, not just a limited time window. Perhaps you can find a better mathematical optimization algorithm. You don't give any details on the optimization algorithm you are using or how you compute the loss function; perhaps using a stochastic version (e.g., analogous to SGD instead of GD) would provide better efficiency. You could also consider Bayesian optimization, which constructs a differentiable model to estimate/predict the output of your loss function, and then uses SGD on it. Another option is to use gradient-based methods and estimate the gradient using finite differences; if you have 10 parameters, you can estimate the gradient by evaluating the function 11 times on 11 different inputs. Then you could combine that with gradient descent.

Or perhaps you can use regularization; for instance, if you have a prior on parameters, you might be able to choose a suitable regularization method. I suspect it's going to be hard to say without getting into the specifics.

Source Link
D.W. ♦
D.W. ♦
  • 165.6k
  • 21
  • 230
  • 490

You say you are overfitting. The usual solution is to use more training data and/or regularization.

The easiest solution is probably going to be to train on all data, not just a limited time window. Perhaps you can find a better mathematical optimization algorithm. You don't give any details on the optimization algorithm you are using or how you compute the loss function; perhaps using a stochastic version (e.g., analogous to SGD instead of GD) would provide better efficiency. You could also consider Bayesian optimization, which constructs a differentiable model to estimate/predict the output of your loss function, and then uses SGD on it.

Or perhaps you can use regularization; for instance, if you have a prior on parameters, you might be able to choose a suitable regularization method. I suspect it's going to be hard to say without getting into the specifics.