You say you are overfitting.  The usual solution is to use more training data and/or regularization.

The easiest solution is probably going to be to train on all data, not just a limited time window.  Perhaps you can find a better mathematical optimization algorithm. You don't give any details on the optimization algorithm you are using or how you compute the loss function; perhaps using a stochastic version (e.g., analogous to SGD instead of GD) would provide better efficiency.  You could also consider Bayesian optimization, which constructs a differentiable model to estimate/predict the output of your loss function, and then uses SGD on it.

Or perhaps you can use regularization; for instance, if you have a prior on parameters, you might be able to choose a suitable regularization method.  I suspect it's going to be hard to say without getting into the specifics.