You say you are overfitting. The usual solution is to use more training data and/or regularization.
The easiest solution is probably going to be to train on all data, not just a limited time window. Perhaps you can find a better mathematical optimization algorithm. You don't give any details on the optimization algorithm you are using or how you compute the loss function; perhaps using a stochastic version (e.g., analogous to SGD instead of GD) would provide better efficiency. You could also consider Bayesian optimization, which constructs a differentiable model to estimate/predict the output of your loss function, and then uses SGD on it.
Or perhaps you can use regularization; for instance, if you have a prior on parameters, you might be able to choose a suitable regularization method. I suspect it's going to be hard to say without getting into the specifics.