12
$\begingroup$

I am using neural networks to predict a time series. The question I'm facing now is how do I encode date/time/serial no. of each input set as an input to the neural network?

Should I use 1 of C encoding (used for encoding categories) as described here?

Or Should I just feed it the time (in milliseconds since 1-1-1970)?

Or is feeding it the time unnecessary as long as I feed it the rest of the data chronologically?

$\endgroup$
  • $\begingroup$ I am not sure why you think feeding it the time unnecessary if you are using neural networks to predict a time series. $\endgroup$ – scaaahu Sep 27 '13 at 9:32
  • $\begingroup$ there are many ways to encode dates/time. it would help to know the nature of the time series. it is not a priori something either to rule out, or rule in. $\endgroup$ – vzn Sep 27 '13 at 15:40
  • $\begingroup$ @vzn I am trying to predict a financial time series in general. If that is helpful. $\endgroup$ – Shayan RC Sep 30 '13 at 8:53
17
$\begingroup$

Neural Networks are not magic. If you treat them like they are and just throw data at them without thinking you're going to have a very bad time.

You need to stop and ask youself "Is milliseconds since 1970 actually going to be predictive of the event I'm interested in?" The answer you should arrive at immediately is no. Why? For every instance you actually care about (events in the future, the past already happened) the time variable will take on a value that is greater than any value the time variable will take in your training data. Such a variable is very unlikely to help. Even worse it is likely to cause overfitting (a serious problem for powerful non-linear models like neural networks) if you aren't careful.

Now what might make sense is a variable like week of the year or month that could help you model seasonal or annual effects. I've done some work in agricultural disease prediction where Julian day ended up being a very important variable. Based on this experience, I suspect you'd be better off encoding this type of variable as a categorical variable rather than ordinal, your experience may very. Notice that month or week of the year are repeatable events that one is likely to see many times in your training data and it is possible to explain why such a variable could impact a financial outcome. Contrast this with milliseconds since 1970 which is just a monotonically increasing value.

Lastly, from your statement "Or is feeding it the time unnecessary as long as I feed it the rest of the data chronologically?" it sounds like you might not have a very good grasp of how neural networks work. With a standard feedforward neural network the order you feed the network your data is going to have no impact on the predictions. Order may impact training if you're using stochastic or mini-batch gradient descent, but this is only an artifact of the iterative (as opposed to batch) training method. If you want to model temporal dependence with a neural network you'll need to use something like a sliding window or a recurrent neural network.

$\endgroup$
2
$\begingroup$

you state in comments you're working on financial time series prediction. whether to include time or not in inputs is somewhat domain-dependent. in general its relevant if you have some kind of "cyclic" time component in your data. for example, maybe there is some weekly effect in your data if it is running over weeks, or some intraday effect if it spans a single day. then if you have many cycles, include an input that reflects the position in the cycle, eg from $[-1,1]$ or $[0,1]$ or say $sin(t)$.

if you find that theres a consistent trend eg either increasing or decreasing over time say $f(t)$ its more appropriate to extract that from the inputs prior to feeding the inputs via what is called "detrending". in other words you subtract the formula $f(t)$ for that underlying trend and predict the trend plus difference $d_t = f_t + g_t$ but the NN focuses only on the difference $g_t$, not the trend $f_t$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.