I am using N-Grams to predict future character input. For n+1, it performs really well.

I've been looking, but I have not been able to find any information on what the feasible maximum distance into the future N-Grams are capable of.

One way I could do this is to predict the next character (n+1) and then use that result to predict n+2, use n+2 to predict n+3... etc... but obviously an incorrect prediction will cascade and impact the future predictions as there is a dependency.

Is there a better (more accurate, reliable) way to do this? Also, I can easily use N-Grams to predict the next (n+1) character in an incoming stream, but what about n+2? How about n+3? When does it become unrealistic to predict further?

I'm sure there is some research on this, so I'd love it if you could link me to a paper or give me a brief overview to get me going.


There's no simple answer to "how far into the future can you predict?". Obviously, the further you go, the less your ability to predict. For English text, I would expect that ability to predict would decrease rapidly.

You asked if there's a better wya to do this. You can of course consider a generalization of $n$-grams where you use the characters at positions $i-n+1,i-n+2,\dots,i-1,i$ to predict the character at position $i+k$ (instead of predicting the character at position $i+1$), for any fixed $k$. This will probably perform worse the larger $k$ is.

You might also be interested in skipgrams.

| cite | improve this answer | |
  • $\begingroup$ Great, thanks. I had thought about doing what skip grams does, but I had no idea what to call! $\endgroup$ – pookie May 20 '17 at 23:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.