On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of the written symbols, you have the way it was written in on-line recognition.

The paper Npen++: An On-Line Handwriting Recognition System describes a system that makes use of Hidden Markov Models (HMMs) and Artificial Neural Networks (ANNs) to recognize symbols.

[...] each character is modelled with three states representing the first, middle and last part of the character. Hence, the MS-TDNN can be regarded as a hybrid recognizer that combines features of neural networks and hidden Markov models (HMMs).

Source: Npen++: An On-Line Handwriting Recognition System, page 6.

One of my problems is that I don't know what a TDNN (Time delay neural network) is. At the moment, I think I've understood what feed-forward neural networks are and I know that they are trained with backpropagation. But all examples I have seen so far have a fixed input vector (e.g. colors of a fixed-size image; positions on a fixed-size board).

How exactly are the lists of coordinates mapped to the input neurons?

Is the following "mind model" I have of those HMMs correct?

enter image description here

When I understand it correct, then you have one HMM for each symbol. How do you get the probabilities $p_1, p_2, p_3, p_{1,2}, p_{2,3}$? How do you get the probability which character to recognize given the trained HMMs and a list of coordinates? When you can get a probability for every symbol with HMMs only, how does the ANN help?

  • $\begingroup$ Should I probably ask this on cstheory.stackexchange.com? $\endgroup$ May 29, 2014 at 9:46
  • $\begingroup$ I doubt you're going to get a better answer on cstheory.stackexchange.com. Maybe the Statistics StackExchange site (Cross Validated), or possibly Digital Signal Processing Stack Exchange? $\endgroup$
    – D.W.
    May 29, 2014 at 19:39


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.