# Hidden Markov Models for Hand Gestures

I am completing a final year project for hand gesture recognition using Hidden Markov Models

I have a fair understanding of Hidden Markov Models and how they work using simple examples such as the Unfair Casino and some Weather examples.

I am looking to implement multiple Hidden markov models where each model corresponds to a single gesture, similarly to this paper where the observed states are the angles between the coordinates of different points. This would create a sequence of numbers from 0 to 18 as seen in Figure 3 and Figure 4. .

What would the hidden states be in terms of this scenario?

The weather example has the observations 'Walk', 'Shop' and 'Clean' which would be the numbers 0-18 in the hand gesture case, however I do not know what the states 'Rainy' and 'Sunny' would correspond to in the hand gesture scenario.

Edit: I am generating a sequence of numbers that will correspond to a certain gesture using the method mentioned above. I will then use that sequence to train a HMM and will then test that HMM using another set of recorded numbers similar to the training set. Here is an example of my scenario:

Recorded data of observed states (theta) during a gesture:

observations = [0,1,4,15,4,3,1,0,19,18,17,16,15,15,16,3,1,1,0,18...]


Recorded data of test gesture:

test = [0,2,4,15,4,2,1,0]


My goal is to create a model from the first set of observations (which will be much longer as the gesture will be recorded many times) and determine the likelyhood of the test gesture to be from said model.

Will I need to generate a hidden state to create an accurate model of the gesture or can I just use unsupervised training for a model?

If i do have to use supervised training, should I create the hidden states using quadrants (i.e. 0-90 degrees = quadrant1, 90-180 degrees = quadrant2...)?

• There might not be any simple interpretation of the states that you infer by constructing a HMM.
– D.W.
Aug 24 '20 at 16:41
• @D.W. I was thinking of two solutions to work around this problem. Fist would be to just use unsupervised training as I would not have to assign hidden states to the obseved states however I am not sure how that would affect the forwards-backward algorithm when determining the probability of a sequence being from that model. The second solution would be to set the hidden states as the quadrants those angles are in (e.g. 0-90 = quadrant 1, 90-180 = quadrant 2, etc..). Aug 25 '20 at 3:27
• Can you edit your question to state the context and what you're trying to achieve? Presumably you want to be able to interpret the states for some reason; what are you going to do with that, and what do you want to accomplish? There might be multiple ways to accomplish it; and we'll need to know that to give you good suggestions for how to solve it.
– D.W.
Aug 25 '20 at 4:57
• I have just updated my question to explain my goal. I hope it is enough to give a clearer understanding of the situation. Aug 25 '20 at 5:43
• Instead of appending "Edit:" and then more information, we prefer that you revise the question so that it reads well for someone who encounters it for the first time. See cs.meta.stackexchange.com/q/657/755
– D.W.
Aug 25 '20 at 7:46