What type of HMM-GMM I need

Question

Context: I have 100 speech sentences that I asked my friend to speak. The vocabulary in the sentences are same but only the order of words are changed. My friend says that he spoke exactly what was asked for each sentence. But I don't know whether he spoke exactly that sentence or something else. What I have here is those 100 reference sentences against which I need to match his speech samples. As I want to do it by computer and not by listening manually therefore I am seeking your guidance.

Data: I have been able to segment the words in each speech sample of my friend. So I have 100 sentences each segmented into individual words with sequence of each word preserved for each sentence. I have extracted required features from each word (MFCC + Delta and Delta Delta).

What I am looking for: I need your guidance and help in informing how can I recognize these words with over 95% accuracy. As I read many papers and articles, GMM + HMM is the reasonably good way to do this. But I have a confusion, when I have already segmented every sentence, why should I try to match the entire speech sentence by transitioning state to state using HMM? I can simply match each word and see if the sequence is same with respect to the reference sentence word sequence. Is GMM + HMM the way to go for this? Can I use DTW or Neural Network or SVM to classify (matching or not matching) each word and get high accuracy?

Answer 1

Normally in speech-to-text we don't already have a perfect segmentation into words, which is why we often use the HMM to match the entire speech sentence. Also, this way the HMM can take into account probabilistic information on the distribution of word bigrams (and even trigrams). If you already have a segmentation and you are sure it is correct, then you can use the HMM to match just the word, if you like. But getting a reliably correct segmentation is tricky, given how people speak in practice, so you might want to check how accurate your segmentation is before relying upon it.

Either GMM + HMM or DTW would be a reasonable approach.

A very simple approach would be to download an existing speech-to-text tool and use it on those sentences, instead of trying to build something yourself.