I am working on a HMM tagger that should be initialized with some small data and then supposedly improved with Baum-Welch algorithm on the data.

However, the number of states is huge, almost $459^2$, so the training would take ages. For evaluation I already implemented Viterbi algorithm in which I prune and consider only the best $n$ options at that time for generating the next states.

Is something similar possible in Baum-Welch? I would like to do the forward pass pruned like in Viterbi, backwards only if forward probability $\alpha>0$.

Then we should estimate the new transition probability like

$$a_{ij} = \frac{\sum_t \alpha(i,t)p(j|i)p(o_{t+1}|j)\beta(j,t+1)}{\sum_t \alpha(i,t)\sum_t \beta(i,t)}$$

But since we do the pruning, some $a_{ij}$ would not be computed. What about those? Should we just not change them, zero them or do we have to normalize the distributions then?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.