1
$\begingroup$

I am working on a HMM tagger that should be initialized with some small data and then supposedly improved with Baum-Welch algorithm on the data.

However, the number of states is huge, almost $459^2$, so the training would take ages. For evaluation I already implemented Viterbi algorithm in which I prune and consider only the best $n$ options at that time for generating the next states.

Is something similar possible in Baum-Welch? I would like to do the forward pass pruned like in Viterbi, backwards only if forward probability $\alpha>0$.

Then we should estimate the new transition probability like

$$a_{ij} = \frac{\sum_t \alpha(i,t)p(j|i)p(o_{t+1}|j)\beta(j,t+1)}{\sum_t \alpha(i,t)\sum_t \beta(i,t)}$$

But since we do the pruning, some $a_{ij}$ would not be computed. What about those? Should we just not change them, zero them or do we have to normalize the distributions then?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.