1
$\begingroup$

I am running a Baum-Welch HMM algorithm (in R). The sequence vector contains a series of observations which have been gathered from a dataset where the data has 17 states.

I can successfully run the HMM algorithm and it converges without any problems when I set pseudo count to 1e-09, below that the algorithm fails.

My question relates to the Baum-Welch algorithm and local minima. When I run the algorithm and obtain the estimates for the emission matrix and transition matrix, I then use these as inputs to calculate the posterior probability of a sequence.

That is all fine except that when I redo all of the above (in a new R session) I get completely different posterior probability estimates. Is this because the BW algorithm is getting stuck in different local minima each time?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ I'm not sure there is enough information in this question to answer it. It seems like the only answer I can imagine is "It could be". I wonder if this might get a better answer over at Computational Science Stack Exchange -- if you like, we can migrate it for you (just flag the question for moderator attention and ask for it to be migrated), or you can delete it here and post it there.. but please don't double-post. $\endgroup$ – D.W.♦ Dec 13 '15 at 18:41
  • 1
    $\begingroup$ We asked the Computational Science folks and they don't think the question would be suitable there as it's mostly about the internals of R. Questions about the algorithm itself would be ok for Computational Science, if you don't assume people know the R language or the internals of its implementation. If you can express your question in source code, Stack Overflow would be better. $\endgroup$ – Gilles 'SO- stop being evil' Dec 14 '15 at 22:05
  • 1
    $\begingroup$ I'll leave this question open here until further notice since it isn't strictly off-topic and it isn't obviously unclear (this is far from my field), but I don't have high hopes that you'll get a good answer here, this is rather far from our main topics. Anything you can do to make your question clearer would help. $\endgroup$ – Gilles 'SO- stop being evil' Dec 14 '15 at 22:06
  • $\begingroup$ If you can post code and data that can reproduce the problem, it'll be much easier to figure this out.... $\endgroup$ – Dougal Dec 16 '15 at 20:52
2
$\begingroup$

It is impossible to locate the issue for sure without inspecting the implementation and results in more detail than is ontopic here. For instance, numeric algorithms always have potential issues related to the precision of the used number format, especially when dealing with (very small) probabilities.

That said, two notes.

  • Baum-Welch does only find local optima.
  • Insufficient training data can cause parameter estimations to vary wildly.

So I recommend you try more training data if possible. Otherwise, just run the algorithm many times and pick the best model (according to the likelihood of the training data according to the resp. model) you find.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.