How are Bayesian Nets, Hidden Markov Chains, Conditional Random Fields and Neural Nets related?

Question

I am having an AI exam in two weeks, and I am still figuring out certain concepts and ideas, related to Bayesian Nets, Hidden Markov Chains, Conditional Random Fields and Neural Nets (yes it is all going to be tested and yes we have a text (AI - A Modern Approach), but no, we did not cover everything in class).

I "know" a few things about the mathematical descriptions of all of those, but I know pretty much nothing about their usage or practical applications.

Here are my questions (and I apologize for my naivety):

1. What kind of machine learning algorithm classes do they belong to? Since they all need training, does it mean that they are all supervised learning algorithms?
2. They all have an underlying structure that allows a graph to represent them, where directed edges denote dependencies between states. The probability of being in a state is computed as a conditional probability from ancestors of the state. Does that sound about right?
3. In what kind of situation do you want to use which of the algorithms? Is it possible to some it up, or does it require subtle differentiation and expert-level knowledge?
4. Why do Neural Nets get special treatment? I heard of many classes teaching Neural Nets, but I have heard of no such thing for the other guys.

Answer 1

1. No, needing training has nothing to do with unsupervised/supervised. Supervised learners need labels, unsupervised learners do not (they do things like clustering, density estimation, dimensionality reduction). In fact you can use all of them for supervised and unsupervised learning, except maybe CRFs, since an unsupervised CRF would more likely be referred to as a markov random field (MRF) since there is no conditional per se. Someone might argue with me on this last point and I probably wouldn't argue back.

2. No, CRFs have undirected edges. ANNs can have undirected edges also (for example autoencodes with tied weigths). ANNs don't need have a probabilistic interpretation.

3. Sometimes it would be obvious. For example, if I'm doing Image Denoising I'm not going to use an HMM. Other times it will depend on things like what the practitioner has experience with, what works well in this domain, or a myriad of other factors.

4. Out of all the thing on your list ANNs are the only one that need not have a probabilistic interpretation. Furthermore, the techniques used for training ANNs normally are of a different flavor than what is used for other methods. That being said there are lots of classes that will focus on other types of models and likely not discuss ANNs, for example Probabilistic Graphical Models, Generalized Linear Models, etc.