10
$\begingroup$

I am new to machine learning. I have read several papers where they have employed deep learning for various applications and have used the term "prior" in most of the model design cases, say prior in human body pose estimation. Can someone explain what does it actually means. I could only find the mathematical formulation of prior and posterior in the tutorials.

$\endgroup$
  • 5
    $\begingroup$ It's a mathematical concept so, yes, it's formulated mathematically. However, the Wikipedia page seems to give plenty of intuition. Did you check it? If so, could you say more about what you didn't understand and what you're looking for in an answer? $\endgroup$ – David Richerby Jun 11 '17 at 13:08
  • $\begingroup$ @David Richerby. Thank you for your response. Yes I had checked that wikipedia page and I could gather a vague idea that its something about knowledge or information about a variable. I had been reading papers on body pose estimation where there were mentions of body pose priors, body kinematic prior, modeling of priors over 3D human pose, learning priors, prior to estimate 3D human pose. I could not clearly figure out what the term "prior" actually means in this context. $\endgroup$ – Amy Jun 11 '17 at 14:58
  • $\begingroup$ jonathanweisberg.org/pdf/VarietiesvF.pdf $\endgroup$ – Andrew Aug 18 at 14:14
11
$\begingroup$

Put simply, and without any mathematical symbols, prior means initial beliefs about an event in terms of probability distribution. You then set up an experiment and get some data, and then "update" your belief (and hence the probability distribution) according to the outcome of the experiment, (the posteriori probability distribution).

Example: Assume we are given two coins. But we don't know which coin is fake. Coin 1 is unbiased (HEADS and TAILS have 50% probability), and Coin 2 is biased, say, we know it gives HEADS with probability 60%. Mathematically:

Given we have HEADS, the probability that it is Coin 1 is 0.4 $$p(H | Coin_1) = 0.4$$ and probability it is Coin 2 is 0.6 $$p(H| Coin_2) = 0.6$$

So, that is all what we know before we set up an experiment.

Now we are going to pick a coin toss it, and based on the information what we have (H or T) we are going to guess what coin we have chosen (Coin 1 or Coin 2).

Initially we assume $p(Coin_1) = p(Coin_2) = 0.5$ both coins have equal chances, because we have no information yet. This is our prior. It is a uniform distribution.

Now we take randomly one coin, toss it, and have a HEADS. At this moment everything happens. We compute posterior probability/distribution using Bayesian formula: $$p(Coin_1 | H) = \frac{p(H | Coin_1)p(Coin_1)}{p(H | Coin_1)p(Coin_1) + p(H | Coin_2)p(Coin_2)} = \frac{0.4\times0.5}{0.4\times0.5 + 0.6\times0.5} = 0.4$$

$$p(Coin_2 | H) = \frac{p(H | Coin_2)p(Coin_2)}{p(H | Coin_1)p(Coin_1) + p(H | Coin_2)p(Coin_2)} = \frac{0.6\times0.5}{0.4\times0.5 + 0.6\times0.5} = 0.6$$

So, initially we had $0.5$ probability for each coin, but now after the experiment our beliefs has changed, now we believe that the coin is Coin 1 with probability 0.4 and it is Coin 2 with the probability 0.6. This is our posterior distribution, Bernoulli distribution.

This is the basic principle of Bayesian inference and statistics used in Machine learning.

$\endgroup$
  • 2
    $\begingroup$ You need to fix the example above. That calculation shows that both coins are biased (First one with prob of Heads 40% and the second one with probability of heads 60%) In case the first one is biased It is still a Bernoulli distribution but with probabilities P(Coin1|H) =5/11 and P(Coin2|H)=6/11 $\endgroup$ – daniels_pa Jan 23 '18 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.