# How to find a possibility of match in Naive Bayes Classifier?

My math level is very very poor so I can't get the statistics.
Can anyone explain in simple words? I.e. if I have frequency data:

Apple:
Colors:
Green = 3, Red = 2, Orange = 0;
Shape:
Sphere = 5, Ellipsoid = 0.
Orange:
Colors:
Green = 0, Red = 0, Orange = 5;
Shape:
Sphere = 5, Ellipsoid = 0.


I have many so fruits with frequencies, of course.
How can I get a possibility for object {Color: Red, Shape: Sphere} that it's an apple?

• It's a bit confusing to use orange as a fruit and color. May I suggest you change the orange fruit to another, like pear? – Reinstate Monica Jun 27 '18 at 1:41
• @Solomonoff'sSecret, yeah, it's a good idea – Шах Jun 27 '18 at 3:17
• In fact there is a shortcut here. I just noticed there are no red oranges. So we can get an answer to this particular question without doing any calculations. – Reinstate Monica Jun 27 '18 at 11:39

## 1 Answer

Let $A$ and $O$ be the events apple and orange (fruit). Let $R$ and $S$ be the events red and sphere.

By Bayes' law, $$\Pr[A | R \wedge S] = \frac{ \Pr[R \wedge S | A] \Pr[A] }{ \Pr[R \wedge S] };$$ $$\Pr[O | R \wedge S] = \frac{ \Pr[R \wedge S | O] \Pr[O] }{ \Pr[R \wedge S] }.$$ By total probability adding to 1, we also have $$\Pr[A | R \wedge S] + \Pr[O | R \wedge S] = 1$$ and therefore $$\Pr[R \wedge S | A] \Pr[A] + \Pr[R \wedge S | O] \Pr[O] = \Pr[R \wedge S]$$ so $$\Pr[A | R \wedge S] = \frac{\Pr[R \wedge S | A] \Pr[A] }{ \Pr[R \wedge S | A] \Pr[A] + \Pr[R \wedge S | O] \Pr[O]}.$$ By the Naive Bayes assumption of independence of features conditioned on class, $$\Pr[R \wedge S | A] = \Pr[R | A] \Pr[S | A];$$ $$\Pr[R \wedge S | O] = \Pr[R | O] \Pr[S | O].$$ Putting it together, we get $$\Pr[A | R \wedge S] = \frac{\Pr[R | A] \Pr[S | A] \Pr[A] }{ \Pr[R | A] \Pr[S | A] \Pr[A] + \Pr[R | O] \Pr[S | O] \Pr[O]}.$$

Now plug and chug. From the data $\Pr[R | A] = \frac{ \Pr[R \wedge A] }{ \Pr[A] } = \frac{2}{5}$, etc. Note: I am assuming the empirical distribution is the true distribution, an assumption your teacher probably expects you to make but not made in practice.

• Thank you for detail answer but does Pr[R&A] mean Pr(R)*Pr(A)? Sorry, I am really a noobie. – Шах Jun 27 '18 at 3:20
• No. It means the probability of $R$ and $A$ simultaneously, or the probability of a fruit being a red apple. This is the number of red apples divided by the total number of fruits, or $\frac{2}{10}$. – Reinstate Monica Jun 27 '18 at 11:38