# Naive Bayes' Classification and Using the Entire Vocabulary in the Denominator

I am working through the NLP notes for Naive Bayes' classification here: https://web.stanford.edu/~jurafsky/slp3/6.pdf

Below $c$ is the class of the observation and $w_i$ is the $i$th word of a text document.

In developing the maximum likelihood estimate for the conditional probability of the feature (word) $w_i$ given class $c$ the following expression is given for the conditional probability:

$$\hat{P}(w_i \mid c) = \frac{count(w_i,c)}{\sum_{w\in V} count(w,c)}$$

It is mentioned that $V$ consists of the union of all the word types in all classes, not just the words in one class $c$. The importance of this fact is stressed later on when add-one (Laplace) smoothing is used to avoid probability estimates that equal 0.

It is mentioned that:

It is crucial that the vocabulary $V$ consists of the union of all the word types in all of the classes, not just the words in one class $c$ (try to convince yourself that this is true).

I have thought for awhile on this aside given in the text, but it does not seem to make much sense. To me, wouldn't $count(w,c)$ always equal $0$ for any $w$ which is not contained in a document of class $c$? Why sum over all words in the vocabulary if they will just equal $0$?

## 1 Answer

If you're using Laplace smoothing, you'll replace the count with a "pseudocount", and that won't be zero. So, it does make a difference.