I am implementing a Naive Bayes algorithm for text categorization with Laplacian smoothing. The problem I am having is that the probability approaches zero because I am multiplying many small fractions. Therefore, the probability eventually yields zero. This is because there are several words within the documents and training sets.
Because of this, I am not able to categorize the texts. Is there a way I can get around this problem? Am I doing something wrong in my implementation?
The usual trick to avoid this underflow is to compute with logarithms, using the identity $$ \log \prod_{i=1}^n p_i = \sum_{i=1}^n \log p_i. $$ That is, instead of using probabilities, you use their logarithms. Instead of multiplying them, you add them.
Another approach, which is not so common, is to normalize the product manually. Instead of keeping just one floating point number $p$, you keep a floating point number $p_0 \in [1,2)$ (say) and a negative exponent $x$ such that $p = p_0 2^x$. After each operation you normalize the resulting number.
