If you multiply two normal distribution PDFs with means $\mu_1$ and $\mu_2$ and variances $v_1$ and $v_2$, then according to this page, the new mean is

$$\mu = \frac{\mu_1 v_2 + \mu_2 v_1}{v_1 + v_2}$$

and the new variance is

$$v = \frac{v_1 v_2}{v_1 + v_2}.$$

However, if I use those expressions directly to combine several distributions sequentially, the floating point errors quickly pile up. Is there an algorithm for doing this stably?

  • $\begingroup$ Can you give a concrete example when "the floating point errors quick pile up"? $\endgroup$
    – John L.
    Dec 18 '18 at 18:29

While there is an entire subject that studies how to prevent or reduce floating point errors from piling up, it happens we have a shortcut here.

Let $\mathcal N(u, \sigma)=\frac{1}{\sqrt{2\pi\sigma^2} } e^{ -\frac{(x-\mu)^2}{2\sigma^2} }$ , the normal distribution with mean $u$ and standard derivation $\sigma$. Suppose we are given $n$ Gaussian normal distribution $\mathcal N_i(u_i, \sigma_i)$ . What is the product of $n$ of them? It turns out,

$$\prod_{i=1}^n \mathcal N(\mu_i,\sigma_i)= s\mathcal N (\mu,\sigma)$$ where $$ \frac1{\sigma^{2}} = \sum_{i=1}^n \frac1{\sigma_i^{2}}\,, \quad \frac\mu{\sigma^2} = \sum_{i=1}^n \frac{\mu_i}{\sigma_i^{2}} $$ and $$ s = (2\pi)^{\frac{1-n}2} \frac{\sigma}{\prod_{i=1}^n \sigma_i}\exp\left(\frac12 \left(\frac{\mu^2}{\sigma^2} - \sum_{i=1}^n \frac{\mu_i^2}{\sigma_i^2}\right) \right) $$

If you use the above formula to compute, floating point errors should not be a concern any more.

Please note the reference you used is misleading in the sense the product function is not a probability distribution function.

Exercise. Prove the above formula for the product of $n$ normal distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.