Draws from a Dirichlet process (DP) are discrete, and exhibit clustering behaviour.

Dirichlet Processes

The Dirichlet Process ($DP$) is a stochastic process used in Bayesian nonparametric modelling. It can be thought as a distribution over probability distributions: its samples are probability distributions whose marginals are a Dirichlet distribution (Ferguson 1973; Antoniak 1974).
Formally, given a distribution $H$ on a probability space $\Theta$, we say that $G$ is distributed according to a $DP$ with parameter $\alpha$ and base distribution $H$ (written $G \sim DP(\alpha, H)$) if, for any finite measurable partition $A_1, \dots A_N$ of $\Theta$, we have $G(A_1, \dots G(A_N)) \sim Dir(\alpha H(A_1)), \dots, \alpha H(A_N)$.
$H$ plays the role of the average of the $DP$, since $E[G(A)] = H(A)$ for every measurable set $A \subset \Theta$.
The draws of a $DP$ are discrete (Blackwell and MacQueen 1973) and exhibit a clustering behaviour: if $\Theta_1, \dots, \Theta_n$ are sampled from a $DP$, the posterior probability for $\Theta_{n+1}$ is $$P(\Theta_{n+1}|\Theta_1, \dots, \Theta_n) = \frac{ \alpha H + \sum\limits_{i=1}^{n}\delta_{\theta_i}}{\alpha + n} $$ where $\delta_{\theta_i}$, is the unit mass measure over $\Theta$. The probability of ending up with an already sampled value is proportional to the number of times it has already been sampled.

Suppose I draw $G_{1:5}$ distributions from a DP. Then the posterior probability for $G_6$ is given by (Blackwell and MacQueen, 1973):

$ \mathbb{P}(G_6 \mid G_{1:5} )= \frac{\alpha H + \sum^5_{i=1} \delta_{G_i} }{\alpha + 5} $.

Understanding this calculation is where I get confused. Suppose $G_{1:5}$ are univariate Gaussian $\mathcal{N}(\mu,\sigma^2)$ and I get five $(n=5)$ draws as so:

$G_1 = \mathcal{N}(0,1); \quad G_2 = \mathcal{N}(0,1); \quad G_3 = \mathcal{N}(1,1); \quad G_4 = \mathcal{N}(1,1); \quad G_5 = \mathcal{N}(2,1); \quad $

Then the posterior probability of $G_6$ is given as:

$\mathbb{P}(G_6 \mid G_{1:5} )= \frac{\alpha H + (1 + 1) + (1+1) +1}{\alpha +5} = \frac{\alpha H + 5}{\alpha +5}$

This is my understanding of the posterior but I do not think it is right. Help would be appreciated (I have misunderstood the delta function I believe, or at least the sum of it).

  • $\begingroup$ Certainly! Hang on. $\endgroup$ – Astrid May 31 '16 at 0:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.