# Chain rule for mutual information

I am a bit confused in the following definition:

what does comma mean. I know I(X;Y) is the mutual information between X and Y but I am not sure how to interpret the I(X_1,X_2;Y)).

• This is not a definition. It is a true mathematical statement. Commented May 14, 2022 at 5:13

$$I(X_1,X_2;Y)$$ is $$I(X;Y)$$, where $$X = (X_1,X_2)$$.

For example, suppose that $$X_1,Y$$ are two independent uniformly random bits, and $$X_2 = X_1 \oplus Y$$. Then $$I(X_1;Y) = 0$$ while $$I(X_1,X_2;Y) = 1$$.

In more detail, the joint distribution of $$(X_1,X_2,Y)$$ is the uniform distribution over the vectors $$(0,0,0),(0,1,1),(1,0,1),(1,1,0)$$. The variables $$X_1$$ and $$Y$$ are independent, and so $$I(X_1;Y) = 0$$. In contrast, the variable $$(X_1,X_2)$$ (which ranges over $$\{0,1\}^2$$) determines $$Y$$, and so $$H(Y|X_1,X_2) = 0$$. Consequently, $$I(X_1,X_2;Y) = H(Y) - H(Y|X_1,X_2) = H(Y) = 1.$$

• Just to clarify how to interpret (X_1, X_2)? Does it mean Cartesian product of X_1 and X_2? And does XOR mean Cartesian product?
– Sam
Commented May 15, 2022 at 3:11
• It’s a two-dimensional random variable. XOR means XOR. Commented May 15, 2022 at 3:56
• 'XOR means XOR': that means all the variables here are binary variable right?
– Sam
Commented May 15, 2022 at 5:55
• That's right, $X_1,X_2,Y$ are bits. Commented May 15, 2022 at 6:21
• Alright thanks for the clarification
– Sam
Commented May 15, 2022 at 9:28