1
$\begingroup$

I am a bit confused in the following definition: enter image description here

what does comma mean. I know I(X;Y) is the mutual information between X and Y but I am not sure how to interpret the I(X_1,X_2;Y)).

$\endgroup$
1
  • $\begingroup$ This is not a definition. It is a true mathematical statement. $\endgroup$ May 14, 2022 at 5:13

1 Answer 1

1
$\begingroup$

$I(X_1,X_2;Y)$ is $I(X;Y)$, where $X = (X_1,X_2)$.

For example, suppose that $X_1,Y$ are two independent uniformly random bits, and $X_2 = X_1 \oplus Y$. Then $I(X_1;Y) = 0$ while $I(X_1,X_2;Y) = 1$.

In more detail, the joint distribution of $(X_1,X_2,Y)$ is the uniform distribution over the vectors $(0,0,0),(0,1,1),(1,0,1),(1,1,0)$. The variables $X_1$ and $Y$ are independent, and so $I(X_1;Y) = 0$. In contrast, the variable $(X_1,X_2)$ (which ranges over $\{0,1\}^2$) determines $Y$, and so $H(Y|X_1,X_2) = 0$. Consequently, $$I(X_1,X_2;Y) = H(Y) - H(Y|X_1,X_2) = H(Y) = 1.$$

$\endgroup$
5
  • $\begingroup$ Just to clarify how to interpret (X_1, X_2)? Does it mean Cartesian product of X_1 and X_2? And does XOR mean Cartesian product? $\endgroup$
    – Sam
    May 15, 2022 at 3:11
  • $\begingroup$ It’s a two-dimensional random variable. XOR means XOR. $\endgroup$ May 15, 2022 at 3:56
  • $\begingroup$ 'XOR means XOR': that means all the variables here are binary variable right? $\endgroup$
    – Sam
    May 15, 2022 at 5:55
  • $\begingroup$ That's right, $X_1,X_2,Y$ are bits. $\endgroup$ May 15, 2022 at 6:21
  • $\begingroup$ Alright thanks for the clarification $\endgroup$
    – Sam
    May 15, 2022 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.