1
$\begingroup$

I am a bit confused in the following definition: enter image description here

what does comma mean. I know I(X;Y) is the mutual information between X and Y but I am not sure how to interpret the I(X_1,X_2;Y)).

$\endgroup$
1
  • $\begingroup$ This is not a definition. It is a true mathematical statement. $\endgroup$ May 14 at 5:13

1 Answer 1

1
$\begingroup$

$I(X_1,X_2;Y)$ is $I(X;Y)$, where $X = (X_1,X_2)$.

For example, suppose that $X_1,Y$ are two independent uniformly random bits, and $X_2 = X_1 \oplus Y$. Then $I(X_1;Y) = 0$ while $I(X_1,X_2;Y) = 1$.

In more detail, the joint distribution of $(X_1,X_2,Y)$ is the uniform distribution over the vectors $(0,0,0),(0,1,1),(1,0,1),(1,1,0)$. The variables $X_1$ and $Y$ are independent, and so $I(X_1;Y) = 0$. In contrast, the variable $(X_1,X_2)$ (which ranges over $\{0,1\}^2$) determines $Y$, and so $H(Y|X_1,X_2) = 0$. Consequently, $$I(X_1,X_2;Y) = H(Y) - H(Y|X_1,X_2) = H(Y) = 1.$$

$\endgroup$
5
  • $\begingroup$ Just to clarify how to interpret (X_1, X_2)? Does it mean Cartesian product of X_1 and X_2? And does XOR mean Cartesian product? $\endgroup$
    – Zzy1130
    2 days ago
  • $\begingroup$ It’s a two-dimensional random variable. XOR means XOR. $\endgroup$ 2 days ago
  • $\begingroup$ 'XOR means XOR': that means all the variables here are binary variable right? $\endgroup$
    – Zzy1130
    2 days ago
  • $\begingroup$ That's right, $X_1,X_2,Y$ are bits. $\endgroup$ 2 days ago
  • $\begingroup$ Alright thanks for the clarification $\endgroup$
    – Zzy1130
    2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.