# How to calculate conditional entropy

I'm new to information theory and I am struggling to understand this problem. Let $$p(x,y)$$ given by:

How can we calculate $$H(X|Y)$$? I know $$H(X|Y)=H(X|Y=0)+H(X|Y=1)$$ but then I don't know how to go further.

$${\displaystyle \mathrm {H} (Y|X)\ =-\sum _{x\in {\mathcal {X}},y\in {\mathcal {Y}}}p(x,y)\log {\frac {p(x,y)}{p(x)}}}$$

Based on chain rule, we have:

\begin{aligned}\mathrm {H} (Y|X)&=\sum _{x\in {\mathcal {X}},y\in {\mathcal {Y}}}p(x,y)\log \left({\frac {p(x)}{p(x,y)}}\right)\\[4pt]&=-\sum _{x\in {\mathcal {X}},y\in {\mathcal {Y}}}p(x,y)\log(p(x,y))+\sum _{x\in {\mathcal {X}},y\in {\mathcal {Y}}}{p(x,y)\log(p(x))}\\[4pt]&=\mathrm {H} (X,Y)+\sum _{x\in {\mathcal {X}}}p(x)\log(p(x))\\[4pt]&=\mathrm {H} (X,Y)-\mathrm {H} (X).\end{aligned}

So for $$H(X|Y)$$ in your sample, we have: $$H(X|Y) = -( 0.25 \times \log(\frac{1}{2}) ) - ( 0.25 \times \log(\frac{1}{2}) ) -( 0 \times \log(\frac{0}{0.5}) ) -( 0.5 \times \log(\frac{0.5}{0.5}) )$$

• I know the formula for conditional entropy but I wanted to calculate it the other way. Or let's say we want to calculate H(X|Y=0). For this, what would be probability p(x=0|y=0)? – Niousha Oct 26 '19 at 11:18
• p(X=0 | Y=0) = P(X=0,Y=0) / P(Y=0) = (0.25)/(0.25) = 1 – aminrd Oct 26 '19 at 19:47