I have found worked examples for special cases of binary channel such as the binary symmetric channel and the Z-channel. However, I am interested in a more general type of binary channel $X \to Y$ in which I only know that $P(Y=1|X=0)=a$ and $P(Y=1|X=1)=c$, hence one hopes that $0<a<c<1$.

The mutual information is defined to be $$ I(X,Y):=\sum_{x,y \in \{0,1\}} P(X=x \wedge Y=y)\log_2 \left ( \frac{P(X=x \wedge Y=y)}{P(X=x)P(Y=y)} \right ). $$

Denoting $t=P(X=1)$, it follows that

$$ I(X,Y)=(1-c)(1-t) \log_2 \left ( \frac{1-a}{1-c} \right ) + a \frac{1-c}{1-a}(1-t) \log_2 \left ( \frac{a(1-a)}{(c-a)t+a(1-c)} \right ) +ct \log_2 \left (\frac{c(1-a)}{(c-a)t + a(1-c)} \right ). $$ The channel capacity is defined as the supremum of the mutual information over all possible distributions of $X$, that is, over all $t \in [0,1]$.

Is there, in this case, a formula for the channel capacity as a function of $a$ and $c$ ?

Thank you for your time.

  • $\begingroup$ Take a look at R. Silverman, "On binary channels and their cascades", IRE Trans. Info., 1955 $\endgroup$
    – Ran G.
    Apr 12, 2016 at 17:47
  • $\begingroup$ Thank you very much, Ran G. That looks very interesting. Thank you also D.W. - have done. $\endgroup$
    – Simon
    Apr 12, 2016 at 18:03
  • $\begingroup$ @RanG. Thank you very much again. I have now had a closer look at Silverman's paper, and also one of his references: "On the capacity of a discrete channel", Saburo Muroga, J. Phys. Soc. Jpn., Vol. 8, No. 4, 484-494, 1953, and these two papers are exactly what I need ! Would you care to make your comment an answer, so that I can "accept" it ? By the way my formula above is mistaken, and I will soon post the correct formula here, perhaps as another "answer". $\endgroup$
    – Simon
    Apr 14, 2016 at 13:31
  • 1
    $\begingroup$ Why don't you self answer your question, giving some details on the answer - it will be more useful for the readers than a link to a paper! $\endgroup$
    – Ran G.
    Apr 14, 2016 at 13:38
  • 1
    $\begingroup$ if you use Lagrange multiplier, you can obtain a formula for the channel capacity of general binary channel. $\endgroup$
    – mohammad
    Mar 13, 2017 at 13:36


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.