I have found worked examples for special cases of binary channel such as the binary symmetric channel and the Z-channel. However, I am interested in a more general type of binary channel $X \to Y$ in which I only know that $P(Y=1|X=0)=a$ and $P(Y=1|X=1)=c$, hence one hopes that $0<a<c<1$.

The mutual information is defined to be $$ I(X,Y):=\sum_{x,y \in \{0,1\}} P(X=x \wedge Y=y)\log_2 \left ( \frac{P(X=x \wedge Y=y)}{P(X=x)P(Y=y)} \right ). $$

Denoting $t=P(X=1)$, it follows that

$$ I(X,Y)=(1-c)(1-t) \log_2 \left ( \frac{1-a}{1-c} \right ) + a \frac{1-c}{1-a}(1-t) \log_2 \left ( \frac{a(1-a)}{(c-a)t+a(1-c)} \right ) +ct \log_2 \left (\frac{c(1-a)}{(c-a)t + a(1-c)} \right ). $$ The channel capacity is defined as the supremum of the mutual information over all possible distributions of $X$, that is, over all $t \in [0,1]$.

Is there, in this case, a formula for the channel capacity as a function of $a$ and $c$ ?

Thank you for your time.

  • $\begingroup$ Take a look at R. Silverman, "On binary channels and their cascades", IRE Trans. Info., 1955 $\endgroup$ – Ran G. Apr 12 '16 at 17:47
  • $\begingroup$ Thank you very much, Ran G. That looks very interesting. Thank you also D.W. - have done. $\endgroup$ – Simon Apr 12 '16 at 18:03
  • $\begingroup$ @RanG. Thank you very much again. I have now had a closer look at Silverman's paper, and also one of his references: "On the capacity of a discrete channel", Saburo Muroga, J. Phys. Soc. Jpn., Vol. 8, No. 4, 484-494, 1953, and these two papers are exactly what I need ! Would you care to make your comment an answer, so that I can "accept" it ? By the way my formula above is mistaken, and I will soon post the correct formula here, perhaps as another "answer". $\endgroup$ – Simon Apr 14 '16 at 13:31
  • 1
    $\begingroup$ Why don't you self answer your question, giving some details on the answer - it will be more useful for the readers than a link to a paper! $\endgroup$ – Ran G. Apr 14 '16 at 13:38
  • 1
    $\begingroup$ if you use Lagrange multiplier, you can obtain a formula for the channel capacity of general binary channel. $\endgroup$ – mohammad Mar 13 '17 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.