Calculate joint entropy of a Hamming Code over a Binary Symmetric Channel?

Question

I have a normal (7,4,3) Hamming Code over GF(2) and a parity check matrix for it (not posted, because I don't think it's involved).

I have a set X of 4 bit source vectors called x. They all have equal probability to be sent. So I would assume p(x) = 1/16. They are sent over a BSC with the bit error probability p=0.15.

Y is the decoded message set. It contains the same elements as X. However it can occur that a message x from X sent and decoded into y from Y would not be x = y (too many errors to correct/detect).

For each pair (x, y) from X x Y (Most likely Cartesian product?) we can derive a probability p(x,y) (the probability that a message x was sent, but received as y).

The task says "it is not hard to see we can calculate p(x,y) if we have the BSC bit error probability."

It is hard to see for me, though. I would have 16 vectors in X and 16 vectors in Y, that would give me a 256 row table. Then for each of that to calculate the probability of 1 scrambled bit, 2 scrambled bits, 3... and so forth?

I can write a program to do it, but this course is meant to be done by hand. Any help appreciated!

Answer 1

You can express $p(x,y)$ as a formula that depends on just the number of bits that differ between $x$ and $y$. Try to see if you can do that. That will provide a nice clean formula that doesn't require examining $16 \times 16$ cases by hand.