In Recent Contributions to The Mathematical Theory of Communication (Weaver 1949), aka The Mathematics of Communication (Weaver 1949) (various copies exist online), and also published as Part I of The Mathematical Theory of Communication (Shannon and Weaver 1949), Weaver says:

We are now in a position to state the fundamental theorem, produced in this theory, for a noiseless channel transmitting discrete symbols. This theorem relates to a communication channel which has a capacity of C bits per second, accepting signals from a source of entropy (or information) of H bits per second. The theorem states that by devising proper coding procedures for the transmitter it is possible to transmit symbols over the channel at an average rate* which is nearly C/H, but which, no matter how clever the coding, can never be made to exceed C/H.

* We remember that the capacity C involves the idea of information transmitted per second, and is thus measured in bits per second. The entropy H here measures information per symbol, so that the ratio of C to H measures symbols per second.

I have not managed to find published errata for this text.

Am I correct to think the phrase "H bits per second" ought to read "H bits per symbol"? If not, why not?


1 Answer 1


Yes, your understanding is correct. The theorem is indeed talking about the entropy of the source, in terms of the amount of information in each symbol. Since entropy is measured in bits, the units are bits per symbol.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.