Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm a pure mathematician interested in learning about information theory. Unfortunately, I'm about as pure as they come - my specialty is mathematical logic, and I have absolutely no experience with programming or anything of a computational nature. I tried looking at Shannon's original paper, but the profusion of numbers intimidated me (I'm a mathematician after all!) When he starts discussing the particulars of coding specific alphabets, my brain begins to asking whether this attention to detail is really necessary and soon zones out... It took Church's thesis for granted some time ago. (Pathetic, I know.)

I'm wondering if there's some introduction to information theory written with people like me in mind. Something which uses the language of measure theory and recursion theory, and works in the greatest generality. Perhaps an expert in the area has written some notes in this direction as an idle folly?

I realize that this might not be the place to ask this question, and apologize if this is the case.

share|cite|improve this question
Here is the reference as I mentioned earlier. – Nicholas Mancuso Jan 30 '13 at 6:43
Sometimes very pure mathematicians need to get their hands dirty. It helps them develop their immune systema. Out of curiosity, what kind of logician has "no experience with anything of a computational nature"? Isn't computability theory "of a computational nature"? – Andrej Bauer Jan 30 '13 at 7:41
You might consider approaching Information Theory via Kolmogorov Complexity. It's a more general field, more concerned with a platonic world of Turing Machines and such, and less with singular practical applications. Information Theory then follows as a kind of special case of Kolmogorov Complexity. It certainly uses the languages of recursion theory and measure theory. – Peter Jan 30 '13 at 12:40
If you want a pure mathematician's viewpoint, why is this question not on Mathematics? – Raphael Jan 30 '13 at 13:13
I call troll on this one. This paper contains no more numbers than do any other "pure mathematics" paper. It has something like two examples which involves number which hardly should be intimidating. And alphabets is an extremely natural abstraction of sets of natural numbers that are used in recursion theory. Is it possible to ask you what background you really have? Because a little mathematical maturity is all that's needed to read Shannon's paper. – Pål GD Jan 30 '13 at 19:47

I think you are looking for the topic of "Algorithmic Information Theory" (AIT). This area is the collision of information theory and computer science. For example "Information and Randomness: An Algorithmic Perspective" written by cristian calude contains interesting subjects in this area. You can find more in the AIT website.

share|cite|improve this answer
Please add more information than just a link. See here for a guideline as to what constitutes a good answer on this platform. – Raphael Jan 30 '13 at 20:40

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.