Skip to main content
Tweeted twitter.com/StackCompSci/status/1064307399144677376
deleted 44 characters in body
Source Link
David Richerby
  • 82.2k
  • 26
  • 144
  • 238

I'm not super knowledgeable on this area so I apologize in advance if this is a silly question.

Given a string of values 'AAAAAAAABC'$AAAAAAAABC$, the Shannon Entropy in log base 2 $2$ comes to 0.922$0.922$. From what I understand, in base 2 $2$ the Shannon Entropy rounded up is the minimum number of bits in binary to represent a single one of the values.

Taken from the introduction on this wikipedia page:

https://en.wikipedia.org/wiki/Entropy_%28information_theory%29

So, how can 3three values be represented by one bit? A could$A$ could be 1 $1$, B could$B$ could be 0; $0$; but how could you represent C $C$?

Thank you in advance.

I'm not super knowledgeable on this area so I apologize in advance if this is a silly question.

Given a string of values 'AAAAAAAABC', the Shannon Entropy in log base 2 comes to 0.922. From what I understand, in base 2 the Shannon Entropy rounded up is the minimum number of bits in binary to represent a single one of the values.

Taken from the introduction on this wikipedia page:

https://en.wikipedia.org/wiki/Entropy_%28information_theory%29

So, how can 3 values be represented by one bit? A could be 1, B could be 0; but how could you represent C?

Thank you in advance.

Given a string of values $AAAAAAAABC$, the Shannon Entropy in log base $2$ comes to $0.922$. From what I understand, in base $2$ the Shannon Entropy rounded up is the minimum number of bits in binary to represent a single one of the values.

Taken from the introduction on this wikipedia page:

https://en.wikipedia.org/wiki/Entropy_%28information_theory%29

So, how can three values be represented by one bit? $A$ could be $1$, $B$ could be $0$; but how could you represent $C$?

Thank you in advance.

Source Link
Sean C
  • 143
  • 5

Shannon Entropy of 0.922, 3 Distinct Values

I'm not super knowledgeable on this area so I apologize in advance if this is a silly question.

Given a string of values 'AAAAAAAABC', the Shannon Entropy in log base 2 comes to 0.922. From what I understand, in base 2 the Shannon Entropy rounded up is the minimum number of bits in binary to represent a single one of the values.

Taken from the introduction on this wikipedia page:

https://en.wikipedia.org/wiki/Entropy_%28information_theory%29

So, how can 3 values be represented by one bit? A could be 1, B could be 0; but how could you represent C?

Thank you in advance.