# Tag Info

1

Well, then your data is clearly wrong!! Bayes Theorem or not, you simply cannot have $$P\left(B\,|\,A \right) \times P\left(A\right) > P\left(B\right)$$ but this "impossibility" is exactly what happens in your last two rows!!

1

Here is a simple intuition. Suppose we consider a random value that takes the values 0 or 1 with equal probability. Then this value can be represented in a single bit. Moreover, its Shannon entropy is $1$. Therefore, it is natural to say that its entropy is 1 bit. More generally, consider a random value that is distributed uniformly on the set of $n$-...

1

The entropy of a random variable $X$ can be described in terms of prefix-free binary encodings. Let $T(X)$ be the minimal average codeword length of a binary prefix code for $X$, and let $X^{\otimes n}$ be the random variable corresponding to $n$ independent samples of $X$. Then $$\lim_{n\to\infty} \frac{T(X^{\otimes n})}{n}.$$ Shannon's source coding ...

2

The problem is your data! For example, the last row shows that $\mathbb{P}(A) = 1$ and $\mathbb{P}(B|A) = 0.8$. If $\mathbb{P}(A) = 1$ means $A$ is equivalent to all possiblities of event world. Hence, $\mathbb{P}(B|A)$ couldn't be anything except 1.

Top 50 recent answers are included