Skip to main content

New answers tagged

0 votes

Expected Entropy of the Empirical Distribution

Not a closed form equation for the expected entropy, but an algorithm to compute it. Here $n$ is the number of outcomes with probabilities $[p_1, p_2, \dots, p_n]$, and $m$ is the number of samples ...
EnEm's user avatar
  • 301
0 votes

Chernoff bound when we only have upper bound of expectation

You can obtain this statement from your given form of Chernoff's bound using a coupling. Write $\mu - \mathbf{E}[X] = i + f$ where $i$ is a nonnegative integer and $ 0 \le f \le 1$. Now define the ...
lily's user avatar
  • 141
0 votes

Chernoff bound when we only have a lower bound of expecation

$$\Pr[X \le (1 - \delta)\alpha] \le \Pr[X \le (1 - \delta) \mathbf{E}[X]] \le \exp(-\delta^2 \mathbf{E}[X] / 2) \le \exp(-\delta^2 \alpha / 2)$$
lily's user avatar
  • 141
3 votes
Accepted

Information content of a real number

In some sense I agree that $\mathcal{U}_{[0,1]}$ has "infinite information" but we should be a little more careful formalizing it. As indeed, Shannon entropy is only defined for discrete ...
Benjamin Kuykendall's user avatar
1 vote

Information content of a real number

Yes. Its information content (entropy) is infinite.
D.W.'s user avatar
  • 162k

Top 50 recent answers are included