3
$\begingroup$

Kolmogorov-Sinai entropy (KS) explains the mathematical concept behind KS entropy.

$$h ( T ) =\sup\limits_{\xi} \, h ( T , \xi )$$

defines the formula for KS where the left-hand side is nothing but the Shannon's entropy. In many papers I have seen that KS is defined as the supremum over source entropy. These conflicting views raises the following questions

  1. What is the difference between Shannon's entropy and source entropy (source entropy given by $\lim \frac{1}{n}\cdot h(T)$ where $n$ is the length of the word or sequence,unsure though).

  2. Would source entropy be the entropy of the raw signal before it has been quantized or should it be that of the quantized?

  3. Is 'n' the length of the data series or the number of quantization levels>

  4. Topological entropy (TS) explains topological entropy whose formula looks exactly as KS. So, what is the difference since KS is also known as metric entropy and not known as Topological entropy?
Improve this question
$\endgroup$
5
  • $\begingroup$ Crossposted on Signal Processing. Please don't do that. $\endgroup$
    – Raphael
    Commented Jul 25, 2012 at 6:31
  • 1
    $\begingroup$ From your first link: "The notion of Metric Entropy of dynamical system, also known as Measure-Theoretic Entropy, Kolmogorov Entropy, Kolmogorov-Sinai Entropy, or just KS entropy," Does that answer 3.? $\endgroup$
    – Raphael
    Commented Jul 25, 2012 at 6:36
  • $\begingroup$ Thank you Raphael,yes it does answer and do not know how I missed that!! $\endgroup$
    – user1214586
    Commented Jul 25, 2012 at 6:42
  • $\begingroup$ In the light of this and the fact that 4. is offtopic here, please edit your question to focus on 1. and 2. (and ideally on only one of them). $\endgroup$
    – Raphael
    Commented Jul 26, 2012 at 6:51
  • $\begingroup$ Here, $n$ is the length of the data series. $\endgroup$
    – Peter Shor
    Commented Oct 23, 2012 at 4:33

1 Answer 1

Reset to default
6
$\begingroup$

If you look at wikipedia you see that topological entropy is modeled after Kolmogorov-Sinai (KS) entropy, but it is defined without reference to a probability measure $\mu$ on the space.

For KS entropy, you need to use Shannon entropy: $-\sum_i\mu(X_i) \log \mu(X_i)$ for some partition of the space $\{X_i\}$, which can only be defined if you have some probability measure $\mu$ on your space, while topological entropy is defined without reference to a probability measure.

Since putting different probability measures on the space will give different KS entropies, while not affecting the topological entropy, these entropies are different. However, they are related by the theorem that the topological entropy is the supremum of the Kolmogorov-Sinai entropy over all probability measures $\mu$.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.