0
$\begingroup$

There is a time series of say $100$ data points. I wish to assign symbols of $0, 1, 2$ for each unique data point. The issue is I have tried but got stuck since no matter I specify the symbols, the program just outputs probability of $1$'s and $0$'s. The following are the questions:

  1. How to find probability or correct my code so that it outputs probablities when number of symbols size > 2?
  2. How to calculate entropy annd mutual information for this case. I don't know although I have read Matlab's entropy calculation Mutual Information & Entropy but alas cannot follow how to apply in this case.
Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Entropy is, in your case

probs = [p_0 p_1 p_2];
logProb = log(probs);
entropy = -1 * dot(probs, logProb);

You'll have to let us know what two random variables you want to calculate the mutual information of, but with your library it looks like you just need to call mutualinfo(vec1,vec2).

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for this reply.However,I am unable to obtain the discretization for non binary case since the statement s=x(:,1) > 0.5; is a binary and would give 0,1 only.I wanted to know how is it possible to partion and assign more symbols and hence calculate the probability.The idea is along the lines of data mining where a data value is assigned a symbol and all such similar data values are assignedthat particular symbol. If the value of entropy for original time series is 0.90 and after symbolization it comes down to 0.52 then waht does this indicate? $\endgroup$
    – user1214586
    Commented Jul 11, 2012 at 18:53
  • $\begingroup$ What is meant by vec1,vec2?Also,the result of entropy going by your formula, I normalized the entropy to, entropy = entropy / log2(100) is approx 0.539 whereas if I use the program provided in the link gives result as 0.21!! What is the implication of this reduced value and why is there a difference? $\endgroup$
    – user1214586
    Commented Jul 12, 2012 at 4:04
  • $\begingroup$ @user: Entropy can be considered "uncertainty", so a reduction in entropy means you are more certain of the outcome after symbolization. (Which is almost guaranteed to happen, since you're combining symbols.) For questions about a specific library, Stack Overflow is probably a better site than this, but my guess is the lib is using a different base for the logarithm. $\endgroup$
    – Xodarap
    Commented Jul 12, 2012 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.