Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Shannon's entropy [plog(1/p)] for an image is a probabilistic method for comparing two pixels or a group of pixels.Suppose an image with a matrix of 3x3 has pixel intensity values

1 1 2
2 3 3
4 4 5

and another image with 3x3 matrix has group of pixels having intensity values

5 5 6
6 7 7
8 8 9

Then shannon's entropy for the images would be the same.So in this case the entropy values would point out that the images are same though in actual they are different.So image matching using this technique doesn't help.On basis of supervised classification where I classify an image based on trained databases of shannon's entropy ,we use the concept of entropy to find similarity between two images.Is there any method or research paper where this entropy can be used or modified for image matching for the above case..?

share|cite|improve this question
Where did you get the idea that if the Shannon entropy for two things were the same, they had to be equal? This isn't the point of Shannon entropy at all. One point of Shannon entropy is to figure out how much you can compress data. – Peter Shor Oct 7 '12 at 21:38
But on basis of supervised classification where I classify an image based on trained databases of shannon's entropy ,we use the concept of entropy to find similarity between two images.Research papers have already been published wherein they have used such techniques to classify satellite images.I am also doing a research project on the same concept.Peter may be your knowledge on this is not good enough.You should read more on this before commenting. – Soumajyoti Oct 7 '12 at 21:51
If you had given some references, then maybe you could expect somebody to read more before commenting. As it was, I don't see how you could expect anybody to go off and research the facts that you left out of your question. – Peter Shor Oct 7 '12 at 22:00
But can I expect somebody who is very sure of on what he has commented ....maybe that is why I brought research into picture – Soumajyoti Oct 7 '12 at 22:04
@Soumajyoti Maybe those researchers try lots of functions and inspect their suitablity as fingerprints. Similar to MD5 or other hashes, these functions are not injective and they don't need to be: you can tell very different images apart quickly, and have to do more expensive things only rarely. – Raphael Oct 7 '12 at 22:46

Shannon's entropy works as a hashing or fingerprinting function here; they are not injective which means we have to be careful not to interpret too much into relations of values.

If $H(I_1) \neq H(I_2)$, then images $I_1$ and $I_2$ are certainly not the same.

If $H(I_1) = H(I_2)$ (or even $H(I_1) \approx H(I_2)$), however, we know nothing. The images might be similar, but the values can also be close because of non-injectivity.

We would like to have some smoothness, that is that two images are more different if $|H(I_1) - H(I_2)|$ is larger. Whether this is the case is certainly a quality criterion of such fingerprinting functions. Given your example, it does not seem as if entropy was a good fingerprint in this sense.

It would be confusing if any kind of entropy would be. Any rectangle of one color holds no information (entropy zero), yet two rectangles of different colors are as different as two images get.

share|cite|improve this answer

I think this an awesome idea. But you won't be able to use it for image matching anymore than you can hope that one image of "white noise" will match another. But it's a great idea for finding similar categories of images.

share|cite|improve this answer

One related approach would be to compute a histogram of the pixel intensities for each of the two images, then compare those two histograms. More typically, we'd compute a histogram of some measure of color (e.g., the hue), instead of pixel intensities. This is a crude measure that can sometimes nonetheless be helpful: e.g., for distinguishing a picture of one room from a picture of another (e.g., if their walls are painted different colors, or they have a different distribution of colors).

There are many ways to compare two histograms. You can think of each histogram as a distribution for some random variable, and then compare the two distributions using of a number of methods. For instance, you could compare them using KL-divergence. Another approach is to use any standard distance measure on vector spaces, e.g., the ${\cal L}_2$ measure (sum of squared differences). You'll often want to bin the values first, before computing the histogram.

See also

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.