7
$\begingroup$

Shannon's entropy [plog(1/p)] for an image is a probabilistic method for comparing two pixels or a group of pixels.Suppose an image with a matrix of 3x3 has pixel intensity values

1 1 2
2 3 3
4 4 5

and another image with 3x3 matrix has group of pixels having intensity values

5 5 6
6 7 7
8 8 9

Then shannon's entropy for the images would be the same.So in this case the entropy values would point out that the images are same though in actual they are different.So image matching using this technique doesn't help.On basis of supervised classification where I classify an image based on trained databases of shannon's entropy ,we use the concept of entropy to find similarity between two images.Is there any method or research paper where this entropy can be used or modified for image matching for the above case..?

|cite|improve this question
$\endgroup$
  • 6
    $\begingroup$ Where did you get the idea that if the Shannon entropy for two things were the same, they had to be equal? This isn't the point of Shannon entropy at all. One point of Shannon entropy is to figure out how much you can compress data. $\endgroup$ – Peter Shor Oct 7 '12 at 21:38
  • $\begingroup$ But on basis of supervised classification where I classify an image based on trained databases of shannon's entropy ,we use the concept of entropy to find similarity between two images.Research papers have already been published wherein they have used such techniques to classify satellite images.I am also doing a research project on the same concept.Peter may be your knowledge on this is not good enough.You should read more on this before commenting. $\endgroup$ – Soumajyoti Oct 7 '12 at 21:51
  • 5
    $\begingroup$ If you had given some references, then maybe you could expect somebody to read more before commenting. As it was, I don't see how you could expect anybody to go off and research the facts that you left out of your question. $\endgroup$ – Peter Shor Oct 7 '12 at 22:00
  • 2
    $\begingroup$ @Soumajyoti Maybe those researchers try lots of functions and inspect their suitablity as fingerprints. Similar to MD5 or other hashes, these functions are not injective and they don't need to be: you can tell very different images apart quickly, and have to do more expensive things only rarely. $\endgroup$ – Raphael♦ Oct 7 '12 at 22:46
  • 1
    $\begingroup$ Shannon entropy can also detect the "skew" of an image in documents. I'm not sure it can be used to compare images. $\endgroup$ – mttdbrd Jan 17 '14 at 1:20
5
$\begingroup$

Shannon's entropy works as a hashing or fingerprinting function here; they are not injective which means we have to be careful not to interpret too much into relations of values.

If $H(I_1) \neq H(I_2)$, then images $I_1$ and $I_2$ are certainly not the same.

If $H(I_1) = H(I_2)$ (or even $H(I_1) \approx H(I_2)$), however, we know nothing. The images might be similar, but the values can also be close because of non-injectivity.

We would like to have some smoothness, that is that two images are more different if $|H(I_1) - H(I_2)|$ is larger. Whether this is the case is certainly a quality criterion of such fingerprinting functions. Given your example, it does not seem as if entropy was a good fingerprint in this sense.

It would be confusing if any kind of entropy would be. Any rectangle of one color holds no information (entropy zero), yet two rectangles of different colors are as different as two images get.

|cite|improve this answer
$\endgroup$
2
$\begingroup$

I think this an awesome idea. But you won't be able to use it for image matching anymore than you can hope that one image of "white noise" will match another. But it's a great idea for finding similar categories of images.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ It would be great to get more details about why white noise will match another. Please, provide some pieces of evidence. $\endgroup$ – Léo Léopold Hertz 준영 Aug 9 '16 at 13:19
2
$\begingroup$

One related approach would be to compute a histogram of the pixel intensities for each of the two images, then compare those two histograms. More typically, we'd compute a histogram of some measure of color (e.g., the hue), instead of pixel intensities. This is a crude measure that can sometimes nonetheless be helpful: e.g., for distinguishing a picture of one room from a picture of another (e.g., if their walls are painted different colors, or they have a different distribution of colors).

There are many ways to compare two histograms. You can think of each histogram as a distribution for some random variable, and then compare the two distributions using of a number of methods. For instance, you could compare them using KL-divergence. Another approach is to use any standard distance measure on vector spaces, e.g., the ${\cal L}_2$ measure (sum of squared differences). You'll often want to bin the values first, before computing the histogram.

See also

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Can you please provide any pieces of evidence about the histogram approach with some sample data. I am really interested about your use of L2 norm, which I think will not be sufficient in most cases. $\endgroup$ – Léo Léopold Hertz 준영 Aug 9 '16 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.