Intuition for convolution in image processing

Question

I have read many documents about convolution in image processing, and most of them say about its formula, some additional parameters. No one explains the intuition and real meaning behind doing convolution on an image. For example, intuition of derivation on the graph is make it more linear for example.

I think a quick summary of the definition is: convolution is multiplied overlap square between image and kernel, after that sum again and put it into anchor. And this doesn't make any sense with me.

According to this article about convolution I cannot imagine why convolution can do some "unbelievable" things. For example, line and edge detection on the last page of this link. Just choose appropriate convolution kernel can make nice effects (detect line or detect edge).

Can anyone provide some intuition (doesn't need to have to be a neat proof) on how it can do that?

Answer 1

I think the simplest way to think of Convolution is as a method of changing a pixel's value to a new value based on the weight of nearby pixels.

It's easy to see why Box Blur:

_____________
|1/9|1/9|1/9|
|1/9|1/9|1/9|
|1/9|1/9|1/9|
-------------

works. Convolving this kernel is the same as going through every pixel of a photo and making the new value of the pixel the average of itself and the eight surrounding pixels.

If you get that, you can see why Gaussian Blur works:

_____________________
|.01|.04|.07|.04|.01|
|.04|.16|.26|.16|.04|
|.07|.26|.41|.26|.07|
|.04|.16|.26|.16|.04|
|.01|.04|.07|.04|.01|
---------------------

It's basically the same thing, except the averaging is weighted more strongly toward pixels that are closer. The function that defines how quickly the weights fall off as you move further away is the Gaussian Function, but you don't need to know the details of the function in order to use it for blurring.

The edge detection kernel in the linked article makes sense if you stare at it long enough too:

__________
|-1|-1|-1|
|-1|.8|-1|
|-1|-1|-1|
----------

It's basically saying that the value of any pixel starts at 8/9ths of it's original value. You then subtract the values of every pixel around it to arrive at your new pixel.

So if the value of a pixel is high and the value of the pixels around it are high too, they will cancel each other out. If the value of the pixel is low and all of the pixels around it are low as well, they will also cancel each other out. If the value of the pixel is high and the value of the pixels around it are low (as in a pixel on the edge of an object) the new pixel value will be high.

Answer 2

One way to think about convolution/crosscorrelation is as if you were searching for some signal in your data. The more the data looks like the kernel, the higher the resulting value will be. I actually take the reverse of the kernel, i.e. as in cross-correlation, but it is basically the same thing.

For example, let's say you are looking for a directional step in your 1d data.

The kernel could be

[-1 1]

and let's apply that to the data

[2 2 2 2 2 1 1 1 1 1]

The result will be

[0 0 0 0 0 1 0 0 0 0]

Which detects the location of the step. A larger step would give a larger value.

This works because when you multiply a pattern by the one appearing in the kernel you get a high value.

Edge detection (or any other pattern detection) works the same way, for example with the kernel

[-1 2 -1]

Extensions to higher dimensions can also be thought if this way.

This should give you intuition at least about some of the applications of convolution image processing.

Answer 3

If you think convolution is a little too hard to understand, I recommend you start searching about Mathematical Morphology applied to image processing, the big idea behind Mathematical Morphology is that you'll do a operation very close to the convolution, to "change" the morphology of the image, but retain the topology information, this way, you can make a image of a standing human, a skeleton, which is pretty much a stick human, try applying the erosion operation, then dilate operation, then go to the open/close operation, you'll start to understand what a mask applied to every pixel of image can do, and how they can be used in a big scenario to achieve great results(like finding a spot to start something using last erosion), once you understand Mathematical Morphology, convolution is a little bit harder, because its based on calculus, and defined after integrate, but still, alot of convolution are easy to understand, like median blur, gaussian blur, sharpen, edge detection, laplace, gradient, etc.