Convolutional Neural Network Example in Tensorflow

Question

I want to ask the dimension change in different convolution and max-pooling layer. I am referring to the example in TensorFlow tutorial:

http://tensorflow.org/tutorials/mnist/pros/index.html#deep-mnist-for-experts

The original image is a 28x28x1

The first convolutional layer:

1. apply convolution to a 5x5 patch with 32 features -> 24x24x32
2. apply max-pooling 2x2 -> 12x12x32

Second convolutional layer:

1. apply convolution to a 5x5 patch with 64 features -> 8x8x64
2. apply max-pooling 2x2 -> 4x4x64

But it said "Now that the image size has been reduced to 7x7" but my calculation seems to claim that it is a 4x4

Did I miss some concept? I am new to CNN so it may be a beginner question.

Thanks

Answer 1

Your calculation would be correct if the example were following the "usual" approach of having convolution chop off the edges.

Instead the example you pointed to says:

How do we handle the boundaries? What is our stride size? In this example, we're always going to choose the vanilla version. Our convolutions uses a stride of one and are zero padded so that the output is the same size as the input.

So they are:

1. zero-padding the 28x28x1 image to 32x32x1
2. applying 5x5x32 convolution to get 28x28x32
3. max-pooling down to 14x14x32
4. zero-padding the 14x14x32 to 18x18x32
5. applying 5x5x32x64 convolution to get 14x14x64
6. max-pooling down to 7x7x64.

They probably have an option to turn the zero padding off. In other infrastructures I've used zero padding is not the default. (In several of the infrastructures I've used zero-padding isn't even possible.)

Answer 2

For CNN basic knowledge, apply the first convolutional layer (5x5x32) to original image, the size of image become from 28x28x1 to 28x28x32 and to 24x24x32, both is right.

The different between 28x28x32 and 24x24x32 is the first result had padding and the second had not padding.

Why the tutorial select padding?
Because if we select NO padding, the result of features will become small, as you see from tutorial, after only one convolutional layer, the size become to 24x24x32. And if we have multiple convolutional layer, the size of feature map will become smaller and smaller, and the same time, the features will not useful anymore.

Detail from tensorflow tutorial: https://www.tensorflow.org/tutorials/layers

If filter width and height have the same value, you can instead specify a single integer for kernel_size—e.g., kernel_size=5.

The padding argument specifies one of two enumerated values (case-insensitive): valid (default value) or same. To specify that the output tensor should have the same width and height values as the input tensor, we set padding=same here, which instructs TensorFlow to add 0 values to the edges of the output tensor to preserve width and height of 28. (Without padding, a 5x5 convolution over a 28x28 tensor will produce a 24x24 tensor, as there are 24x24 locations to extract a 5x5 tile from a 28x28 grid.)

The step explain for example could see: