I'd like to understand convolutional neural network.
Consider this quote from Stanford CS notes on Convolutional Neural Networks for Visual Recognition (CNNs/ConvNets):
Example Architecture: Overview. We will go into more details below, but a simple ConvNet for CIFAR-10 classification could have the architecture [INPUT - CONV - RELU - POOL - FC]. In more detail:
- INPUT [32×32×3] will hold the raw pixel values of the image, in this case an image of width 32, height 32, and with three color channels R,G,B.
- CONV layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume. This may result in volume such as [32×32×12] if we decided to use 12 filters.
- RELU layer will apply an elementwise activation function, such as the $\max(0,x)$ thresholding at zero. This leaves the size of the volume unchanged ([32×32×12]).
- POOL layer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as [16×16×12].
- FC (i.e. fully-connected) layer will compute the class scores, resulting in volume of size [1×1×10], where each of the 10 numbers correspond to a class score, such as among the 10 categories of CIFAR-10. As with ordinary Neural Networks and as the name implies, each neuron in this layer will be connected to all the numbers in the previous volume.
I don't understand why the result of applying convolution to the 32by32by3 input image is not of size 32 by 32 by 3 by 12? From what I understand, we there are 12 filters and we apply each filter to each of the 3 channels of the image to get 3 new images for each filter.
Suppose the result of the first convolution is really 32 by 32 by 12, I don't understand how you get them.
Also, what happen when we do a second convolution? Say we do a second full convolution with 5 filters, then the result of the second convolution is 32 by 32 by 3 by 12 by 5?
