It is possible to replace the fully-connected layers of a CNN with convolutional layers, making it fully convolutional. Fully-convolutional networks (FCNs) can be applied to inputs of various sizes, whereas a network involving fully-connected layers can't. Still, for the input size the network was designed for (e.g. 224x224 in the case of VGGNet) the mathematical operations performed are exactly the same in both. What are the drawbacks of a FCN? Can we just throw away all fully-connected layers and replace them with their convolutional counterpart?

  • 1
    $\begingroup$ How would you classify different labels then? If you have 4 outputs for example, not each of those outputs will have access to the same field of the previous convolutional layer; label 'cat' won't see everything label 'dog' sees. Interesting: people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf and cs231n.github.io/convolutional-networks/#fc $\endgroup$ – Thomas Wagenaar May 8 '17 at 18:59
  • $\begingroup$ Thanks for the useful links, especially the second one is a good resource for FC->FCN conversion in general. Classification is usually done by averaging, this is outlined in the VGGNet paper: arxiv.org/pdf/1409.1556.pdf. That's the same as classifying crops and averaging the predictions, just with a single forward pass. If the input is "native" (224x224 for VGGNet) though, there will be only a single output vector. $\endgroup$ – thertweck May 8 '17 at 19:25
  • $\begingroup$ @ThomasW Why should the output labels have different perceptive fields in FCNs? $\endgroup$ – Martin Thoma May 9 '17 at 9:33
  • $\begingroup$ @thertweck As far as I understand it, you can simply replace every fully connected layer by a convolutional layer in a CNN. No drawbacks as far as I know. $\endgroup$ – Martin Thoma May 9 '17 at 9:34
  • $\begingroup$ @MartinThoma well that's how how CNN's work, right? You slide a receptive field over the input, it then computes one output for each field with the same weights as the other receptive fields. If you would have 2 labels, then you would have two receptive fields. If you would have the same receptive fields for each lable, you would have the same output for each label (because each receptive field has the same computation) $\endgroup$ – Thomas Wagenaar May 9 '17 at 9:36

All-convolutional network is a great idea exactly because it has much more advantages than disadvantages. Most of modern convolutional networks are designed to use CONV for everything. If you are focused specifically on disadvantages, here're a few:

  • An FC to CONV layer replacement means great reduction in the number of parameters. It's cool to save the memory, but it's loss of flexibility nevertheless. In my experiments, CIFAR-10 classification accuracy dropped slightly after this single change, though it was certainly within one standard deviation. But I'm sure there are cases when this loss of flexibility makes a bigger impact.

  • A convolution is a significantly slower operation than, say maxpool, both forward and backward. If the network is pretty deep, each training step is going to take much longer.

  • The network is a bit too slow and complicated if you just want a good pre-trained model. That's why the researches still use AlexNet and VGGNet for experiments. They are simple (in terms of architecture), well-known and provide good performance - that's exactly needed.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.