It is possible to replace the fully-connected layers of a CNN with convolutional layers, making it fully convolutional. Fully-convolutional networks (FCNs) can be applied to inputs of various sizes, whereas a network involving fully-connected layers can't. Still, for the input size the network was designed for (e.g. 224x224 in the case of VGGNet) the mathematical operations performed are exactly the same in both. What are the drawbacks of a FCN? Can we just throw away all fully-connected layers and replace them with their convolutional counterpart?

  • 1
    $\begingroup$ How would you classify different labels then? If you have 4 outputs for example, not each of those outputs will have access to the same field of the previous convolutional layer; label 'cat' won't see everything label 'dog' sees. Interesting: people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf and cs231n.github.io/convolutional-networks/#fc $\endgroup$ Commented May 8, 2017 at 18:59
  • $\begingroup$ Thanks for the useful links, especially the second one is a good resource for FC->FCN conversion in general. Classification is usually done by averaging, this is outlined in the VGGNet paper: arxiv.org/pdf/1409.1556.pdf. That's the same as classifying crops and averaging the predictions, just with a single forward pass. If the input is "native" (224x224 for VGGNet) though, there will be only a single output vector. $\endgroup$
    – thertweck
    Commented May 8, 2017 at 19:25
  • $\begingroup$ @ThomasW Why should the output labels have different perceptive fields in FCNs? $\endgroup$ Commented May 9, 2017 at 9:33
  • $\begingroup$ @thertweck As far as I understand it, you can simply replace every fully connected layer by a convolutional layer in a CNN. No drawbacks as far as I know. $\endgroup$ Commented May 9, 2017 at 9:34
  • 1
    $\begingroup$ @ThomasW No, that's not what you would do. You would have two filters (one for each label). This means each filter can have different weights, but they have exactly the same receptive field. The receptive field means how much of the input can be used for the computation of the output. The computation itself can be totally different. $\endgroup$ Commented May 9, 2017 at 10:09

1 Answer 1


All-convolutional network is a great idea exactly because it has much more advantages than disadvantages. Most of modern convolutional networks are designed to use CONV for everything. If you are focused specifically on disadvantages, here're a few:

  • An FC to CONV layer replacement means great reduction in the number of parameters. It's cool to save the memory, but it's loss of flexibility nevertheless. In my experiments, CIFAR-10 classification accuracy dropped slightly after this single change, though it was certainly within one standard deviation. But I'm sure there are cases when this loss of flexibility makes a bigger impact.

  • A convolution is a significantly slower operation than, say maxpool, both forward and backward. If the network is pretty deep, each training step is going to take much longer.

  • The network is a bit too slow and complicated if you just want a good pre-trained model. That's why the researches still use AlexNet and VGGNet for experiments. They are simple (in terms of architecture), well-known and provide good performance - that's exactly needed.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.