1
$\begingroup$

In literature, Convolutional Neural Nets (CNNs) are presented as a special case of Deep Neural Nets (DNNs) (e.g., here). I do not understand how the convolutional layer can be implemented through a layer of neurons though. As far as I understand, a $n \times m$ kernel is used to calculate a single output feature over a combination of $n \cdot m$ input features in a certain stride. I see how this could be implemented as a layer in a Deep Neural Network, where basically $n \cdot m$ input features are the inputs for one output neuron, respectively. However, how would we ensure during training, that all the output neurons corresponding to the same kernel have the same weights for their inputs?

Is it not the case that CNNs are a generalization of DNNs rather than the other way around?

$\endgroup$

1 Answer 1

0
$\begingroup$

You're reading too much into the phrasing 'special case'. You already accurately understand the sense in which a CNN can be considered as a special case of a DNN; it is a DNN where many of the weights are repeated. Here people are probably referring to the model that is produced by the training procedure: every CNN could be expressed as a DNN with a particular pattern of repeated weights. This means that any function that can be computed by a CNN, can also be computed by a DNN. Don't worry about the rest, no one means anything more than that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.