2
$\begingroup$

I am pretty new to Neural Networks and I have two questions about hidden layers:

1. What is the purpose of hidden layers? I was wondering this because obviously you can get every result you want with just one weight per input, as you're performing a multiplication.

2. How should I choose the number of hidden layers and how many cells should they have?

$\endgroup$
  • $\begingroup$ This is a rather broad question. I suggest taking a course on neural networks. $\endgroup$ – Yuval Filmus Sep 22 '17 at 15:46
  • $\begingroup$ Also, the usual rule is one question per post. I answered mainly your first question. $\endgroup$ – Yuval Filmus Sep 22 '17 at 15:53
2
$\begingroup$

It is actually not true that you can get any result that you want without any hidden layers. Consider for example a neural network with one input and one output. The only functions that such a network can be compute is threshold functions – whether the input is at most something or at least something (assuming the output is converted to Boolean).

Roughly speaking, the strength of neural networks as a computation model (rather than as a machine learning instrument) depends on two resources: size and depth. Every function can be computed (approximately) using only one hidden layer, but adding more hidden layers can dramatically reduce the size of the network needed to compute the function.

I think it is fair to say that at the moment, choosing the best network architecture involves some experience and some experimentation, since there is no good understanding of why neural networks work. Look up a paper on a similar task, and try similar parameters. There are known methods for estimating whether the network is too large or too small, and probably there are similar rules of thumb regarding depth.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.