The universal approximation theorem states that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of Rn, under mild assumptions on the activation function.

How many neurons is needed? For a function in 3 variables and one output. And how many adjustable parameters does that give in the neural network?

  • 1
    $\begingroup$ The number of neurons presumably depends on the quality of approximation. $\endgroup$ – Yuval Filmus Jan 27 '19 at 20:17

There is not a particular number. You have to decide how much you want to approximate your function.

UAT, in informal language, means that neural network can approximate any given function f when given appropriate parameters (which are to be found experimentally; their learnability isn't a question here!).

If you have a function of 3 variables, then a neural network with 20 neurons may approximate it far better than one with the 10 neurons. It is possible that after a certain number of neurons, the difference between the outputs, i.e., (F - f) becomes very less.

I answer this question following Occam's razor, thus not mentioning any un-necessary terms. Please ask, if you have any additional or specific questions in it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.