I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden layers? Or is it another feature?
$\begingroup$
$\endgroup$
-
$\begingroup$ I don't have the expertise to answer but this link will help. colah.github.io/posts/2014-03-NN-Manifolds-Topology $\endgroup$ – Gareth A. Lloyd May 18 '15 at 19:13
$\begingroup$
$\endgroup$
A single-layer network is already nonlinear, but it's only a limited kind of nonlinearity.
Yes, the ability to have multiple layers and multiple hidden nodes is what allows multi-layer neural networks to express any function.
Let me give you an analogy that provides intuition but shouldn't be taken too seriously. A single NAND gate can compute only a single, simple function. However, when you consider circuits containing NAND gates, they can express any boolean function: the ability to have multiple layers of NAND gates, and multiple intermediate gates in the middle, allows you to express any boolean function. Something vaguely similar happens with multi-layer neural networks: each individual unit provides a limited amount of nonlinearity, like a NAND gate, and the ability to compose them is what lets you express more complex nonlinear functions.
$\begingroup$
$\endgroup$
Artificial neural networks are intrinsically supposed to handle the simulation and tracking of every function, if the sufficient number of the neurons in addition to enough number of the trials (to acquire the desired resolution) will be taken into account.
Of course, one might assert that above argument is actually realizable within the territory of the pattern recognition. In the other words, ANNs are able to use the imitations of the simulated functions within the decision making problems of pattern recognition stuffs.
Multilayer neural networks can increase their ability to cluster the hyper space of the network and approximate more complicated polynomial functions. As from a theoretical perspective, multitude of nonlinear functions could be approximated with the functions of polynomials. Hence, increment the number of hidden neurons and even extending the input stimulative signals and outputs will lead to better performance of the ANN regarding its functionality for aforementioned approximation.
