Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a multivariable polynomial? Here is what I am wondering about. Removing the affine linear transformation part of the neuron does chaining multiple non-linear function together approximate a polynomial? For example if we chain bunch of tang or sigmoid or even ReLU would they approximate a polynomial? for example $$tanh(W_2 * tanh(W_1 X+b_1) + b_2)$$

  • 3
    $\begingroup$ What have you tried so far? Have you seen neuralnetworksanddeeplearning.com/chap4.html ? $\endgroup$ – Evil May 7 '18 at 21:18
  • 2
    $\begingroup$ No I hadn't seen tis page. Thank you very much. $\endgroup$ – user650585 May 7 '18 at 21:26
  • $\begingroup$ Please edit the question to define what you mean by "approximate a polynomial" more carefully. Do you mean, given a neural network, can you find a polynomial that approximates it? Or do you mean, given a polynomial, can you find a neural network that approximates it? See also en.wikipedia.org/wiki/Universal_approximation_theorem. It might also help to give us some context, e.g., about why you are asking, and what your motivation is. $\endgroup$ – D.W. May 7 '18 at 23:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.