Does a deep feedforward neural net approximate a (single or multivariate)polynomial?

Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a multivariable polynomial? Here is what I am wondering about. Removing the affine linear transformation part of the neuron does chaining multiple non-linear function together approximate a polynomial? For example if we chain bunch of tang or sigmoid or even ReLU would they approximate a polynomial? for example $$tanh(W_2 * tanh(W_1 X+b_1) + b_2)$$

• What have you tried so far? Have you seen neuralnetworksanddeeplearning.com/chap4.html ? – Evil May 7 '18 at 21:18
• No I hadn't seen tis page. Thank you very much. – user650585 May 7 '18 at 21:26
• Please edit the question to define what you mean by "approximate a polynomial" more carefully. Do you mean, given a neural network, can you find a polynomial that approximates it? Or do you mean, given a polynomial, can you find a neural network that approximates it? See also en.wikipedia.org/wiki/Universal_approximation_theorem. It might also help to give us some context, e.g., about why you are asking, and what your motivation is. – D.W. May 7 '18 at 23:15