# Solving analytic gradient of loss function for neural networks [closed]

Please note that I am talking in about theory rather than ''what someone would do in a real, practical situation''.

Given a multi-layer Perceptron with at least 1 hidden layer, and sigmoid (or other non-linearities), and a loss function, let it be a quadratic loss function such as $$L= ||\phi(X,W)-Y_{target} ||_{2}^2$$, where $$\phi(X,W)$$ is the output of the net, W are the weights, X the data matrix, $$Y_{target}$$ the correct labels. I think it does not make sense to solve for $$\nabla_W L = 0$$ to find a minimum ? I think it does not because, first of all, the function $$L$$, even though being ''the square of something'', is non-convex (possibly very complex with many ''hills'' and ''valleys'') and there can be many different values of $$W$$ giving a value of 0 for the gradient. Moreover, these points (W) where the gradient $$\nabla_W L = 0$$ could also be maxima rather than minima! Therefore, I think solving $$\nabla_W L = 0$$ would only make sense if $$L$$ is convex.