I am trying to implement the wikipedia example of the Gauss-Newton algorithm. Although it does work as intended, when changing the starting values, it takes significantly more iterations to converge and does not always land on the same values.

I am trying to understand how the example computes the initial starting values input to the algorithm.

I would appreciate if you kept in mind in your answers that I am new to algorithms and without a strong background in maths.

  • $\begingroup$ If there are several solutions, you won't always land at the same one, see for example the Newton fractal. Also, you are only guaranteed to converge quickly under some conditions, see Wikipedia. $\endgroup$ – Yuval Filmus Mar 18 '17 at 8:18
  • $\begingroup$ In general, it can be hard to come up with a good initial guess unless you use some algorithmic strategy like picking the best guess out of a set of $N$ random initial values. Using some strategy like this could give you a potentially good starting point, but there's still no guarantee that you'll end up at a globally optimal solution if your model is nonlinear in the parameters you are estimating. $\endgroup$ – spektr Mar 18 '17 at 17:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.