Using gradient descent in d dimensions to find a local minimum requires computing gradients, which is computationally much faster than Newton's method, because Newton's method requires computing both gradients and Hessians.
However, gradient descent generally requires many more iterations than Newton's method to converge within the same accuracy.
My question, then, is:
Assuming they both converge, in terms of the number of elementary floating-point operations, which is usually faster: Newton's method or gradient descent? Why?