Gradient descent sometimes works better than simulated annealing and vice versa.
Are there conditions under which we can prove that, given perhaps a restriction on the set of allowed algorithms, one of these is optimal for solving an optimization problem?
I am particularly interested in those two examples, but in general interested in this question for metaheuristics for search and optimization problems.
(The no free lunch theorems show that (for certain problem statements) this is not possible in general, since for a uniform diatribution on problems, there usually isnt an optimal algorithm. However it might hold for special cases.)