# Finding many different minima of nonlinear cost function

Given a nonlinear cost function $G(\vec{x})$ of many variables, does there exist a method that allows one to find successive local minima $\vec{x}_0, \vec{x}_1, \dots$ so that $\vec{x}_n$ is orthogonal to $\vec{x}_{n-1}, \vec{x}_{n-2}, \dots, \vec{x}_0$?

Sorry for the apparent lack of effort on my part, but the field of optimization is vast and I do not have a lot of experience. I have searched for variants of a nonlinear conjugate gradient method that have the above property, but haven't been able to find anything.

Suppose you have already found $x_0,\dots,x_{k-1}$ and now your goal is to find $x_k$. Let $\mathcal{R}$ denote the region of points that are orthogonal to $x_0,\dots,x_{k-1}$. Then your problem is to minimize $G(x_k)$ subject to the requirement $x_k \in \mathcal{R}$. This is a straightforward equality constraint in optimization, and can be handled in a number of ways. Probably the simplest is to use projected gradient descent, which is like gradient descent but in each step you project the point onto the hyperplane $\mathcal{R}$. Given how you have defined $\mathcal{R}$, the projection operation in this case is easy.