2
$\begingroup$

Given a nonlinear cost function $G(\vec{x})$ of many variables, does there exist a method that allows one to find successive local minima $\vec{x}_0, \vec{x}_1, \dots$ so that $\vec{x}_n$ is orthogonal to $\vec{x}_{n-1}, \vec{x}_{n-2}, \dots, \vec{x}_0$?

Sorry for the apparent lack of effort on my part, but the field of optimization is vast and I do not have a lot of experience. I have searched for variants of a nonlinear conjugate gradient method that have the above property, but haven't been able to find anything.

$\endgroup$
1
$\begingroup$

Suppose you have already found $x_0,\dots,x_{k-1}$ and now your goal is to find $x_k$. Let $\mathcal{R}$ denote the region of points that are orthogonal to $x_0,\dots,x_{k-1}$. Then your problem is to minimize $G(x_k)$ subject to the requirement $x_k \in \mathcal{R}$. This is a straightforward equality constraint in optimization, and can be handled in a number of ways. Probably the simplest is to use projected gradient descent, which is like gradient descent but in each step you project the point onto the hyperplane $\mathcal{R}$. Given how you have defined $\mathcal{R}$, the projection operation in this case is easy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.