Say I had the choice of choosing one out of the following two optimization problems which I could use to solve my problem. Which choice is the fastest? How much of a trade-off would it be? Is the improvement in speed by many factors!?
Minimizing a convex function $L(X)$ in one matrix variable with orthogonality constraints over the matrix-essentially in my case this ends up to solving an eigen-decomposition.
Minimizing the same convex function $L(X)$ with linear constraints in $X$.
I know that 2.) should be faster. But what is the direction of work I need to do- to compare the improvement in speed-especially in terms of using the fastest available eigen solver for 1.)-what would be the corresponding fastest approach to solve 2.)?
