# PCA with Bishop's book

I am reading PCA in Bishop's Pattern Recognition and Machine Learning pg. 562, here in Lagrange multiplier but I don't get it in the highlighted: I wonder is it $u_1$ or $u^T_1$?

Your help is appreciated! • The title you have chosen is not well suited to representing your question. Please take some time to improve it; we have collected some advice here. Thank you! – Raphael Apr 29 '18 at 15:38
• Don't use images as main content of your post. This makes your question impossible to search and inaccessible to the visually impaired; we don't like that. Please transcribe text and mathematics (note that you can use LaTeX) and don't forget to give proper attribution to your sources! – Raphael Apr 29 '18 at 15:38

## 1 Answer

You should take the gradient of the function, that is, the vector of partial derivatives with respect to all coordinates of $\mathbf{u}_1$, and equate it to the zero vector. Let us suppose that the individual coordinates of $\mathbf{u}_1$ are $u_1^1,\ldots,u_1^n$, so that (12.4) reads $$\sum_{i=1}^n \sum_{j=1}^n S_{ij} u_1^i u_1^j + \lambda_1 \left(1 - \sum_{i=1}^n (u_1^i)^2\right).$$ The derivative with respect to $u_1^t$ is $$\sum_{i=1}^n (S_{it} + S_{ti}) u_1^i - 2\lambda_1 u_1^t = 2 (\mathbf{S} \mathbf{u}_1)_t - 2\lambda_1 u_1^t,$$ since $S$ is symmetric. The gradient is thus $$\nabla (12.4) = 2\mathbf{S} \mathbf{u}_1 - 2\lambda_1\mathbf{u}_1.$$ Equating it to zero, we obtain the condition $\mathbf{S} \mathbf{u}_1 = \lambda_1 \mathbf{u}_1$, that is, $\mathbf{u}_1$ is an eigenvector of $\mathbf{S}$ corresponding to the eigenvalue $\lambda_1$.