1
$\begingroup$

I have a question about Linear Discriminant Analysis (LDA) for the purpose of Dimensionality Reduction.

So I understand for the algorithm to calculate for $k$ projection vector(s) you need to determine the eigenvector(s) that corresponds to the top $k$ eigenvalue(s). But can anyone explain what you do with those eigenvectors after you have calculated them to get a final output?

My guess is to multiply all of the eigenvectors (projection vectors) together and then multiply that with each point, $x$, in the original dataset to produce a new corresponding point $y$. Does this sound right?

$\endgroup$
1
$\begingroup$

Given normalized top $k$ eigenvectors $v_1,\ldots,v_k$, you send a point $x$ to the tuple $(\langle x,v_1 \rangle, \ldots, \langle x,v_k \rangle)$.

Alternatively, you put the eigenvectors as rows in a matrix $M$, and you map the column vector $x$ to $Mx$ (this is exactly the same thing as above).

Multiplying vectors is an operation that doesn't have much semantic sense here. (How would you multiply them anyway, and why?)

There are reasons why we apply this particular operation, and they have to do with the goal of LDA. If you want to know more, I suggest picking up a decent textbook that explains LDA not only via formulas and algorithms, but also as the solution to an optimization problem.

$\endgroup$
2
  • $\begingroup$ so if I understand correctly, would it be wrong to say something like the following y(i) = Mx(i), given M is a matrix of eigenvectors $\endgroup$ – Nadimprodutions May 3 '20 at 17:35
  • $\begingroup$ Right, $y = Mx$ and so $y_i = (Mx)_i$. $\endgroup$ – Yuval Filmus May 3 '20 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.