2
$\begingroup$

Assume I have some data in 2D. How to draw the first and second principal component in PCA method?

By referring to the image: the plot on the right is confusing, how I can detect the position of first PCA?

$\endgroup$
  • $\begingroup$ Do you ask about calculating PCA? Or when you have result where the vectors have origin or how to approximately draw vectors by looking at data or something else? $\endgroup$ – Evil Feb 27 '17 at 23:33
  • $\begingroup$ Exactly: how to approximately draw vectors by looking at data? I understand the first PCA should be along the direction of major data, and second one is perpendicular. But what if they are separated? What if there is no major direction of data? $\endgroup$ – Mo Farouk Feb 27 '17 at 23:40
  • 1
    $\begingroup$ stats.stackexchange.com/questions/2691/… If you imagine an ellipse that has to cover the data, it will probably be it. There are cases when this might fail, but drawing components by looking is not the way to go anyway. It may happen that data is ideal circle or ring, well then cover it as you like or keep axis aligned. There is one answer already, is there anything missing? (Sorry, I am still a bit confused about your objective). $\endgroup$ – Evil Feb 28 '17 at 17:06
  • $\begingroup$ Well my question (according to the plot on the right), why you have the two PCs rotated? Why we just don't have a plus sign, one line will separate the two circles of dots, and another line is perpendicular to it. What is the idea behind drawing the two lines as above? $\endgroup$ – Mo Farouk Feb 28 '17 at 23:58
2
$\begingroup$

In the case of 2d data, the first principal component will be aligned along the direction of major continuity, and the second will be perpendicular to that, aligned along the least continuous direction. Generally just the eigen vectors at the mean x / mean y.

Sort of like so (sorry for the poor drawing)

principal components

My favorite reference on pca is A TUTORIAL ON PRINCIPAL COMPONENT ANALYSIS by Jon Shlens: pdf His example is in figure 2.

To address your comment: principal components circles

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you a lot for this. But, what if the data are separated by each other, e.g. a circle shape on the right, and another circle shape on the left? $\endgroup$ – Mo Farouk Feb 27 '17 at 23:01
  • $\begingroup$ I've added a picture to address your comment as I understand it. With 2d data PCA is just least squares fitting for the major direction. (SVD is PCA is Least squares (essentially..)) $\endgroup$ – Matt D Feb 28 '17 at 15:24
  • $\begingroup$ What do you mean with "major direction"? I have added a picture which I am trying to understand how the two PCA components are displayed. I got the left one, but the right one is similar to your second plot. How I can -intuitively- draw the two PCA? $\endgroup$ – Mo Farouk Feb 28 '17 at 15:32
  • $\begingroup$ By Major direction I mean the first principal component. Begin by defining the first principal component which will be the line such that you minimize the squared distance from every data point to that line, then draw the second component perpendicular to that. Intuitively the first principal component is just the best fit line, the line such that you divide your data into two sets, one on the left one on the right with the 'least' distance to the line. $\endgroup$ – Matt D Feb 28 '17 at 15:48
2
$\begingroup$

If you draw a line in any direction, you can calculate the variance of the data points' projections on that line, i.e. how much they vary in that particular direction.

If you plot the curve where the distance from the origin to the curve in any direction is equal to the variance of your data in that direction, you will get an ellipse.

The major and minor axes of this ellipse are the first and second principal components.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I don't get which ellipse you mean? I am sorry your second statement isn't so clear. Could you please elaborate it? And according to the plot on the right in my image; why we just don't have a plus sign, one line will separate the two circles of dots, and another line is perpendicular to it? $\endgroup$ – Mo Farouk Mar 1 '17 at 10:35
  • $\begingroup$ Draw a plus sign on your right-hand plot. Look at the horizontal line. See how there are more dots above it on the left, and more dots below it on the right? That is why it's not a principal component. $\endgroup$ – Matt Timmermans Mar 1 '17 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.