I understand that PCA takes a data set with input size, with output labels, and reduces the inputs to a set of principal components size r, where r < n.

My question is whether or not this can be applied beyond a simple classification problem where the outputs are not simply labels. For instance, in sheet metal forming of a part, the inputs could be temperature, initial thickness, applied force, etc. while the output is thickness.

Could we use PCA (or kernel PCA) to reduce the inputs to a smaller number while relating them to the output as a thickness and not a label?


Your understanding of PCA appears to be faulty. PCA does not look at labels or outputs. Instead, it takes a dataset, where each data point is a (high-dimensional) vector; and it maps that vector to a low-dimensional vector.

When you use PCA as a preprocessing stage in a pipeline for classification, typically you do PCA on the feature vectors: so the PCA ignores the labels and just transforms the feature vectors to a lower-dimensional vector. Then, some other classifier takes the lower-dimensional vector as input and tries to predict the label from it.

  • $\begingroup$ Thanks D.W. I understand that PCA is used as a preprocessing stage for classification problems. But is there a way to reduce input dimensionality for a neural network that maps inputs to discrete outputs rather than labels? It appears that because PCA doesn't consider the output/label, it would not be able to correctly reduce the dimensionality. $\endgroup$
    – Jeffery
    Mar 19 at 13:26
  • 1
    $\begingroup$ @Jeffery The fact that PCA is independent of the targets is precisely why you can use it for any kind of target. What you are really doing is choosing a different basis for the feature space, where the basis vectors can be ordered in terms of the variance they explain, which is why we can drop vectors with lower singular values without losing too much information. $\endgroup$
    – awillia91
    Mar 19 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.