Let $X$ be an $m\times n$ ($m$: number of records, and $n$: number of attributes) dataset. When the number of attributes $n$ is large and the dataset $X$ is noisy, classification gets more complicated and the classification accuracy decreases. One way to over come this problem is to use linear transformation, i.e., perform classification on $Y=XR$, where $R$ is an $n\times p$ matrix, and $p\leq n$. I was wondering how linear transformation simplifies classification? and why classification accuracy increases if we do classification on the transformed data $Y$ when $X$ is noisy?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Multiplying by the $n * p$ matrix decreases the dimensionality of the data set. Think of this as projecting the highly dimensional space into a smaller dimensional space. For example, you could do principle component analysis and project it into a small space. This way things that are correlated together are projected into the same dimension and if one of those correlated dimensions is off because of noise, then it would be cancelled out by the values in the other correlated dimensions.
You have to choose your matrix $R$ in a way that essentially averages together things that reinforce each other and reduce noise. Principle component analysis is one way of choosing such a matrix.
-
$\begingroup$ What you might be facing is often called "The Curse of Dimensionality" en.wikipedia.org/wiki/Curse_of_dimensionality. It really depends on the classifier you are using. Any classifier that uses Euclidian distance or something similar (e.g. Cosine Similarity) will be impacted by the fact that as dimensionality increases, all samples tend to look more similar. See the section on distance functions at the link above. $\endgroup$– MattDCommented Apr 30, 2014 at 10:34
-
$\begingroup$ Are the anomalies outside the range of the 4 ellipsoidal clusters? If so, the transformation may be projecting several of these into similar places on a hyperplane. This would happen less if the signs in the matrix are mixed because the extreme values in the various dimensions would cancel each other out instead of combining together. $\endgroup$– MattDCommented May 1, 2014 at 3:54
-
$\begingroup$ No they are all in range of 0-1 $\endgroup$ Commented May 1, 2014 at 4:35