Pattern classification: what goes into the sample?

Question

I am trying to compare the performance of a classification result with Bayes classifier, K-NN, Piece wise component analysis (PCA). I have doubts regarding the following (please excuse my lack of programming skills since I am a biologist and not a programmer thus finding the Matlab documentation hard to follow).

In the Matlab code

    Class = knnclassify(Sample, Training, Group, k)
    Group =  [1;2;3]   //where 1,2,3 represents Class A,B,C respectively.

What goes in the sample because my data is a 100 row 1 column for each of the classes? So Group 1 contains data like $[0.9;0.1;......n]$ where $n=100$. Would the sample be a vector containing random mixtures of the data points from the three classes? Same question for the Training matrix.

Answer 1

First a bit about the classifier: The knn classifier works by majority voting. It takes an input record, finds the k nearest labeled data points, looks at the class labels on each data point and assigns the current record the most common class label. For instance, if I use a 3-NN classifier and my three neighbors are of class [1 1 2], then I will select class 1 for this record.

That being said, "Training" should contain the feature vectors of your labeled dataset (probably 1 record per row). "Group" contains the class label for the corresponding record. So the $i^{th}$ value in Group is the class label associated with the $i^{th}$ record in "Training". "K" specifies the number of neighbors you'd like to look at when classifying new objects. Lastly, "Sample" will be the data point you want to classify.

Also, I've never heard of piece-wise component analysis, do you mean Principle Component Analysis? If this is the case, just note that PCA is a method commonly used for dimensionality reduction, so you'll still have to use a classifier to get labels for your data.

One more note: Matlab has an SVM classifier as well, its calls are similar to that of the Naive Bayes classifier for training and testing.