2
$\begingroup$

I'm trying to teach myself a bit about machine learning, so one of the first things I did was implement a KNN classifier in ruby. My goal was to classify text product reviews into 8 classes:

  • books-positive
  • books-negative
  • kitchen-positive
  • kitchen-negative
  • dvd-positive
  • dvd-positive
  • electronics-positive
  • electronics-negative

I created features from my reviews by converting them into a set of bi_grams, removing stop words and then using a bag of words model. I calculate the closeness of feature by euclidean distance.

However the result wasn't very good, the max percentage of correct classifications I've gotten is about 28% which is little better than just guessing. Are there any one know of anymore improvements I can make to my classifier to make it better? Or any resources I can use to research from. I've included my source code and training/testing data below if anyone wants to take a look.

https://pastebin.com/x6DbQtts

https://pastebin.com/jDBsndPx

https://pastebin.com/SKikxz5G

Improve this question
$\endgroup$
  • $\begingroup$ Perhaps KNN just isn't good enough for this problem? $\endgroup$ – Yuval Filmus May 27 '17 at 20:24
  • $\begingroup$ You aren't describing your algorithms. We don't usually review code here. So perhaps this is not the perfect forum for this question. $\endgroup$ – Yuval Filmus May 27 '17 at 20:25
  • $\begingroup$ I'm not really asking for a code review, I know the code works as it's written. I was looking for more information on text classifying techniques. $\endgroup$ – user2320239 May 27 '17 at 20:43
  • 2
    $\begingroup$ I don't understand your features. What do you mean by bi_grams? Pairs of adjacent words? Or pairs of adjacent letters? If you're using bag-of-words where do bigrams come into it? It would help to describe your feature set more carefully. Are you using stemming? Tagging as part of speech? How many features do you have? How large is your training set? Where did you get the reviews? How long is the typical review? This isn't a place for reviewing your code or implementation; we focus on concepts, ideas, algorithms, science, etc. $\endgroup$ – D.W.♦ May 28 '17 at 3:29
  • $\begingroup$ One solution is playing around with features. Try using unigrams, TF-IDF vectors, try including stop words , use both unigrams and bigrams instead of only bigrams as the latter might be overfitting. Again it might be that data is bad. Try using any standard libraries to see how good they do. Also, you might try using dimensionality reduction. Try using L1 distance instead of Euclidean $\endgroup$ – iLoveCamelCase Jun 1 '17 at 0:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.