I'm working on a kNN classifier to classify whether a text is written by a man or a woman on the basis of the most frequent words. However, a kNN doesn't show which features were the most important in the classification. I got the suggestion to fit a logistic regression on my training data and labels to see which words were the most important in the classification task.

Now I have a list of the top 20 positive words and coefficients and top 20 words with negative coefficients. I do not know how to interpret these. I can see that the logistic regression has the classes F and M (don't know which one is 0 and which is 1), but is it possible to argue that the positive coefficients belong to the F class and the negative belong to the M class?

sample output

  • $\begingroup$ Write the equation for the classifier and see for yourself. $\endgroup$ May 29, 2017 at 10:31
  • $\begingroup$ Do you remove the stop words? $\endgroup$
    – Marlysson
    May 29, 2017 at 11:51
  • $\begingroup$ What research have you done? How to compute feature importance / feature rankings for logistic regression is well documented in lots of places (including in multiple places on Cross Validated). $\endgroup$
    – D.W.
    May 30, 2017 at 1:57

1 Answer 1


I don't have the rep to comment, but the answer to your question is "sort of." It would take me too long to answer this question completely and with the proper care, so i will point you to a good resource http://www.appstate.edu/~whiteheadjc/service/logit/intro.htm

There is a section midway through on interpreting your regression coefficients. In your specific case (as I imagine all of your independent variables only take on positive values), your interpretation is fine. If $b_1$ is the slope coefficient of an indicator variable for some word, then a 1 unit change in $b_1$ (i.e. if the word corresponding to $b_1$ is seen) corresponds to an $\exp{(b_{1})}$ change in the log odds of your model (e.g. the probability of the person being a boy is $\exp{(b_{1})}$ more likely). Hope this helps


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.