I'm using machine-learning algorithms to solve binary classification problem (i.e. classification can be 'good' or 'bad'). I'm using
SVM based algorithms,
LibLinear in particular. When I get a classification result, sometimes, the probability estimations over the result are pretty low. For example, I get a classification 'good' with probability of 52% - those kind of results I rather throw away or maybe classify them as 'unknown'.
EDITED - by D.W.'s suggestion
Just to be more clear about it, my output is not only the classification 'good' or 'bad', I also get the confidence level (in %). For example, If I'm the weather guy, I'm reporting that tomorrow it will be raining, and I'm 52% positive at my forecast. In this case, I'm sure you won't take your umbrella when you leave home tomorrow, right? So in those cases where my model does not have a high confidence level I throw away this prediction and don't count it in my estimations.
Unfortunately, I can't find articles regarding thresholding the probability estimations...
Does anyone have an idea what is a normal threshold that I can set over the probability estimations? or at least can refer me to a few articles about it?