1
$\begingroup$

I am training a model for ham and spam classification using LSTM. I am indicating the spams as 0, and the hams as 1. However, the dataset has much more hams than spams, so I tend to get an output very close to 1. That means the output is almost always above 0.5. So I have two questions about this :

Q1. Should I rate my answers based on the ratio between ham and spam in the dataset? (etc. If there are 4000 ham and 1000 spam, then should I count an output higher than 0.8 as ham and lower than 0.8 as spam?)

Q2. If this is not a legitimate way, do you have any solutions for my problem?

$\endgroup$

1 Answer 1

1
$\begingroup$

The usual starting point is that if the score is above 0.5, classify it as ham, otherwise as spam. If most emails are ham, then it makes sense that most emails give you a score above 0.5, so you have not said anything that indicates there is a problem.

This approach assumes that the proportion of ham vs spam in the training set is the same as the proportion at test time.

If that doesn't work, one standard approach is to choose a threshold, and everything with a score above the threshold is treated as ham, everything below as spam. A standard way to set a threshold is, after you've trained the LSTM, choose the optimal threshold based on the training set (i.e., that maximizes the accuracy on the training set, etc.), or on a validation set.

$\endgroup$
1
  • $\begingroup$ Thanks for the advice! I will find out more about choosing thresholds. $\endgroup$
    – TorchFire
    Feb 1, 2021 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.