1
$\begingroup$

I am reading the book "An introduction to statistical learning with applications in R". I am reading Logistic Regression and I don't understand why when it's compared to linear regression model, the author said that "the problem with this approach: for balances close to zero we predict a negative probability of default; if we were to predict for very large balances, we would get values bigger than 1." as in the highlight. Could anyone please explain me how the probability can be smaller than 0 or larger than 1?

Your help is really appreciated!

Thank you enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You should just keep reading:

These predictions are not sensible, since of course the true probability of default, regardless of credit card balance, must fall between 0 and 1.

A probability cannot be smaller than 0 or larger than 1. The problem is that the model might predict such a "probability".

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, but according to the left figure above, the line goes under 0.0 on the y-axis. I wonder what does it mean? $\endgroup$
    – user2842390
    Mar 14, 2018 at 14:18
  • $\begingroup$ It means that the predicted probability is negative, which is of course meaningless since probabilities cannot be negative. This highlights the limits of logistic regression as a method to predict probabilities. Pragmatically, the results should be "clipped": negative probabilities should be clipped to 0, and probabilities larger than 1 should be clipped to 1. $\endgroup$
    – Yuval Filmus
    Mar 14, 2018 at 14:38
  • $\begingroup$ Thank you! Could you check whether I am thinking right or not: because linear regression with formula (4.1) depends on \beta_0 and \beta_1, they are influenced by these parameters, therefore \p(X) can be negative. That's a limitation of LR. Hence we need a "clip". Am I thinking correctly? $\endgroup$
    – user2842390
    Mar 14, 2018 at 14:50
  • $\begingroup$ Yes, that's the basic idea. $\endgroup$
    – Yuval Filmus
    Mar 14, 2018 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.