0
$\begingroup$

Suppose the use of linear regression. The result of the MSE is 120.5(mean squared error) for the train-set. Wev'e reached the minimum of training the data.

Is it possible that by applying Lasso(L1 form) we would get a lower MSE for the train? and for the test? is the same also holds for Ridge(L2 form)?

  • Lasso Train
  • Lasso Test
  • Ridge Train
  • Ridge Test
$\endgroup$

1 Answer 1

1
$\begingroup$

Regularization will always increase the MSE loss on the training set (training loss). Intuitively, it constraints/limits the set of regression lines you can use.

However, the MSE loss on the test set can be either increase (for the same reason) or decrease (if it has helped combat overfitting and helped bias the classifier towards a simpler hypothesis, and if simpler hypotheses are more likely to be correct).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.