4
$\begingroup$

As I know, there are several upper bounds on the generalization error of hypothesis set with respect to sample complexity. For hypothesis space $V$, if the learning algorithm output the set of hypotheses that are consistent with the training data, i.e., $\sum\limits_{1 \leq i \leq n}{\frac{I[h(x_i) \not= y_i]}{n}} = 0$, the VC theory gives the bound on generalization error. If we consider the "margin" or confidence of classification, PAC Bayes theory gives tighter bounds on generalization error with margin incorporated.

My question is: what other properties or structure of the set of all possible hypotheses output by the learning algorithm matter for the bound on generalization errors? Name a few.

$\endgroup$
2
$\begingroup$

Rademacher complexity is also a tool used to bound generalization error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.