1
$\begingroup$

I have an old exam question for my pattern recognition course which is stated as follows:

Consider a pool of Haar features, determine if these Haar features are independent or not. Is it a problem in boosting algorithms (e.g. AdaBoost)? Why (not)?

I have a really hard time figuring out an appropriate response to this question. One could argue that having dependent features is not desirable because then one of two classifiers does not really bring any new information or could possibly perform the same classification as the first one. The more independent features, the more independent, unique (weak) classifiers you have, but I don't know if this even makes sense.

Improve this question
$\endgroup$
3
  • $\begingroup$ Does AdaBoost work when some of the features are dependent, say, equal? How about other algorithms, like LDA and PCA? $\endgroup$
    – Yuval Filmus
    Jun 29, 2017 at 19:11
  • $\begingroup$ I have not yet heard of LDA. PCA, I believe, would not work because only having one feature/component (or all the same features) to represent that data will probably not account for enough variance to be feasible. When I pass this thought to AdaBoost, one could say that, again, having one feature is not enough: the resulting 'strong classifier' will still be weak, seen as though it is a linear combination of the same/only one weak classifier. I'm fairly new to feature extraction and the likes, so I'm far from certain about my answers. Please correct me if I'm (fundamentally) wrong. $\endgroup$
    – Robin Haveneers
    Jun 29, 2017 at 19:33
  • $\begingroup$ If I remember correctly, some of the linear methods will be "confused" if there are some linear dependencies. For example, LDA involves inverting some correlation matrix, and this is problematic if there are some approximate linear dependencies. On the other hand, for AdaBoost it shouldn't matter at all. $\endgroup$
    – Yuval Filmus
    Jun 29, 2017 at 20:07

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.