implementing any nontrivial machine learning problem is difficult! but there are some basic different levels of difficulty (a continuum/hierarchy so to speak). the professor is contrasting the problem with another problem. consider these two problems. image classification is a common challenging problem at the edge of ML feasibility but of course not the only kind of classification problem; the example is based on it.
- given a set of images, find out distinct objects that are in the images. ie the result is a set of arbitrary choices about which category each images is in. no information is given about categories at all, not even how many categories there are.
- given a set of images and finite set of labels ("categories"), classify the images based on the labels. one has training data such that the images are correctly classified. classify the images in the test set by choosing one of the finite known labels.
clearly in option B there is more data to work with, it is less arbitrary, so the ML algorithm can potentially be more successful, and also there are infinite possible labels in option A.
this basic difference in challenge is why the recent Google results that found image classifications without labels in the training data is considered such a dramatic milestone/ breakthrough in ML where even the tabloid headlines were not necessarily overhyped!