On the english Wikipedia it says about lazy learning systems:

Because the target function is approximated locally for each query to the system, lazy learning systems can simultaneously solve multiple problems […]

What does this mean? I can only guess what "approximated locally" is supposed to say but even then I have no idea how one is supposed to follow from the other.


Let's see:

Because the target function is approximated locally for each query to the system,

If i am going to predict the class (for example) of a brand new instance i compare it with the most similar in the train dataset. Imagine a 2D graphic with lots of points and i put the new point, so i compare with the other points locally, and the class of the nearest to it will help me say what is its class.

lazy learning systems can simultaneously solve multiple problems and deal successfully with changes in the problem domain.

Maybe, these problems are the ones created by the eager methods that try to simulate the hyperplanes (boudaries) in a function, in logic predicates, etc between the classes and these hyperplanes make lots of mistakes. And simple local comparisons with the similarity with the train dataset could do a better work.

  • $\begingroup$ I fail to see how that relates to the simultaneously-part. Could you clarify? $\endgroup$ – Shedeki Sep 12 '12 at 21:24
  • $\begingroup$ The simultaneously is just to say that the lay learning methods are powerful and can solve lots of problems, that i assume to be the ones created by the eager, (see answer) and the changes in the problem domain occur when the data changes, new data is added, and so the lazy learning deals with this changes, while the eager don't (because the models are already created). But, there are better concepts to lazy learning, because it is a research area. $\endgroup$ – Augusto Sep 13 '12 at 15:54

For the first part of your question about

"approximated locally"

Eager learning methods use all available training examples to build a classifier in advance that is later used for classification of all query instances.

Lazy learning, or instance based learning, is a learning method that delays the building of the classifier until a query is made to the system. The algorithm is trying to find similar examples from the training data to the query and uses them to build hypothesis for the classification. Examples that are used are localized near the query by some similarity.

For example if we have points in the plane that are classified with (+) sign and (-) sign, the eager learning will build single rule about how to classify any new point. In contrast, the lazy learning will aproximate only the nearest points signs to predict what will be the sign of the new point.

For the second part about

"simultaneously solve"

I guess that here the author means the online learning. This means that each new query is added to the traing data after its value is known.

Because of this, the eager learning must update its hypothesis after each new query and should process each query one at a time.

In contrast, the lazy learning can take many simultanious queries (if they are not locally close) because it uses only the examples, locally close to it.


After pondering the question for a while longer and talking to a professor of mine, I was able to gain enough insight to say that it is actually not that difficult to answer. Still, I will consider an example:

Given a number of points on plane which, in addition to a position, also have further characteristics, e.g. a color. If the problem is to estimate the color of a new, proposed point at a given position, it can be solved by looking at its $k$-nearest neighbors and determined through a majority vote, i.e. if most of its neighbors are red, assume that the new point will be red as well. If another characteristic is, say, size, this can be determined in the same way without the need to run the $k$-nearest neighbors algorithm again, thereby solving two problems simultaneously.

An interesting side-note: By searching the internet for the quote that I have taken from Wikipedia, one may turn up at least one book and one bachelor's thesis that use the exact same wording – apparently without citing any references or giving further explanations that might indicate any understanding on the respective author's side.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.