Object recognition - given an image, does it contain a particular 3D object of interest?

Question

I have in mind a particular 3D object. Given an image taken by a camera, I want to check whether that image contains an instance of my object.

For instance, let's say that the object is a bathroom sink. There are many kinds of bathroom sinks, but they tend to share some common elements (e.g., shape, size, color, function). There can also be significant variation in lighting and pose. Given an image, I want to know whether the image contains a bathroom sink.

How do I do that? What technique/algorithm would be appropriate? Is there research on this topic?

Of course, it is easy to use Google Images to obtain many example images that are known to contain a bathroom sink (or whatever the object I'm looking for might be), which could be used for training some sort of machine learning algorithm. This suggests to me that maybe some combination of computer vision plus machine learning might be a promising approach, but I'm not sure exactly what the specifics might look like.

Answer 1

If you have one or several images of a particular "sink" or object, then you can possibly use SIFT or SURF or other feature extraction/matching algorithms to match the objects up. You can investigate the OpenCV project (wikipedia, project); it includes many of these (possibly patented) algorithms. You can get an idea of how they work and what they can do by searching youtube for "opencv". If you specialize in matching previously known instances of a particular object, this might be appropriate. Additionally, there is much research specializing in facial recognition, so you can possibly find work on that.

Another project of interest: Sketch2Photo: Internet Image Montage. I don't know how they do it though, or if it can reliably identify anything; it seems to rely on some sort of recognition, and semantics search as well (you must name the objects in the sketch). You can read their paper (PDF)

If you want to recognize a generic "sink" concept or many multiple generic concepts, then it is bordering on strong(er) AI. There are lots of machine learning projects, and I am not sure which are useful for this or practical, but to demonstrate how cutting edge this is (and possibly not practical unless you spend a lot of time and research), Google (paper (PDF)) has made a giant GPU neural network that it successfully trained to recognize cats, presumably learning some sort of ability to conceptualize a more abstract "idea" of what a cat looks like. Related work by Yoshua Bengio; wired article; open source project.

This is "cutting edge", so again not practical, but just pointing out that you are trying to recognize generic things, and Google is working so hard just to recognize generic cats, you are in for a lot of work, or a restriction/trick/shortcut.

Answer 2

There are multiple approaches which can be taken into account. Some of them are mentioned above.

From what you want I think you should apply a combination of techniques to get good results.

In my opinion you should start with blob matching (affine invariant) -> followed by keypoint feature matching inside the blob to verify the result.

I think this approach should give reasonable results. Once you have some preliminary results then you can definitely move onto more complicated algorithms using machine learning etc.