I want to survey the state-of-the-art in image segmentation. Is there a paper on how to segment multiple objects in an image? The segmentation papers I read, normally identify one of the many objects in the image. This object is typically in the center of the frame, or most-foreground compared to other objects in the image. I'm looking for a way to identify all different objects in an image simultaneously. Is there any recent work on this?

  • 1
    $\begingroup$ I would like to help you, but segmentation is itself multiobject. It starts from blob detection, and based on that you have group of objects and background. Then each of them goes to classifier. What you gave described (one object) is the most of the time one-blob detector with one specialized classifier. What are you segmenting? Text? Patterns? Know templates? $\endgroup$
    – Evil
    Commented Aug 22, 2015 at 20:19
  • $\begingroup$ I actually just want to identify different regions in a photo that contain foreground objects. The objects may not belong to one class or even a known class. I've not yet read a paper that does this.. Can you point me to one? $\endgroup$
    – sanjeev mk
    Commented Aug 23, 2015 at 5:31

1 Answer 1


I think that you are interested in MFC - Multiple Foreground Cosegmentation.
MFC article

Awesome material: Articulated Motion and Deformable Objects : 5th International Conference, AMDO 2008, Port d'Andratx, Mallorca, Spain, July 9-11, 2008, Proceedings

In advance I warn you, these are not all out of the box working solutions, but with small changes all can do segmentation so if first link is what you have expected treat these below as starting point to work your own way.

Every "one object only in the foreground" can start iterative scheme: find first, cut, colour it with backgroundish color (presumably gray darker then mean luminance of the image), cut next.

When some classifiers are used, or patterns are known it is easier, but when background is complex and you want just regions there are oldschool ways: use Otsu, or modified K-Means clustering, try region growing, edge detection, make decision from colour or shadows.
In the end you will find yourself in situation that there are some components, now it is time to decide which is background (darker; connected with all edges of the image; bigger? Every of those could fail).

Image segmentation is based on some kind of connecting self similar regions, cuts them by edges even follows perspective lines if possible.

Foreground - Background separation is the most of the time asuming that foreground is smaller, better exposed. Very rarely there is perspective reconstruction and from one image it is virtually impossible (for street views we can detect lines and follow to horizon, but other types fail).

Below are additional links. If nothing helped write me a comment and describe more details.

Examples are connected in this paper but it works for separate objects: automatic foreground extraction

Foreground prediction

Must read:
Otsu image segmentation
Graph based

When you expect some objects: Blobs

Sequence of images:
Kalman filter

  • $\begingroup$ Thank you so much, MFC was that I was looking for, but didn't know the name to search by! $\endgroup$
    – sanjeev mk
    Commented Aug 24, 2015 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.