Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. It only takes a minute to sign up.
Sign up to join this community
Given that some shapes overlap in an image, are there any papers or articles or code which can infer the original shapes from the overlapping?
I am thinking to apply some machine learning to this since I can generate overlapping shapes using code and use a neural network to try to work out the original shapes. Perhaps another way is just heuristic rules.
I'd suggest you start with image segmentation, to identify each of the separate (potentially occluded) shapes in the image.
If there was no occlusion, the problem would then be easy. For each shape in the image, you could compare it each possible shape in your library of shapes and see if it's a match. If you don't have to deal with pose/rotation, only possible variation in the size and location of the shape, that will be easy to do: given a shape $S$ from the image and a shape $S'$ from your library of shapes, you first normalize their sizes, then check if they are a match using normalized cross-correlation or something similar. If you do have to deal with variation in pose/rotation as well, you could use something like SIFT to align $S$ to $S'$, then proceed from there.
However in your example you have to deal with occlusion, and it's occlusion that makes the problem more challenging. So, let's re-formulate the problem. You get a bunch of partial shapes $S_1,\dots,S_k$ from your image. For each $S_i$, you can also derive an occlusion mask $M_i$ that indicates which pixels were occluded by other shapes. Now you want a way to test whether $S_i,M_i$ is a match to some shape $S'$ in your library. How to do this?
If you only had to deal with variation in location but not rotation/size/pose, then this would be easy. It basically comes to checking that $S_i$ matches $S'$ at the non-occluded pixels (they don't have to match at the pixels indicated by $M_i$, but they do have to match elsewhere), i.e., whether $S_i \lor M_i = S' \lor M_i$, where $\lor$ denotes bitwise OR.
So the remaining challenge is how to deal with variation in size and/or rotation/pose. One approach that might work is to extract all the points on the boundary of $S_i$ (ignoring "boundaries" caused by occlusion) and all points on the boundary of $S'$, and then see if there's a way to make them line up. Note that we'll have to allow an incomplete match, because the occlusion may mean that the inferred boundary of $S_i$ is incomplete. One approach for finding a match might be to use RANSAC to estimate a homography between the two sets of points. (This is vaguely reminiscent of SIFT, but we're using the points on the boundary as the keypoints rather than using SIFT's methods to derive keypoints.)
If you have to deal with more complex images, more complex shapes, or other more complex situations, you could look at the literature on shape analysis and shape context.
