2
$\begingroup$

I am currently working on a project that involves taking a single color frame, performing image segmentation and then visualization of the scene.

I am currently at the stage of performing alignment between two objects - one in my database and one segmented from picture.

Picture data is processed: for each pixel in the color frame, I have depth data, so from a color frame I am capable of generating a (x,y,z) point for each pixel. Therefore, given my segmentation I am capable of generating a Point Cloud for each object.

So given model in my database and a point cloud ( which might not represent full object - example: I might just see a corner of a table, but I still need to align full table model), I would like to align object in my database with points in the cloud.

I have done some research and it seems that the method widely used is the Iterative Closest Point (ICP) algorithm. But I have also a different idea, and I would appreciate your evaluation, as I have some concerns regarding ICP.

METHOD 1:ICP

Given that my model and point cloud might not be aligned by default, I have decided to pick points that should be aligned in the result output (assigning corresponding points). So I pick for instance 4 points that should be aligned. However, the issue is obviously that my selection will not be precise and therefore two points sets do not differ just by rotation and translation, but also by small point misalignment.

I want to simplify calculations as much as possible to achieve some kind of convergence.

So I was wondering whether someone has any ideas on how to solve it, and whether you can actually use ICP given this misalignment..

METHOD 2: NON-ICP

Not use ICP, but rather use a simple one iteration calculation. Calculate centroid of selected points and pick one point from model and a corresponding point from the point cloud (obviously this is approximation, because I can't pick exact point), and create two vectors : "model centroid-to-point" vector and "point cloud centroid-to-point".

Then I could just calculate rotation and translation between these two vectors.

QUESTIONS:

1) Is there a way of solving the ICP issue?

2) Which method seems to be more sensible in terms of accuracy and possible errors?

Improve this question
$\endgroup$
4

1 Answer 1

Reset to default
1
$\begingroup$

I think you've misunderstood how to apply ICP. If you apply it correctly, ICP shouldn't have the problem you're worried about.

You're supposed to apply ICP to all the points. Don't just select 4 of the points and apply ICP to those points; apply it to the entire set of points. If you pick only 4 points, yes, ICP will have the problem you describe (it'll be sensitive to small errors in the point locations). That's why we don't do that. Instead, ICP applies least-squares regression to all the points. This eliminates the problem: any small errors in the point locations are averaged out, by the least-squares regression. So, try that. I suspect you'll be fine.

I don't expect your centroid method to work very well. It has several potential problems:

  • It matches the "centroid of visible points" (from the picture) to "centroid of all points" (from the database). However, there's no reason to expect those two centroids to be in the same place. Consider an asymmetrical object with parts that are not visible; the two centroids might be at different positions, and if so, this will introduce a systematic and potentially large error into your results.

  • Matching only one vector isn't enough to allow you to correct for both translation and rotation.

  • It requires you to know, for the selected point from the picture, which the corresponding point in the database is. However the whole point of this problem is that we don't know the correspondence. You could try to select the closest point in the database, but this might be wrong, since the two point-clouds haven't been registered into the same coordinate system, so this might go horribly awry.

For these reasons I don't think your method sounds very promising.

Last remark: The problem you're trying to deal with is called point set registration. I recommend you read up on standard methods for that problem before trying to invent a scheme of your own.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Right,well I think I need to clarify couple things: 1) It wouldn't be a centroid of all points, but rather the ones selected on a model. So I would pick 4 points on the model and 4 points on the point cloud that should be approximately matching in the final output. 2) ICP requires some correspondence as well. It uses closest distance, but in my case as some of my database models might not be centered, it would falsely assume that this is the actual correspondence. Moreover, it introduces another issue, which is convergence, as ICP might take ages converge, which is not desirable $\endgroup$
    – Johhny Bravo
    Commented Jan 12, 2017 at 19:56
  • 1
    $\begingroup$ @JohnnyBravo, 2) ICP repeatedly iterates the process until convergence, which is intended to address that issue. The first round might have some mismatches but we can expect/hope that those will decrease as we iterate. In any case, this is the best answer I can give; if you doubt it, you can always implement both and try them and see how well that works -- that is the ultimate test. $\endgroup$
    – D.W. ♦
    Commented Jan 12, 2017 at 20:15
  • $\begingroup$ Yeah, I belive that this is the case. However, ICP can just perform rigid registration ( translation + rotation), but it cannot perform scaling ( What happens in case my table model is bigger/smaller than the one on the image), so I assume the best solution would be to use SICP ( scaling iterative closest point algorithm) $\endgroup$
    – Johhny Bravo
    Commented Jan 13, 2017 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.