How is RANSAC used to estimate a homography given the descriptors of both images computed with SIFT?

Question

Let image $x$ be an original (non-warped) image. Let $I_1$ be $I_1'$ having suffered some projective transformation and possibly some noise. Suppose we have computed SIFT descriptors for both images.

Can you explain how RANSAC is concretely used to find the homography that aligns image $x$ and $y$?

More concretely, I understand we give to the RANSAC function the keypoints ($I_1$) and keypoints ($I_1'$).

My understanding (from various papers/tutorials I've read) is that it will take the positions of keypoints in $I_1$, which I denote as $(x,y)$, and the positions of keypoints $I_1'$, $(x', y')$, i.e. RANSAC($(x,y),(x',y')$), and then take randomly pairs of $(x',y')$ and $(x,y)$, and then compute the homography by trying to solve for each $x_i', y_i'$: $$[x'_i, y'_i]=H*[x_i, y_i]$$

But then how to estimate inliers? I guess by looking at the Euclidean distance between each pair of points found in the homography.

However, I don't really see the 'big picture'. I don't completely link this process of 'finding homography' and checking inliers etc.

It would be great if someone could summarize that in a clear way?

P.S. In general, tutorials say that they "find the homography" and then check for inliers - in some way - but they don't explain in a more pragmatic way what really happends.... w.r.t sift descriptors

Answer 1

How do you estimate inliers? You use RANSAC. That's what RANSAC does for you: it computes a homography and gives you a prediction for which pairs are inliers and which are outliers.

In SIFT, we first generate keypoints and the feature vector for each keypoint. Also, if $k$ is a keypoint in image $x$ and $k'$ is a keypoint in image $x'$, the feature vectors give a way to tell whether $k,k'$ are good matches. For each $k$, we look for a good match: a $k'$ from $x'$ that is significantly closer to $k$ than any other keypoint in $x'$. The set of good matches forms a set of possible inliers.

Then, we apply RANSAC. We randomly pick four good matches, compute a homography from these four, and test how good this homography is by checking how many of the good matches are consistent with the homography (if $k,k'$ are a good match, we're hoping that in most cases the homography will map $k$ to something near $k'$). Good matches that are consistent with the homography are called inliers (for this homography), and those that aren't are called outliers (for this homography). We count the number of outliers for this homography. Then, we repeat this 1000 times (picking a set of four good matches anew each time), in each iteration deriving a homography and counting the number of outliers associated with it. We keep the homography with the smallest number of outliers.

This is a simplification of the procedure but it will give you the gist of how SIFT works.

Answer 2

Ok. I think that I found out by myself. If I understand correctly we first need to do a 'direct matching' i.e. find some couple of matching keypoints (SIFT descriptors having lowest Euclidean distance, e.g. using the technique proposed by D.Lowe, which is to say we have a match if no other candidate keypoint has a lower or equal Euclidean distance as the best match)

then we give to findHomography function those pair of matching keypoints and we let him try to find a decent homography matrix.