2
$\begingroup$

I'm working on a problem to find the camera position in the world system when each picture was shot.

I currently have several pictures taken from different angles, and for each picture there is only a black square on a white paper.

like this

I have the actual size of the black square on the paper and the intrinsic matrix of the camera. And since the square is very easy to crop from each image, I can also obtain the perspective transformation matrix for each image. But I think to eventually find the camera position, the key is to get the extrinsic matrix of the camera. What I've researched is that we need to use a chessboard, but I wonder if I can continue my work without using that.

$\endgroup$
  • $\begingroup$ Hi @D.W. thank you for your share, but it seems like I have to write the wheel from bottom to top. Is there any libraries that I can make a use of? I pretty much prefer openCV but they don't support the no chessboard calibration. $\endgroup$ – AnnieBin Dec 11 '16 at 8:30
  • 1
    $\begingroup$ I don't know. Unfortunately library recommendations / software recommendations are off-topic here. $\endgroup$ – D.W. Dec 11 '16 at 16:11
  • $\begingroup$ @D.W. I see. But here is another question. Since my target, i.e. the black square is put in a stationary position in 3D world, and I move my camera to take different pictures from various angles, does that mean that the world coordinate systems changes every time when I move my camera? $\endgroup$ – AnnieBin Dec 11 '16 at 18:59
2
$\begingroup$

I think the answer is yes, it is possible to get the extrinsic matrix of hte camera with just a single square, though the error might be higher as there is less redundancy. Let's look at how much information we obtain:

  • How many unknowns? The extrinsic matrix has 5 real-valued parameters (the rotation matrix has 2 angles; the translation matrix has a 3D location). In other words, there are 5 degrees of freedom or 5 unknowns.

  • How many equations? We obtain 8 equations/constraints. I am assuming we know the 3D location of each corner of the square (in world coordinates). We also know the 2D location of each corner (in image coordinates). The camera matrix maps each 3D point to the corresponding 2D point. So, each point gives us 2 equations (from the known 2D image coordinates), and there are 4 corners, so we get 8 equations.

In short, we have 8 equations in 5 unknowns. That's enough that it should be possible to recover the extrinsic matrix. Of course, there's not a lot of redundancy here, so the error might be higher than if you used a chessboard. A chessboard would provide a lot more equations (each corner inside the chessboard provides an additional point with a known 3D->2D mapping).

How would you recover it? Follow the guidelines in https://ksimek.github.io/2015/03/29/QA-recovering-pose-of-calibrated-camera/. In particular, you could solve for the 5 parameters of the extrinsic matrix using some optimization routine, such as gradient descent or L-BFGS.

Given some value for those 5 parameters, you can compute the camera matrix and map the 3D world coordinates of each corner to its corresponding 2D image coordinates; then you can compute the $L_2$ error (sum of squared error) between the inferred 2D coordinates and the known 2D coordinates from the picture. This gives you an objective function that maps a value for the 5 parameters to an error term (which you want to minimize). Next, you can apply any off-the-shelf optimization library to try to minimize this objective function. This sounds pretty doable, with not too many lines of code to implement.

$\endgroup$
  • $\begingroup$ Wow that's a really explicit clarification. Thanks a lot! I will try to establish the wheel and see where I can reach. $\endgroup$ – AnnieBin Dec 12 '16 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.