4
$\begingroup$

I'm having some trouble interpreting the camera matrix $K = \begin{bmatrix} f_x & s & x_0 \\ 0 & f_y & y_0 \\ 0 & 0 & 1 \end{bmatrix}$ after it multiplies some 3D vector.

Assume that $f_x = f_y = f$ and $s = 1$

$\begin{bmatrix} f & 1 & x_0 \\ 0 & f & y_0 \\ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} X \\ Y \\ Z \end{bmatrix} = \begin{bmatrix}Xf+Y+x_0Z \\ Yf + y_0Z \\ Z \end{bmatrix} \propto \begin{bmatrix} \frac{X}{Z}f + \frac{Y}{Z} + x_0 \\ \frac{Y}{Z}f + y_0 \\ 1\end{bmatrix}$

So the last vector is the imaged point in homogenous coordinates. However, I'm having trouble interpreting what each of the components are physically doing. An explanation would be much appreciated!

I should mention that I'm familiar with the constants, focal length, skew, and principle point. I just can't tell what they are doing in this configuration.

$\endgroup$
1
$\begingroup$

I will try to help you with this, even if incomplete.

Generally speaking (I don't know if you already know this or not), the camera matrix $K$ tells you how the pinhole camera projects each point in a 3D world, with some coordinates $(X,Y,Z)$ into an image-plane point, with coordinates $(x,y)$. A good reference is Wikipedia. Looking at the diagrams provided there, the 3D point has coordinates labelled $(X_1,X_2,X_3)$, which can be the same ones as our $(X,Y,Z)$ here, and the projected 2D point has coordinates $(y_1,y_2)$, which map to our $(x,y)$ here.

Let's delve into specific cases:

  • In the case the 3D point is $(0,0,Z)$, the point will be mapped to $(x_0,y_0)$, as the equation you provided shows. Why: in this case, the 3D point is along the line that is perpendicular to the image plane and which goes through the principal point of the camera, regardless of distance $Z$ to the camera.
  • For some fixed $(X,Y)$ in the 3D world, let's increase $Z$ endlessly. What happens is that, the bigger the distance $Z$ to the camera, the nearer to the camera's principal point $(x_0,y_0)$ the projected point will be. As an example, imagine a road photographed by a camera, such as this one. See how the 3D points that form the surface of the road converge to the principal point of the camera when their distance to the camera increases. This relates to the fact "parallel lines intersect themselves at infinity".

Now that we presented some specific cases, let's go a little backward in terms of why this matrix makes sense in the first place. It has to do with simple trigonometry around similar triangles. Please look at the Wikipedia article where it shows the similar triangles diagram and the corresponding mathematical formulation; you'll see the resemblance with $f\frac{X}{Z}$ and $f\frac{Y}{Z}$ elements you provided.

Now, you provided a camera matrix with high skew ($s=1$). In a pinhole camera model, you should have $s=0$, which means no skew at all, and so kill that $\frac{Y}{Z}$ parcel you have at your first entry, which only confuses you further. Skew in a camera is typically due to digitization effects and is very close to zero, not $s=1$ as you provided.

For further knowledge about this topic, I highly recommend the book Computer Vision: Algorithms and Applications by Richard Szeliski available freely as PDF at the same link.

I just came across this good article that also thoroughly explores the pinhole camera model. It includes an interactive applet at its bottom.

I hope this helped you get a first understanding of how those entries make any sense.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.