I was wondering once you have recovered the fundamental matrix F such that xFx' = 0. If you try to recover translation and rotation from F, I am told that this is possible only up to scale? Why so?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This can be seen intuitively as follows:
The fundamental matrix $\mathtt{F}$ can be written in terms of the epipolar matrix $\mathtt{E}$, in which case the epipolar constraint equation becomes: \begin{align} \mathbf{x}^{\prime\top} \mathtt{F} \mathbf{x} &= 0\\ \mathbf{x}^{\prime\top} \mathtt{K}^{\prime-\top} \mathtt{E} \mathtt{K}^{-1} \mathbf{x} &= 0,\\ \end{align} where $\mathtt{K}, \mathtt{K'}$ are the intrinsic camera matrices of the two images corresponding to the points $\mathbf{x}, \mathbf{x'}$, respectively, and I have used $(\cdot)^{-\top} = (\cdot^\top)^{-1}$.
Now, we also know that the epipolar matrix has the form \begin{equation} \mathtt{E} = \mathbf{t}^\times \mathtt{R}, \end{equation} where $\mathbf{t}, \mathtt{R}$ are the relative translation vector and the relative rotation matrix, respectively, between the two images. The operator $(\cdot)^\times$ turns a $3-$vector into a $3\times 3$ skew-symmetric matrix. Since $\mathtt{E}$ is $3 \times 3$, $\mathbf{x}$ and $\mathbf{x'}$ are $3 \times 1$ vectors represented in what we call homogenous coordinates. These have an equivalence property of $\mathbf{x} = (x_1, x_2, x_3)^\top = (\lambda x_1, \lambda x_2, \lambda x_3)^\top = \lambda \mathbf{x}$ for any $\lambda \neq 0$. If you want to convert a homogeneous point into their more familiar 2D image plane representation, you simply take the first two components and divide them by the third one.
This means that, if we replace all of this in the epipolar constraint and multiply the translation $\mathbf{t}$ by an arbitrary scale factor, we get the following: \begin{align} \mathbf{x}^{\prime\top} \mathtt{K}^{\prime-\top} (\lambda \mathbf{t})^\times \mathtt{R} \mathtt{K}^{-1} \mathbf{x} &= \mathbf{x}^{\prime\top} \mathtt{K}^{\prime-\top} \mathbf{t}^\times \mathtt{R} \mathtt{K}^{-1} (\lambda \mathbf{x})\\ &= \mathbf{x}^{\prime\top} \mathtt{K}^{\prime-\top} \mathbf{t}^\times \mathtt{R} \mathtt{K}^{-1} \mathbf{x}\\ &= \mathbf{x}^{\prime\top} \mathtt{F} \mathbf{x} = 0, \end{align} which leads to the original constraint. This shows there is an overall scale ambiguity. More precisely, the epipolar matrix (and the fundamental matrix) are homogeneous quantities too. This can be further verified by taking the SVD decomposition of $\mathtt{E}$: if it is a valid essential matrix, two of its singular values are equal, hence $\mathtt{E}$ has 5 degrees of freedom and not 6.
