Could someone, in plain english, explain the distinction between the fundamental matrix and the essential matrix in multi-view computer vision?

How are they different, and how can each be used in computing the 3D position of a point imaged from multiple views?


Both matrices relate corresponding points in two images. The difference is that in the case of the Fundamental matrix, the points are in pixel coordinates, while in the case of the Essential matrix, the points are in "normalized image coordinates". Normalized image coordinates have the origin at the optical center of the image, and the x and y coordinates are normalized by Fx and Fy respectively, so that they are dimensionless.

The two matrices are related as follows: E = K' * F * K, where K is the intrinsic matrix of the camera.

F has 7 degrees of freedom, while E has 5 degrees of freedom, because it takes the camera parameters into account. That's why there is an 8-point algorithm for computing the fundamental matrix and a 5-point algorithm for computing the essential matrix.

One way to get a 3D position from a pair of matching points from two images is to take the fundamental matrix, compute the essential matrix, and then to get the rotation and translation between the cameras from the essential matrix. This, of course, assumes that you know the intrinsics of your camera. Also, this would give you up-to-scale reconstruction, with the translation being a unit vector.

  • 2
    $\begingroup$ Thanks very much @Dima. Your explanation of point correspondences -> fundamental matrix -> essential matrix -> 3D position up-to-scale (under the assumption of known camera intrinsics) is very clear. Would it also be correct to say that in the case of unknown camera calibration, we could compute the fundamental matrix -> camera matrices -> the triangulated 3D position up-to a projective transformation? $\endgroup$
    – s-low
    Apr 14 '15 at 11:09
  • $\begingroup$ @s-low, that's correct. $\endgroup$
    – Dima
    Jul 30 '15 at 13:29

The other answer does not emphasize some important points.

  1. the fundamental matrix $F$ is a generalization of the essential matrix $E$
  2. the fundamental matrix does not assume the cameras to be calibrated (i.e. you don't know $M$ and $M'$, which are the homographies needed to project a 3d point onto the image plane of camera $C$ and $C'$, respectively)
  3. the essential matrix was discovered before the fundamental matrix
  4. in principle, to estimate the fundamental matrix you need more point-to-point correspondences than to estimate the essential matrix (because the fundamental matrix has more degrees of freedom, i.e. more parameters you need to find)

I want to add one thing to @Dima's answer, just so things are absolutely clear:

$E = (K')^TFK$ (note the transpose)

You can see how this also captures that the fundamental matrix takes image coordinates as inputs and not normalized 3D points. Let $x$, $x'$ be 2 normalized 3D points:

$(x')^T*E*x = (x')^T(K')^TFKx = (K'x')^TF(Kx)$

I can't comment yet because I don't have 50 reputation ...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.