# Fundamental and essential matrices, value of intrinsic parameters

I observe a scene with two cameras, c1 and c2, that produce two images i1 and i2, respectively. What I ultimately want to do is to use information of image i1 and image2 simultaneously, e.g., for object detection.

As far as I understand it, I would not be required to find the intrinsic parameters of c1 and c2, also I do not need to know their relative position and rotation if I want to rectify i1 and i2. I can "simply" calculate F from a set of corresponding points - with no explicit relation in 3D of the points (like known distances). Which disadvantages would that have for me? I would not be able to do 3D reconstruction then, correct? But apart from that, there is no difference?

Or, asked differently, for rectification I do not need the intrinsic parameters. I only need them to do 3D reconstruction? What are the intrinsic parameters good for?

What are the intrinsic parameters good for? They potentially allow mapping objects in the images to locations in the real world (in 3D). For some applications, that can be very useful. For instance, consider pedestrian detection, where we want to detect pedestrians. Usually, it's not enough to detect the presence of a pedestrian: we also want to know exactly where the person is, and how far away they are.

Calculating alignment from a set of corresponding points is not an unreasonable approach to aligning two images. The primary disadvantage is that you need a way to identify a set of corresponding points. This typically requires either (a) explicit calibration or (b) heuristic methods.

Explicit calibration is a perfectly fine approach, if it is feasible for your application. However, for some applications we only have images post-facto and we don't have any way to do an explicit calibration, so it's not always an option.

Heuristic methods are another possibility. They try to find corresponding points, e.g., using Lucas-Kanade, optical flow, or other methods. As heuristics, they are not guaranteed to work. Sometimes they might fail to detect corresponding points, or output false detections. Another issue is that their effectiveness depends upon characteristics of the scene that you're capturing: they're more effective for some kinds of scenes than others. But this also could be a valid approach.

To be clear: 3D reconstruction requires two cameras (as mentioned in the question) or depth information; just one 2D camera plus its intrinsics are not enough to reconstruct where each object is in the real world, in 3D space.

• Hi! Mapping to locations in 3D: This only works when using two cameras or depth sensors I believe, maybe clarify that. Alignment: As far as I know explicit calibration needs at least two differently oriented chessboards. You could however just use one chessboard to obtain corresponding points and do image alignment. This I guess would be somewhere between (a) and (b). Aug 6, 2015 at 13:06
• @user1809923, I'm not sure I understand what you mean by "only works when using two cameras or depth sensors". Your question says "I observe a scene with two cameras", so I assumed we're already talking about a situation with two cameras.
– D.W.
Aug 6, 2015 at 17:04
• Yes, you are right about the two cameras. When reading only your answer however one could think that 3D reconstruction is also possible with one camera and its intrinsics. Aug 7, 2015 at 6:17
• @user1809923, OK, thanks! I've added a statement to the end of my answer to try to state that explicitly.
– D.W.
Aug 7, 2015 at 17:06