3D Reconstruction from multiview calibrated camera

I have a calibrated camera whose intrinsics were calculated prior. Suppose I have multi images with the same camera. Using the first two views.I have the

1) initial reconstruction using 8 point algorithm to find the fundamental matrix and then I computed the essential matrix as E = transpose(K)FK

2) camera matrices P1 and P2 where the origin is set to that of camera P1.

My question is, how would I add more views?. How to calculate the projection matrix for all the views?

• One thing you can do is reconstruct a point cloud using the two camera matrices that you know, then solve for each camera matrix individually using the fact that you know where each point in your point cloud must project in each image. We have $P_i X_j = \lambda_j x_j$ for $j = 1, \ldots, N$, where $N$ is the number of points in the point cloud. It follows that $x_j \times P_i X_j = 0$ for $j = 1, \ldots, N$. You can use the DLT algorithm from Hartley and Zisserman to find $P_i$ from these equations. I wrote code that does this here: github.com/danielvoconnor/MultipleViewGeometry – littleO May 9 '20 at 8:57