I have 2 images of a scene taken at one moment by two identical cameras (similar cameras intrinsic parameters) by to arbitrary locations and at two arbitrary orientations (different cameras poses). On the images n points correspondence p_i1={x_i1, y_i1, z_i1}↔p_i2={x_i2, y_i2, z_i2}, i=1,2,...,n are given. Let for simplicity z_i1=z_i2=f for all the points.
Is it possible at any number of corresponding points n to derive a transformation relation from one camera view to another so that for an additional point p_01 known only at one image to be able to locate it on the second image p_02?
My guess is (due to the fact that projection on a camera sensor plane is 3d-to-2d) it is not possible! And at most one may find a line on the 2nd image on which the additional point could be positioned.
I know that the task is solved with a homography matrix for two cameras having one center. I know it must be solvable for the case of all the points being coplanar.
But is it possible for arbitrary positioned cameras and arbitrary situated (non-coplanar in world frame) corresponding points?