In the book multiple view geometry, the epipole is defined as follows:
The epipole is the point of intersection of the line joining the camera centres (the baseline) with the image plane. Equivalently, the epipole is the image in one view of the camera centre of the other view. It is also the vanishing point of the baseline (translation) direction.
The image of one camera center could easily not be in view of another camera center without the cameras being parallel. So, in this case, the line connecting the camera centers would not intersect at least one of the image planes. So, where is the epipole in this case?