Epipolar geometry

Epipolar geometry refers to the geometry of stereo vision. When two cameras view a 3D scene from two distinct positions, there are a number of geometric relations between the 3D points and their projections onto the 2D images that lead to constraints between the image points. These relations are derived based on the assumption that the cameras can be approximated by the pinhole camera model.

The figure below depicts two pinhole cameras looking at point X. In real cameras, the image plane is actually behind the focal point, and produces a rotated image. Here, however, the projection problem is simplified by placing a "virtual image plane" in front of the focal point of each camera to produce an unrotated image. O_L and O_R represent the focal points of the two cameras. X represents the point of interest in both cameras. Points x_L and x_R are the projections of point X onto the image planes.

Each camera captures a 2D image of the 3D world. This conversion from 3D to 2D is referred to as a perspective projection and is described by the pinhole camera model. It is common to model this projection operation by rays that emanate from the camera, passing through its focal point. Note that each emanating ray corresponds to a single point in the image.

Epipole

Since the two focal points of the cameras are distinct, each focal point projects onto a distinct point into the other camera's image plane. These two image points are denoted by E_L and E_R and are called "epipoles". Both epipoles E_L and E_R and both focal points O_L and O_R lie on a single line.

Epipolar line

The line O_L-X is seen by the left camera as a point because it is directly in line with that camera's focal point. However, the right camera sees this line as a line in its image plane. That line (E_R-x_R) in the right camera is called an "epipolar line". Symmetrically, the line O_R-X seen by the right camera as a point is seen as epipolar line E_L-x_Lby the left camera.

Epipolar plane

As an alternative visualization, consider the points X, O_L & O_R that form a plane called the "epipolar plane". The epipolar plane intersects each camera's image plane where it forms lines - the epipolar lines. All epipolar lines intersect the epipole regardless of where X is located.

Epipolar constraint and triangulation

If the relative translation and rotation of the two cameras is known, the corresponding epipolar geometry leads to two important observations

* If the projection point x_L is known, then the epipolar line E_R - x_R is known and the point X projects into the right image, on a point x_R which must lie on this particular epipolar line. This means that for each point observed in one image the same point must be observed in the other image on a known epipolar line. This provides an "epipolar constraint" which corresponding image points must satisfy and it means that it is possible to test if two points really correspond to the same 3D point. Epipolar constraints can also be described by the essential matrix or the fundamental matrix between the two cameras.

* If the points x_L and x_R are known, their projection lines are also known. If the two image points correspond to the same 3D point X the projection lines must intersect precisely at X. This means that X can be calculated from the coordinates of the two image points, a process called "triangulation".

Simplified cases

The epipolar geometry is simplified if the two camera image planes coincide. In this case, the epipolar lines also coincide (E_L-x_L = E_R-x_R). Furthermore, the epipolar lines are parallel to the line O_L - O_R between the focal points, and can in practice be aligned with the horizontal axes of the two images. This means that for each point in one image, its corresponding point in the other image can be found by looking only along a horizontal line. If the cameras cannot be positioned in this way, the image coordinates from the cameras may be transformed to emulate having a common image plane. This process is called image rectification.

References

*cite book
author=Richard Hartley and Andrew Zisserman
title=Multiple View Geometry in computer vision
publisher=Cambridge University Press
year=2003
id=ISBN 0-521-54051-8

*cite web
author=Quang-Tuan Luong
title=Learning Epipolar Geometry
url=http://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.html
accessdate=2007-03-04

*cite web
author=Robyn Owens
title=Epipolar geometry
url=http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT10/node3.html
accessdate=2007-03-04

*cite book
author=Linda G. Shapiro and George C. Stockman
title=Computer Vision
publisher=Prentice Hall
year=2001
id=ISBN 0-13-030796-3
pages=395-403