Image stitching

Image stitching or photo stitching is the process of combining multiple photographic images with overlapping fields of view to produce a segmented panorama or high-resolution image. Commonly performed through the use of computer software, most approaches to image stitching require nearly exact overlaps between images and identical exposures to produce seamless results.^[1] Some digital cameras can stitch their photos internally.

Alcatraz Island, seen in an example of a panorama created by image stitching

Stages of the stitching process

Example for geometrical registration and stitch line in panorama creation.

The image stitching process can be divided into three main components - image registration, calibration and blending.

Image registration involves matching features^[2] in a set of images or using direct alignment methods to search for image alignments that minimize the sum of absolute differences between overlapping pixels.^[3] When using direct alignment methods one might first calibrate one's images to get better results. Additionally, users may input a rough model of the panorama to help the feature matching stage, so that - for example - only neighboring images are searched for matching features. Since there are smaller group of features for matching, the result of the search is more accurate and execution of the comparison is faster.

Image calibration aims to minimize differences between an ideal lens models and the camera-lens combination that was used, optical defects such as distortions, exposure differences between images, vignetting,^[4] camera response and chromatic aberrations. If feature detection methods were used to register images and absolute positions of the features were recorded and saved, stitching software may use the data for geometric optimization of the images in addition to placing the images on the panosphere. Panotools and its various derivative programs use this method.

Image blending involves executing the adjustments figured out in the calibration stage, combined with remapping of the images to an output projection. Colors are adjusted between images to compensate for exposure differences. If applicable, high dynamic range merging is done along with motion compensation and deghosting. Images are blended together and seam line adjustment is done to minimize the visibility of seams between images.

Projection geometry

Comparing distortions near poles of panosphere by various cylindrical formats.

For image segments that have been taken from the same point in space, stitched images can be arranged using one of various map projections.

Rectilinear projection, where the stitched image is viewed on a two-dimensional plane intersecting the panosphere in a single point. Lines that are straight in reality are shown as straight regardless of their directions on the image. Wide views - around 120° or so - start to exhibit severe distortion near the image borders. One case of rectilinear projection is the use of cube faces with cubic mapping for panorama viewing. Panorama is mapped to six squares, each cube face showing 90 by 90 degree area of the panorama.

Cylindrical projection, where the stitched image shows a 360° horizontal field of view and a limited vertical field of view. Panoramas in this projection are meant to be viewed as though the image is wrapped into a cylinder and viewed from within. When viewed on a 2D plane, horizontal lines appear curved while vertical lines remain straight.^[5] Vertical distortion increases rapidly when nearing the top of the panosphere. There are various other cylindrical formats, such as Mercator and Miller cylindrical which have less distortion near the poles of the panosphere.

Spherical projection or equirectangular projection — which is strictly speaking another cylindrical projection — where the stitched image shows a 360° horizontal by 180° vertical field of view i.e the whole sphere. Panoramas in this projection are meant to be viewed as though the image is wrapped into a sphere and viewed from within. When viewed on a 2D plane, horizontal lines appear curved as in a cylindrical projection, while vertical lines remain vertical.^[5]

Since a panorama is basically map of a sphere, various other mapping projections from cartographers can also be used if so desired. Additionally there are specialized projections which may have more aesthetically pleasing advantages over normal cartography projections such as Hugin's Panini projection^[6] - named after Italian vedutismo painter Giovanni Paolo Pannini^[7] - or PTgui's Vedutismo projection.^[8] Different projections may be combined in same image for fine tuning the final look of the output image.^[9]

Stereographic projection or fisheye projection can be used to form a little planet panorama by pointing the virtual camera straight down and setting the field of view large enough to show the whole ground and some of the areas above it; pointing the virtual camera upwards creates a tunnel effect. Conformality of the stereographic projection may produce more visually pleasing result than equal area fisheye projection as discussed in the stereographic projection's article.

Challenges of image stitching

The use of images not taken from the same place (on a pivot about the entrance pupil of the camera)^[10] can lead to parallax errors in the final product. When the captured scene features rapid movement or dynamic motion, artifacts may occur as a result of time differences between the image segments. "Blind stitching" through feature-based alignment methods (see autostitch), as opposed to manual selection and stitching, can cause imperfections in the assembly of the panorama.

Image stitching software

See also: Comparison of photo stitching applications

Dedicated programs include Autostitch, Hugin, Ptgui, Panorama Tools, Microsoft Research Image Composite Editor, and CleVR Stitcher. Many other programs can also stitch multiple images; a popular example is Adobe Systems' Photoshop, which includes a tool known as Photomerge and, in the latest versions, the new Auto-Blend.

See also

• ActionShot panoramic photography
• Panography
• Panoramic photography

References

1. ^ Ward, Greg (2006). "Hiding seams in high dynamic range panoramas". Proceedings of the 3rd symposium on Applied perception in graphics and visualization. 153. ACM International Conference Proceeding Series. ACM. doi:10.1145/1140491.1140527. ISBN 1-59593-429-4. 
2. ^ Szeliski, Richard (2005) (PDF). Image Alignment and Stitching. http://www.caam.rice.edu/~zhang/caam699/p-files/Im-Align2005.pdf. Retrieved 2008-06-01. 
3. ^ S. Suen; E. Lam, and K. Wong (2007). "Photographic stitching with optimized object and color matching based on image derivatives". Optics Express 15: 7689–7696. doi:10.1364/OE.15.007689. PMID 19547097. http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-15-12-7689. 
4. ^ Pablo d'Angelo (2007). Radiometric alignment and vignetting calibration. http://hugin.sourceforge.net/tech/icvs2007_final.pdf. 
5. ^ ^{a b} Wells, Sarah et al. (2007). IATH Best Practices Guide to Digital Panoramic Photography. http://www.iath.virginia.edu/panorama/section4.html#4.1. Retrieved 2008-06-01. 
6. ^ Hugin.sourceforge.net, hugin manual: Panini
7. ^ Groups.google.com, hugin-ptx mailing list, December 29, 2008
8. ^ PTgui: Projections
9. ^ Tawbaware.com, PTAssembler projections: Hybrid
10. ^ Littlefield, Rik (2006-02-06) (PDF). Theory of the "No-Parallax" Point in Panorama Photography. ver. 1.0. http://www.janrik.net/PanoPostings/NoParallaxPoint/TheoryOfTheNoParallaxPoint.pdf. Retrieved 2008-06-01.