Tone mapping

Tone mapping is a technique used in image processing and computer graphics to map a set of colours to another; often to approximate the appearance of high dynamic range images in media with a more limited dynamic range. Print-outs, CRT or LCD monitors, and projectors all have a limited dynamic range which is inadequate to reproduce the full range of light intensities present in natural scenes. Essentially, tone mapping addresses the problem of strong contrast reduction from the scene values (radiance) to the displayable range while preserving the image details and color appearance important to appreciate the original scene content.

Purpose and methods

The goals of tone mapping can be differently stated depending on the particular application. In some cases producing just "nice-looking" images is the main goal, while other applications might emphasize reproducing as many image details as possible, or maximizing the image contrast. The goal in realistic rendering applications might be to obtain a perceptual match between a real scene and a displayed image even though the display device is not able to reproduce the full range of luminance values.

Various tone mapping operators have been developed in the recent years ref|devlin. They all can be divided in two main types:
*"global" (or "spatially uniform") operators: they are non-linear functions based on the luminance and other global variables of the image. Once the optimal function has been estimated according to the particular image, every pixel in the image is mapped in the same way, independent of the value of surrounding pixels in the image. Those techniques are simple and fast (since they can be implemented using look-up-tables), but they can cause a loss of contrast.

*"local" (or "spatially varying") operators: the parameters of the non-linear function change in each pixel, according to features extracted from the surrounding parameters. In other words, the effect of the algorithm changes in each pixel according to the local features of the image. Those algorithms are more complicated than the global ones, they can show artifacts (e.g. halo effect and ringing), the output can look un-realistic, but they can provide the best performance, since the human vision is mainly sensitive to local contrast.

A simple example of global tone mapping filter is $L=Y/(Y+1)$. This function will map scene radiance values "Y" in the domain $[0,infty)$ to a displayable output range of $[0,1)$.

A more sophisticated group of tone mapping algorithms is based on contrast or gradient domain methods, which are 'local'. Such operators concentrate on preserving contrast between neighboring regions rather than absolute value, an approach motivated by the fact that the human perception is most sensitive to contrast in images rather than absolute intensities. Those tone mapping methods usually produce very sharp images, which preserve very well small contrast details; however, this is often done at the cost of flattening an overall image contrast. Examples of such tone mapping methods include: gradient domain high dynamic range compression ref|fattal and A Perceptual Framework for Contrast Processing of High Dynamic Range Imagesref|mantiuk (a tone mapping is one of the applications of this framework).

An interesting approach to tone mapping of HDR images is inspired by the anchoring theory of lightness perception ref|gilchrist. This theory explains many characteristics of the human visual system such as lightness constancy and its failures (e.g. the same color illusion), which are important in the perception of images. The key concept of this tone mapping method (Lightness Perception in Tone Reproductionref|krawczyk) is a decomposition of an HDR image into areas (frameworks) of consistent illumination and the local calculation of the lightness values. The net lightness of an image is calculated by merging of the frameworks proportionally to their strength. Particularly important is the anchoring -- relating the luminance values to a known brightness value, namely estimating which luminance value is perceived as white in the scene. This approach to tone mapping does not affect the local contrast and preserves the natural colors of an HDR image due to the linear handling of luminance.

Example of the imaging process

The images on the right show the interior of a church, a scene which has a variation in radiance much larger than that which can be displayed on a monitor or recorded by a conventional camera. The six individual exposures from the camera show the radiance of the scene in some range transformed to the range of brightnesses that can be displayed on a monitor. The range of radiances recorded in each photo is limited, so not all details can be displayed at once: for example, details of the dark church interior cannot be displayed at the same time as those of the bright stained-glass window. An algorithm is applied to the six images to recreate the high dynamic range radiance map of the original scene (a high dynamic range image). Alternatively, some higher-end consumer and specialist scientific digital cameras are able to record a high dynamic range image directly, for example with RAW images.

In the ideal case, a camera might measure luminance directly and store this in the HDR image; however, most high dynamic range images produced by cameras today are not calibrated or even proportional to luminance, due to practical reasons such as cost and time required to measure accurate luminance values — it is often sufficient for artists to use multiple exposures to gain an "HDR image" which grossly approximates the true luminance signal.

The high dynamic range image is passed to a tone mapping operator, in this case a non-local operator, which transforms the image into a low dynamic range image suitable for viewing on a monitor. Relative to the church interior, the stained-glass window is displayed at a much lower brightness than a linear mapping between scene radiance and pixel intensity would produce. However, this inaccuracy is perceptually less important than the image detail, which can now be shown in both the window and the church interior simultaneously.

Visual effect

Tone-mapping is a standard photographic method, long predating digital image processing, although not historically known as "tone-mapping." This method is used to compress high image contrast for reproduction on a medium with smaller dynamic range. Reducing dynamic range with tone-mapping is often useful in bright sunlit scenes, where the difference in intensity between direct illumination and shadow is great. Use of tone-mapping in this context may not be apparent from the final

Tone-mapping can also be used as an effect to produce distinctive images from scenes where the dynamic range may — but need not — be particularly high. A visual effect characteristic of some local tone-mapping operators is the production of bright 'halos' around darker objects, such as that which can be seen in the Cornell Law School photo.

Gallery

References

# Kate Devlin, Alan Chalmers, Alexander Wilkie, Werner Purgathofer. "STAR Report on Tone Reproduction and Physically Based Spectral Rendering" in Eurographics 2002. DOI: [http://doi.acm.org/10.1145/1073204.1073242 10.1145/1073204.1073242]
# Raanan Fattal, Dani Lischinski, Michael Werman. [http://www.cs.huji.ac.il/~danix/hdr/ "Gradient Domain High Dynamic Range Compression"]
# Rafal Mantiuk, Karol Myszkowski, Hans-Peter Seidel. [http://www.mpi-sb.mpg.de/~mantiuk/contrast_domain/ "A Perceptual Framework for Contrast Processing of High Dynamic Range Images"]
# Alan Gilchrist. [http://psychology.rutgers.edu/~alan/theory3/ "An Anchoring Theory of Lightness Perception"] .
# Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel. [http://www.mpi-inf.mpg.de/resources/hdr/lightness/ "Lightness Perception in Tone Reproduction for High Dynamic Range Images"]

ee also

* Color translation

External links

* [http://www.mpii.mpg.de/resources/tmo/ pfstmo: implemenation of tone mapping operators]
* [http://scanline.ca/exrtools/ exrtools: a collection of utilities for manipulating OpenEXR images] (includes some tone mapping operators)
* [http://www.mpi-inf.mpg.de/resources/pfstools/ pfstools] is an open-source set of command line programs for reading, writing and manipulating high-dynamic range (HDR) images and video frames
* [http://qtpfsgui.sourceforge.net/ QtPfsGui] is a free (open-source) HDR-workflow software for Linux, Windows and Mac OS X based around the pfstools package

Tone mapping algorithms

* [http://www.cs.utah.edu/~reinhard/cdrom/ Photographic Tone Reproduction for Digital Images]
* [http://www.mpi-inf.mpg.de/resources/hdr/lightness/ Lightness Perception in Tone Reproduction for High Dynamic Range Images]
* [http://www.mpi-inf.mpg.de/~mantiuk/contrast_domain/ Contrast Processing of High Dynamic Range Images]
* [http://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/ Fast Bilateral Filtering for the Display of High-Dynamic-Range Images]
* [http://people.csail.mit.edu/sparis/bf/ A Fast Approximation of the Bilateral Filter using a Signal Processing Approach]
* [http://www.cs.huji.ac.il/~danix/hdr/ Gradient Domain High Dynamic Range Compression]