Raw image format

Infobox file format
name = RAW image file
extension = .raf (Fuji)
.crw .cr2 (Canon)
.tif .k25 .kdc .dcs .dcr .drf (Kodak)
.mrw (Minolta)
.nef .nrw (Nikon)
.orf (Olympus)
.dng (Adobe)
.ptx .pef (Pentax)
.arw .srf .sr2 (Sony)
.x3f (Sigma)
.erf (Epson)
.mef .mos (Mamiya)
.raw (Panasonic)
.cap .tif .iiq
(Phase One)
.r3d (Red)
.fff (Imacon)
.pxn (Logitech)
.bay (Casio)
.rwz (Rawzor)
mime =
owner =
creatorcode =
genre = Image file formats
containerfor =
containedby =
extendedfrom =
extendedto =A raw image file contains minimally processed data from the image sensor of a digital camera or image scanner. Raw files are so named because they are not yet processed and ready to be used with a bitmap graphics editor or printed. Normally, the image will be processed by a raw converter in a wide-gamut internal colorspace where precise adjustments can be made before conversion to an RGB file format such as TIFF or JPEG for storage, printing, or further manipulation. These images are often described as "RAW" image files" (note capitalization) based on the erroneous belief that they represent a single file format, and thus deserve a common filename extension, ".RAW". In fact there are dozens if not hundreds of raw image formats in use by different models of digital cameras. [ [http://www.cybercom.net/%7Edcoffin/dcraw/#i18n Decoding raw digital photos in Linux ] ]

Raw image files are sometimes called digital negatives, as they fulfill the same role as film negatives in traditional chemical photography: that is, the negative is not directly usable as an image, but has all of the information needed to create an image. In addition to raw files from cameras, raw data from film scanners can also be referred to as digital negatives. Likewise, the process of converting a raw image file into a viewable format is sometimes called developing a raw image, by analogy with the film development.

Like a photographic negative, a digital negative may have a wider dynamic range or color gamut than the eventual final image format. The selection of the final choice of image rendering is part of the process of white balancing and color grading.

File contents

Providing a detailed and concise description of the content of raw files is highly problematic. There is no single raw format; formats can be similar or radically different. Different manufacturers use their own proprietary and typically undocumented formats, which are collectively known as raw format. Often they also change the format from one camera model to the next. Multiple major manufacturers, including Nikon, Canon, and Sony encrypt portions of the file in an attempt to prevent third-party tools from accessing them.cite web|url=http://www.dpreview.com/news/0504/05042701davecoffininterview.asp |title=Raw storm in a teacup? |publisher=Dpreview.com |accessdate=2007-12-09 |date=2005-04-27 Dave Coffin, creator of dcraw, discusses some of his successful reverse-engineering in this interview, and mentions his enthusiasm for the DNG format.] This industry-wide situation has concerned many photographers who worry that their very valuable raw photos may someday become inaccessible, as operating systems and applications become obsolete and abandoned raw formats are dropped from new software. The availability of high-quality open source software which decodes raw image formats, particularly dcraw, has helped to alleviate these concerns. Adobe has developed and promoted a standardized raw image format called DNG ("digital negative"); this has been received enthusiastically by open-source developers but has received little support from major camera makers other than Pentax.

Ultimately, any raw format's purpose is to faithfully record both 100% of exactly what the sensor "saw" (the data) and the conditions surrounding the recording of the image (the metadata).

Digital camera raw files contain the pixel data from a rectangular image sensor, the modern equivalent of traditional film, usually at 12 or 14 bits per sensor bucket. The sensor is almost invariably overlaid with a so-called Bayer filter, consisting of a mosaic of red, blue and green filters in alternating rows of RG and GB. Given that three colors fit uncomfortably in a rectangular grid, green was chosen to be doubly present, since the human eye is more sensitive to it. Green also often serves as the luminance channel, and as the dominant channel for in-camera black-and-white conversions. To retrieve an image from a raw file, this mosaic of data must be converted into a full RGB image. This is formally known as demosaicing, but is often referred to as digital development, by analogy with the development process used to convert photographic film into viewable prints.

One variation on the Bayer scheme is the RGBE sensor of the Sony DSC-F828, which experimented with exchanging the green in the RG rows with Emerald (cyan). Other sensors, such as the Foveon X3 sensor capture information directly in RGB form, having three pixel sensors in each location, one for each color component; these camera RGB raw data still need to be processed to make an image file.

Flatbed and film scanner sensors are typically straight narrow RGB or RGBI (where "I" is infrared) strips that are swept across an image; other than that, the remainder of the discussion about raw files applies to them as well. (Some scanners do not allow the user access to the raw data at all, as a speed compromise. The raw data are processed very rapidly inside the scanner to select out the best part of the available dynamic range so only the result is passed to the computer for permanent storage.)

The contents of raw files include more information, and potentially higher quality, than the converted results, in which the rendering parameters are fixed, the color gamut is clipped, and there may be quantization and compression artifacts. Each pixel in a raw file has a greater bit-depth (compared to typical 8-bit renderings), and can thus store more subtle variations and range in color and detail. Hence, large transformations of the data, such as increasing the exposure of a dramatically under-exposed photo, result in less visible artifacts when done from raw data than when done from already rendered image files. Raw data leave more scope for both corrections and artistic manipulations, without resulting in images with visible flaws such as posterization.

The generally-accepted standard for digital negatives in the digital cinema industry is the SMPTE DPX format.

Benefits

Nearly all digital cameras can process the image from the sensor into a JPEG file using settings for white balance, color saturation, contrast, and sharpness that are either selected automatically or entered by the photographer before taking the picture. Cameras that support raw files save these settings in the file, but defer the processing. This results in an extra step for the photographer, so raw is normally only used when additional computer processing is intended. However, raw has numerous advantages over JPEG such as:
* Higher image quality. Because all the calculations (such as applying the gamma curve, demosaicing, white balance, brightness, contrast, etc...) used to generate pixel values (in RGB format for most images) are performed in one step on the base data, the resultant pixel values will be more accurate and exhibit less posterization.
* JPEG is a lossy compression format. Raw formats are either uncompressed or use lossless compression, so the maximum amount of image detail is always kept within the RAW file.
* Finer control. Using RAW conversion software allows users to manipulate more parameters (such as lightness, white balance, hue, saturation, etc...) and do so with greater variability. For example, the white point can be set to any value, not just discrete values like "daylight" or "incandescent".
* Camera raw files have 12 or 14 bits of intensity information, not the gamma-compressed 8 bits typically stored in processed TIFF and JPEG files; since the data is not yet rendered and clipped to a color space gamut, more precision may be available in highlights, shadows, and saturated colors.
* The working color space can be set to whatever is desired.
* Different demosaicing algorithms can be used, not just the one coded into the camera.

Drawbacks

Camera raw files are typically 2–6 times larger than JPEG files. [cite web|url=http://www.ineasysteps.com/resources/articles/read/?id=10 |title=Understanding Camera RAW] Some raw formats do not use compression, some implement lossless data compression to reduce the size of the files without affecting image quality and others use lossy data compression where quantization and filtering is performed on the image data. While use of raw formats avoids the compression artifacts inherent in JPEG, fewer images can fit on a given memory card. It also takes longer for the camera to write raw images to the card, so fewer pictures can be taken in quick succession (affecting the ability to take, for example, a sports sequence).

There is still no widely accepted standard raw format. Adobe's Digital Negative (DNG) format has been put forward as a standard, but has not been adopted by many major camera companies; Pentax's K20D is one recent DSLR camera that can shoot directly into DNG format. Numerous different raw formats are currently in use and new raw formats keep appearing, while others are abandoned. [cite web|url=http://www.openraw.org/info/|title=The RAW Problem|publisher=OpenRAW|author=Larry Strunk|date=2006-03-19]

Because of the lack of a standard raw format, more specialized software may be required to open raw files than for standardized formats like JPEG or TIFF. Software developers are also having to frequently update their products to support the raw formats of the latest cameras.

The time taken in the image workflow is an important factor when choosing between raw and ready-to-use image formats.

Software support

Cameras that support raw files typically come with proprietary software for conversion of their raw format to TIFF or JPEG. Other conversion programs and plugins are available from vendors that have either licensed the technology from the camera manufacturer or reverse-engineered the particular raw format.

Raw file formats are proprietary, and differ greatly from one manufacturer to another, and sometimes between cameras made by one manufacturer. In 2004 Adobe Systems published the Digital Negative Specification (DNG), which is intended to be a unified raw format. Adobe Photoshop CS2 and CS3 contain extensive support of RAW as does Adobe Photoshop Lightroom. As of 2006, several camera manufacturers have started to announce support for DNG in newer camera models, including Leica, Samsung, Ricoh, Pentax (native camera support) and Hasselblad (export). The Leica Digital-Modul-R (DMR) was first to use DNG as its native format.

Microsoft has submitted its HD Photo specification for ratification as JPEG XR for use as a raw format.

Microsoft's Digital Image 2006 recognizes and organizes raw image formats such as .crw, .cr2, and .nef, which are file formats produced by Canon and Nikon,Fact|date=September 2007 but that product was discontinued in 2007. [ [http://www.microsoft.com/products/imaging/default.mspx Microsoft Digital Image Home page ] ]

For Windows XP, there is a free download available that integrates viewing and printing into other included photo tools, but it is not supported by Microsoft. [ [http://www.microsoft.com/windowsxp/using/digitalphotography/prophoto/raw.mspx Microsoft RAW Image Thumbnailer and Viewer for Windows XP ] ]

Windows XP and Vista both support the WIC codec standard. Products such as Windows Photo Gallery, Windows Live Photo Gallery and FastPictureViewer Professional [ [http://www.fastpictureviewer.com/formats/ FastPictureViewer's Image Formats Compatibility Chart] ] can view raw formats for which the necessary WIC codecs are installed. Camera manufacturers Canon, Nikon, Sony, Olympus and Pentax have released WIC codecs although some manufactures are only providing codec support for the 32bit versions of Vista. [ [http://windowsvistablog.com/blogs/windowsvista/archive/2007/02/15/understanding-raw-image-support-in-windows-vista.aspx Understanding RAW Image Support in Windows Vista: Windows Vista team blog] ] A commercial DNG codec is also available from Ardfry Imaging. [ [http://www.ardfry.com/dng-codec DNG Thumbnail and Preview Support for Windows Photo Gallery and Windows Live Photo Gallery] ]

In 2005, Apple Computer introduced several products which offered raw file support. In January, Apple released iPhoto 5, which offered basic support for viewing and editing raw files. In April, Apple introduced a new version of its operating system, Mac OS X v10.4, which added raw support directly to the operating system, as part of the ImageIO framework, which adds raw support automatically to the majority of Mac OS X applications both from Apple (such as Preview, Mac OS X's PDF and image viewing application and Aperture, a photo post-production software package for professionals) as well as all third party applications which make use of the ImageIO frameworks. Semi-regular updates to OS X generally include updated support for new RAW file formats introduced in the intervening months by camera makers.

There are many other "raw workflow applications" designed to provide efficient processing and post-processing of raw images, including Helicon Filter, Phase One's Capture One and Bibble Labs' Bibble Pro. Like Apple Aperture, Adobe Photoshop, Adobe Photoshop Lightroom, LaserSoft Imaging's SilverFast and PhotoLine, these programs provide sophisticated controls for processing the information stored in the Raw file and converting raw files to JPEG or TIFF. Picasa, a free image editing and cataloguing program from Google, can read and display many raw formats, but like iPhoto, Picasa provides only limited tools for processing the data in a raw file.

A portable open source program, dcraw, supports most raw formats and can be made to run on operating systems not supported by most commercial software (such as Unix).

UFRaw is free software based on dcraw. It can be used as a GIMP plugin and is available for most operating systems.

RawShooter Essentials 2005/6 was freeware software developed by Pixmantec. In 2006 Adobe Systems Inc acquired the assets of Pixmantec ApS. RawShooter Essentials is no longer being updated (The last update added support for the Canon 5D and the Nikon D200). It could still be downloaded for free until Adobe's Photoshop Lightroom 1.0 was released in March 2007. The software was fully featured, including wide support for various raw formats, file priority sorting and batch processing. Light Crafts' LightZone photo editing software provides the ability to edit RAW natively. Most tools are "raw converters," but LightZone allows a user to edit RAW just as if it were TIFF or JPEG.

Processing

There is no single standard algorithm for converting data from a Bayer filter or Foveon sensor into RGB format; a number of different algorithms have been proposed, and some have been patented in the USA. Different programs may give slightly different results, better or worse subjectively, for any particular image.

Although the term "raw" describes files in the classical sense of "raw data" vs. "cooked data", raw files typically are slightly processed in the camera. In general, this processing is limited to algorithms that require direct access to the camera's hardware. This includes "long exposure noise reduction" (aka “dark frame subtraction”) and the mapping out of "hot" (too bright) or "dead" (too dim) pixels. Also information about standard processing parameters are stored in the file (so a Raw converter can generate the same JPEG file as the camera would create).

Some newer Raw formats also allow nonlinear quantization. This allows to compress the raw data without visible degradation of the image by removing invisible and irrelevant information from the image. Although noise is discarded this has nothing to do with (visible) noise reduction.Fact|date=January 2008

ee also

* Image file formats
* List of cameras supporting a raw format

References

External links

* Adobe: [http://www.adobe.com/digitalimag/pdfs/understanding_digitalrawcapture.pdf Understanding Raw Files] (PDF); Background on how camera sensors treat raw files
* [http://lclevy.free.fr/cr2 How to decode a Canon .CR2 file] An tentative to document the Canon Raw format v2. (L. Clevy)
* [http://jrawio.dev.java.net jrawio] : An Open Source Java library for reading camera raw files from various digital camera models.
* [http://libopenraw.freedesktop.org/ libopenraw] , an ongoing effort to provide a free software implementation for camera RAW files decoding.
* [http://www.openraw.org/ Open RAW] A working group of photographers and other people interested in advocating the open documentation of digital camera raw files
* [http://www.rawtherapee.com Raw Therapee] (THe Experimental RAw Photo Editor) is a free RAW converter and digital photo processing software.

;Photographers' views on the RAW format:

* Bob Atkins: [http://www.photo.net/learn/raw/ Raw, JPEG, and TIFF] ; common file formats compared.
* John Roling: [http://www.connectedphotographer.com/issues/issue200505/00001529001.html A RAW Deal: Using the RAW image format] ; explanation and arguments for raw formats.
* Jim M. Goldstein: [http://www.jmg-galleries.com/articles/raw_vs_jpeg_is_shooting_raw_right_for_me.html RAW vs JPEG: Is Shooting RAW Format For Me?]