Bayer filter

The Bayer arrangement of color filters on the pixel array of an image sensor

Profile/cross-section of sensor

A Bayer filter mosaic is a color filter array (CFA) for arranging RGB color filters on a square grid of photosensors. Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image. The filter pattern is 50% green, 25% red and 25% blue, hence is also called RGBG^[1][2], GRGB^[3], or RGGB.^[4]

It is named after its inventor, Bryce E. Bayer of Eastman Kodak. Bayer is also known for his recursively defined matrix used in ordered dithering.

Explanation

Bryce Bayer's patent (U.S. Patent No. 3,971,065) in 1976 called the green photosensors luminance-sensitive elements and the red and blue ones chrominance-sensitive elements. He used twice as many green elements as red or blue to mimic the physiology of the human eye. The luminance perception of the human retina uses M and L cone cells combined, during daylight vision, which are most sensitive to green light. These elements are referred to as sensor elements, sensels, pixel sensors, or simply pixels; sample values sensed by them, after interpolation, become image pixels.

The raw output of Bayer-filter cameras is referred to as a Bayer pattern image. Since each pixel is filtered to record only one of three colors, the data from each pixel cannot fully determine color on its own. To obtain a full-color image, various demosaicing algorithms can be used to interpolate a set of complete red, green, and blue values for each point.

Different algorithms requiring various amounts of computing power result in varying-quality final images. This can be done in-camera, producing a JPEG or TIFF image, or outside the camera using the raw data directly from the sensor.

Alternatives

The Bayer filter is almost universal on consumer digital cameras. Alternatives include the CYGM filter (cyan, yellow, green, magenta) and RGBE filter (red, green, blue, emerald), which require similar demosaicing. The Foveon X3 sensor (which layers red, green, and blue sensors vertically rather than using a mosaic) and arrangements of three separate CCDs (one for each color) don't need demosaicing.

Three new Kodak RGBW filter patterns

"Panchromatic" cells

On June 14, 2007, Eastman Kodak announced an alternative to the Bayer filter: a color-filter pattern that increases the sensitivity to light of the image sensor in a digital camera by using some "panchromatic" cells that are sensitive to all wavelengths of visible light and collect a larger amount of light striking the sensor.^[5] They present several patterns, but none with a repeating unit as small as the Bayer pattern's 2×2 unit.

Earlier RGBW filter pattern

Another 2007 U.S. patent filing, by Edward T. Chang, claims a sensor where "the color filter has a pattern comprising 2×2 blocks of pixels composed of one red, one blue, one green and one transparent pixel," in a configuration intended to include infrared sensitivity for higher overall sensitivity.^[6] The Kodak patent filing was earlier.^[7]

Such cells have previously been used in "CMYW" (cyan, magenta, yellow, and white)^[8] "RGBW" (red, green, blue, white)^[9] sensors, but Kodak has not compared the new filter pattern to them yet.

Microlenses and dichroic mirrors

Nikon's dichroic color separation patent drawing.

On November 21, 2006, Nikon Corporation was granted U.S. patent 7,138,663 which places a microlens over a triplet of photoreceptors. Specific wavelengths of light are separated and passed to specific photoreceptors designated to record red, green, and blue wavelengths. Light separation is achieved with dichroic mirrors. This system emulates three-CCD imaging systems with a single array. Image quality is theoretically improved due to much smaller gaps between photoreceptors assigned to a wavelength (each microlens lens acts as a collector for all pertinent wavelengths of light) and reduced light loss through the absence of wavelength filters.

Foveon's vertical color filter

Foveon's scheme of vertical filtering for color sensing

Another type of color separation, sometimes called vertical color filter, as opposed to lateral color filter, is used in the Foveon X3 sensor. Each location on the image sensor array has three stacked active pixel sensors, utilizing the sensor's silicon itself as the color filter, based on the wavelength-dependent absorption of light in silicon.^[10][11]

Color co-site sampling

With Color co-site sampling, color interpolation is not necessary. Using this method, the whole sensor is displaced in pixel distances after the first acquisition, in order to obtain full color information from each filter color of the Bayer pattern.

See also

• Color filter array
• Demosaicing
• CYGM filter
• RGBE filter
• Foveon X3 sensor
• Autochrome Lumière

References

Front page of Bryce Bayer's 1976 patent on the Bayer pattern filter mosaic, showing his terminology of luminance-sensitive and chrominance-sensitive elements

• US patent 3971065, Bryce E. Bayer, "Color imaging array", issued 1976-07-20  on web

Notes

1. ^ Jeff Mather (2008). "Adding L* to RGBG". http://jeffmatherphotography.com/dispatches/2008/05/adding-t-to-rgbg/. 
2. ^ dpreview.com (2000). "Sony announce 3 new digital cameras". http://www.dpreview.net/news/article_print.asp?date=0002&article=00020202sonydigicams. 
3. ^ Margaret Brown (2004). Advanced Digital Photography. Media Publishing. ISBN 0958188858. http://books.google.com/?id=nTWr_Lvkzu8C&pg=PT8&dq=GRGB+bayer. 
4. ^ Thomas Maschke (2004). Digitale Kameratechnik: Technik digitaler Kameras in Theorie und Praxis. Springer. ISBN 3540402438. http://books.google.com/?id=-THNPhdVIF8C&pg=PA40&dq=RGGB+bayer. 
5. ^ John Compton and John Hamilton (2007-06-14). "Color Filter Array 2.0". A Thousand Nerds: A Kodak blog. Archived from the original on 2007-07-20. http://web.archive.org/web/20070720002510/http://johncompton.1000nerds.kodak.com/default.asp?item=624876. Retrieved 2011-02-25. 
6. ^ "US patent publication 20070145273 "High-sensitivity infrared color camera"". http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PG01&s1=%22High-sensitivity+infrared+color+camera%22.TTL.&OS=TTL/%22High-sensitivity+infrared+color+camera%22&RS=TTL/%22High-sensitivity+infrared+color+camera%22. 
7. ^ "US Patent Application 20070024879 "Processing color and panchromatic pixels"". http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=6&p=1&f=G&l=50&d=PG01&S1=%28%28Kodak.AS.+AND+panchromatic%29+AND+clear%29&OS=an/Kodak+and+panchromatic+and+clear&RS=((AN/Kodak+AND+panchromatic)+AND+clear). 
8. ^ L. J. d'Luna et al. (1989). "A digital video signal post-processor for color image sensors". Proceedings of the Custom Integrated Circuits Conference 1989: 24.2/1. doi:10.1109/CICC.1989.56823. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=56823. "A variety of CFA patterns can be used, with various arrangements of red, green, and blue (RGB) or of cyan, magenta, yellow, and white (CMYW) colors." 
9. ^ Sugiyama, Toshinobu, US patent application 20050231618, "Image-capturing apparatus", filed March 30, 2005
10. ^ "Digital Photography Essentials #003: "Color Separation"". Digital Outback Photo. http://www.outbackphoto.com/dp_essentials/dp_essentials_03/essay.html. 
11. ^ Thomas Kreis (2006). Handbook of Holographic Interferometry: Optical and Digital Methods. Wiley-VCH. ISBN 3527604928. http://books.google.com/?id=CNMPLjC0li8C&pg=PA77&dq=vertical-color-filter. 

External links

• RGB "Bayer" Color and MicroLenses, Silicon Imaging (design, manufacturing and marketing of high-definition digital cameras and image processing solutions)
• eLynx image processing library, Big set of Bayer mosaic manipulation source code under GPL.
• Efficient, high-quality Bayer demosaic filtering on GPUs
• Evaluation of compression algorithms for Bayer format