Primatte chromakey technology

Primatte is a high-end chroma key technology used in motion picture, television and photographic host applications to remove solid colored backgrounds (greenscreen or bluescreen usually) and replace them with transparency to facilitate ‘background replacement’. It uses a unique algorithm based on three multi-faceted polyhedrons floating in RGB colorspace that are used to isolate color regions in the foreground image. Primatte is often referred to as a compositing technology and is usually used as a plug-in for host products such as Adobe After Effects, Adobe Photoshop, Autodesk Media and Entertainment Inferno or Flame, Eyeon Fusion and several other compositing and editing software packages.

Workflow

Primatte is usually activated on a foreground image with a person or other foreground object filmed or digitized against a solid colored background or backing screen; usually a bluescreen or a greenscreen. Using the Primatte tool set, the solid colored background area is removed and replaced with transparency. This allows the user to replace the solid colored background with a background image of his choice.

If the foreground object was filmed close to the backing screen or with less than ideal lighting conditions, the foreground object will usually have ‘spill’ somewhere on it. This is most common when filming blonde people against a blue or greenscreen as their translucent hair will absorb the backing screen color. Using the Primatte tool set, this ‘colorspill’ can be removed and replaced with several options to achieve a realistic result.

Another problem is chromakeying images that were created on film stock is that some film stocks have a distinct ‘film grain’ to them. A large amount of film grain will often cause rough edges around the foreground object. Again, using the Primatte tool set, these rough edges can be minimized to achieve a smooth transition between the edges of the ‘chroma keyed’ foreground object and the user selected background image.

History

The Primatte algorithm was created by Yasushi Mishima while working at IMAGICA Corporation in Tokyo, Japan in 1992. The basic algorithm utilized in Primatte was originally presented at the 8th NICOGRAPH Conference and the 23rd Imaging Technology Conference and a U.S. patent was granted in 1994.

It was initially released as a stand-alone product on Silicon Graphics workstations but it was later determined to be more useful as a ‘plug-in’ actuated from within a host application. This alleviated the need to save the images, exit the host compositing application, start Primatte, load the images, create the chroma key, save the images, start the host application, load the images and continue creating the composition. Since then it has been ported to almost every major compositing and editing software application on the market today. The current version is the fourth generation of the Primatte technology and has enhanced features such as an ‘Auto-Compute’ feature that automatically detects the backing screen color, eliminates it and does clean-up on the foreground and backing screen area noise making Primatte almost a 'one step' operation. It is now available on the Microsoft Windows, Red Hat Linux, SGI IRIX and the Apple Macintosh platforms.

The Algorithm

The Primatte chromakey algorithm is a sophisticated method of color space segmentation that can be easily explained to help a user achieve maximum effectiveness with the tool. Basically Primatte segments all the colors in the foreground image into one of four separate categories. The result is a 'spill suppressed' foreground image and a matte which is used to apply the modified foreground to a suitable background.

Primatte works in 3D RGB color space. Here is a visual representation of the Primatte algorithm after an image has been processed.

By operating the Primatte interface, the user essentially creates three concentric, multi-faceted polyhedrons. These can be pictured as three globes (or polyhedrons or polys), one within the other, which share a common center point. The creation of these polyhedrons separates all possible foreground colors into one of four regions; inside the small polyhedron (1), between the small and medium polyhedrons (2), between the medium and the large polyhedrons (3) and outside the large polyhedron (4).The four regions created are described as follows:

Region 1 (inside the small polyhedron) - This region contains all of the foreground image colors that are considered 100% background. These are the green or blue or whatever colors that were used as the backing color of the foreground image.

Region 2 (between the small and medium polyhedrons) - This region contains all the foreground colors that are at the edges of the foreground object(s), in glass, glass reflections, shadows, sheets of water and other transparent and semi-transparent color regions. These color regions also have spill suppression applied to them to remove color spill from the backing screen.

Region 3 (between the medium and large polyhedrons) - This region contains all the foreground image colors that are 100% foreground but have spill suppression applied to them to remove color spill from the backing screen. Otherwise they are 100% solid foreground colors.

Region 4 (outside the large polyhedron) - This region contains all the 100% foreground image colors that are not modified from the original foreground image. There is no spill suppression applied to these colors.

In the first step in using Primatte, Select BG Color, the user is asked to indicate the backing color on the original foreground image. The sample should usually be taken from a 'medium shaded' area near the foreground object. By 'medium shaded' area, it is meant that if green is the backing color and the green area of the foreground image has many shades of green ranging from very pale green to almost black, a shade of green in-between these extreme ranges should be chosen. If good results are not obtained using this sample, Primatte should be reset and another sample taken using a slightly darker or lighter shade of green. The first sample of Primatte often determines the final result as the center point of all three polyhedrons is created based on this first sample.

NOTE: An alternative method of keying in the Primatte Version 4 release is an Auto-Compute button the algorithmically detects the background color, eliminates it and does foreground and background noise removal. All the user might have to do is clean up some spill n the image.

A single pixel may be selected or a range of pixels (snail trail or rectangular sample). If a range of pixels is taken, the sample will be averaged to get a single color sample. This single pixel or averaged color sample then becomes the center of the small polyhedron. A few other shades around that color are included in the original small polyhedron.

After making a sample of the backing screen color in the first step, the result is a small golf ball-shaped poly as shown in the following image.

The second step in using Primatte is to clean up the backing color area by adding additional shades of green or blue to the small poly. This second step, Clean BG Noise, is usually executed while viewing the black and white Matte View.

While in the Clean BG Noise sampling mode, the user samples the white milky regions as shown in the left-hand image above. As the user samples these regions, they turn to black as shown in the right-hand image above.

What is happening in the Primatte algorithm is that these new shades of green (the white milky areas) are added to the small poly where all the shades of green or blue are moved. The next two images show the new pixels sampled (white dots) in relation to the small poly and the image next to it shows how the small poly extends outward to encompass the newly sampled colors into the small poly.

The advantage of this technique is that the polyhedron distorts to enclose only the shades of green that are in the backing screen. Other shades of green around these colors are left undisturbed in the foreground. Other chromakeyers expand from a golf ball-sized shape to a baseball to a basketball to a beach ball. Since it expands in all directions, many shades of green are relegated to 100% background making it hard to get good edges around the foreground objects.

Now that the user has created a small polyhedron, he must shape the medium and large polys. A default medium and large poly are both automatically created and are then modified based on the next couple of Primatte operations. The third Primatte step, Clean FG Noise, is to sample and eliminate gray areas in the 100% foreground area of the image.

Again, the user makes several samples on the dark, grayish areas on the foreground object until it is solid white in color. Primatte is shaping the large polyhedron with each color region that is sampled. Care should be taken in both this and the previous steps to not sample too close to the edges of the foreground object. Getting too close to the foreground object's edges will result in hard edges around the foreground object. Primatte uses these samples to modify and shape the medium and large polys to the desired shape. At this point, the matte or key has been created and would allow the foreground objects to be composited into a new background image.

If the user changes the display mode from the black and white Matte View to the color Composite View, there is usually 'color spill' on the edges (and sometimes the center) of the foreground objects. When on the edges of the foreground object, this spill comes from where the edges of the foreground object blended into the backing color. If it is on the center of the foreground object, it usually results from reflected color from the backing screen. The next Primatte step, either Spill Sponge, Fine Tuning or Spill(-), can now be used to eliminate this spill color.

What is happening in the Primatte algorithm while this next step is performed? Here is what the various tools in Primatte do to the Polyhedrons when they are used:

The Spill Sponge bulges the large polyhedron in the color region specified. A color region is specified by clicking on the image in a particular area with spill present. For example, if the user clicks on some spill on the cheek of a foreground person, Primatte goes to the section of the large polyhedron closest to that particular flesh tone and bulges the polyhedron there. As a result, the flesh tones move from outside the large poly to in-between the medium and large polys. This is Region 3 and is 100% foreground with spill suppression. As a result of the suppression, the spill is removed from that cheek color and all other shades of that color on the foreground. The user would then continue to sample areas of the image where spill exists and each sample would remove spill from another color region.

When all spill has been removed, the user should have a final composite. As a last step, go back to the Matte View and make sure that gray, transparent areas have not appeared in the foreground area. If there are any, the Matte Sponge operation mode should be selected and those gray pixels are sampled until they have all turned white again.

The Matte Sponge and Spill Sponge tools bulge or dent the polyhedrons a pre-selected amount. If the desired results are not achieved or the results are too extreme for the image, a manual method can be applied. The user should choose the Fine Tuning sliders, select a color region of interest and then move the appropriate slider to get the desired results.

For example, to remove spill, select a region of the composite image with spill on it. Move the spill or large poly slider to the right a little bit, the large poly will bulge and the spill should disappear. Move it a little more, if necessary. Moving this slider to the right removes spill (moves the colors from outside the large poly to between the medium and large polyhedrons) and moving it to the left, dents the large poly and moves that color region to outside the large poly.

If the user samples a foreground object shadow and then moves the Matte or medium poly slider to the right, the shadow will become more transparent. This is useful for matching composited shadows to shadows on the plate photography. It can also be used to make clouds or smoke more transparent.

If some foreground detail disappears during the composite, the user can select where the detail should be and then move the detail or small poly slider to the left. This dents the small poly in that color region and releases the detail pixels from the small poly into the visible region between the small and medium polyhedrons.The Spill Sponge and Matte Sponge tools are 'shortcut tools' that automatically move the sliders a pre-selected amount as a timesaving step for the user. Other 'shortcut tools' include the Make FG Trans. tool and the Restore Detail tool.

These 'shortcut tools' are one-step operations where the user clicks on a color region of interest and Primatte performs a pre-calculated operation. Most operations using Primatte would only require these tools, but the manual operation of the sliders is always an option.

The Spill(-) tool bulges the large poly a small amount incrementally in the color region that is clicked on and the Spill(+) tool dents it a small amount with each click. The Matte(-) and Matte(+) tools do the same to the medium poly and the Detail(-) and Detail(+) do it to the small poly.

See also

*Computer Generated Imagery
*Chroma key
*Special Effects

External links

* [http://www.primatte.com Primatte Web Site]
* [http://www.seanet.com/~bradford/bluscrn.html The Blue Screen/Chromakey Page]
* [http://entertainment.howstuffworks.com/blue-screen.htm How Blue Screens Work]