Microsoft Talisman

Talisman was a Microsoft project to build a new 3D graphics architecture based on quickly compositing 2D "sub-images" onto the screen, an adaptation of tiled rendering. In theory, this approach would dramatically reduce the amount of memory bandwidth required for 3D games and thereby lead to lower-cost graphics accelerators. The project took place during the introduction of the first high-performance 3D accelerators, and these quickly surpassed Talisman in both performance and price. No Talisman-based systems were ever released commercially, and the project was eventually cancelled in the late 1990s.

Description

Conventional 3D

Creating a 3D image for display consists of a series of steps. First, the objects to be displayed are loaded up into memory from individual "models". The display system then applies mathematical functions to transform the objects into a common coordinate system, the "world view". From this world view, a series of polygons (typically triangles) is created that approximates the original models as seen from a particular viewpoint, the "camera". Next, a compositing system produces an image by rendering the triangles and applying "textures" to the outside. Textures are small images that are painted onto the triangles to produce realism. The resulting image is them combined with various special effects, and moved into the display buffers. Of this process, the "display pipeline", only the geometry tends to change frequently, due to changes in the camera location, while the models and textures generally only change over much longer periods of time.

Typical 3D accelerators of the mid-90s helped only with one step of this process, compositing the image. The cards had memory buffers for the textures that were loaded up only as required, and were then instructed to composite those textures onto geometry being handed to it by the CPU. The CPU was still responsible for all of the other work, mostly the conversion of the model objects coordinates into the world coordinates, and then sending the resulting models to the card as a series of polygons. By offloading the "easy part" of the operation to the card, the CPU could spend more time working on the models, allowing the complexity of the scenes to improve. As the textures were often the memory intensive part of the system, placing them semi-permanently on the card in high-speed memory allowed the compositing steps to be greatly sped up.

In this mode of operation every frame was created though what was essentially brute force. As soon as one frame had completed drawing, the CPU would start the process over, sending in a new set of polygons and asking the card to composite and render them into a framebuffer for eventual display. There was no memory of the already rendered objects, so even objects that did not move in the display were nevertheless re-rendered from scratch for every frame. Although the CPU's workload was reduced, the bandwidth needed to send these more complex models to the cards was relatively high.

Much greater improvements in performance could be had if the graphics card also stored the polygons. However, these had to be re-created from scratch every time the scene changed or the user's viewpoint (the "camera") changed. This was common, changing almost every frame, whereas textures would generally change only when new models were introduced. Adding this functionality on the card was certainly possible, but it would require the graphics card to have considerable processing power, on par with the computer's CPU. Avoiding this complexity on the card meant that the number of polygons visible on the display had to be limited to what could be comfortably sent to the graphics card every time the frame updated.

A typical gaming display of the era using 640 x 480 resolution with 24-bit color, with basic 3D compositing with trilinear filtering and no anti-aliasing, would require 1,900 MB/s of memory bandwidth. The same rendering settings on a 1024 x 768 display would require 14,200 MB/s, and even basic anti-aliasing would be expected to roughly double that figure.^[1] For reference, SGI's then-current RealityEngine2 machines featured a then-high memory bandwidth of about 10,000 MB/s, which was the reason these machines were widely used in 3D graphics. A typical PC of the era using AGP 2X could offer only 508 MB/s.

Tiled rendering

Talisman attempted to reduce the memory bandwidth requirements primarily by leaving unchanged post-rendered objects in memory to be re-composited without change. The system used a series of memory buffers and parallel compositing engines to quickly "assemble" a display that was made up of a number of 2D images, or "tiles". This technique is immediately suitable for 2D images, as any possible change in the display mapping can be made by individual mappings on the tiles. The same is not true from 3D, however, where rotations of the user's viewpoint requires objects in the display to rotate in 3D, something that can only be simulated by modifying a 2D image.

To produce the illusion of 3D, each of the individual tiles were manipulated through an affine transform before compositing, allowing the image to appear to "move" in space, at least in terms of perspective changes. The host CPU would only be asked to re-render a particular tile when the changes in perspective were great enough that the transform could no longer result in an artifact-free image. In that case the CPU would run its transformations again, handing the card a new model to be rendered back into the buffer. Microsoft calculated that each tile could be re-used for about four frames on average, thereby reducing load on the CPU by about four times.

The Talisman image buffers were broken down into 32 x 32 pixel "chunks" that were individually rendered using the 3D objects and textures provided by the CPU. The card would determine which objects overlapped which chunks, transforming those objects into 2D objects describing the chunk, which was then compressed and stored. Pointers to the chunks were then stored in a z-ordered (front to back) list for every 32 scan-lines on the display. One concern is that the chunks cannot be cleanly "stitched together", a problem that has sometimes been visible in various videogames using software rendering. To avoid this, Talisman also stored a separate "edge buffer" for every chunk that stored an "overflow" area that would cover gaps in the mapping.

Rendering pipeline

In a conventional 3D system, geometry is periodically generated, sent to the card for composition, composed into a framebuffer, and then eventually picked up by the video hardware for display. Talisman systems essentially reversed this process; the screen was divided into the same 32-high scan-line strips, and while the video hardware was drawing one of these strips, the hardware would call the Talisman side and tell it to prepare the details for the next strip.

The system would respond by selecting the next strip and retrieving any chunks that were visible in that strip given the current camera location. In the typical case many of the chunks would be obscured by other chunks, and could be ignored during compositing, saving time. This is the reason for the z-sorting of the chunks, which allows them to be efficiently retrieved in "visibility order". If the chunks could be modified without distortion, the proper affine transform was called to update the chunk in-place. If it could not, say because the camera had moved too much since the last full update, the CPU was asked to provide new geometry for that chunk, which the card then rendered and placed back in storage.

Talisman had no analog of a framebuffer, rendering chunks on demand directly to the screen as the monitor's scan line progressed down the screen. This is an interesting analog with the Atari 2600, which uses a similar system to render 2D images on the screen, a method known as "racing the beam". In both cases, this reduced the amount of memory needed, and the memory bandwidth being used between the display system and video hardware. In both cases this also required dramatically tighter integration between the video system and the programs running it. In the case of Talisman, the programs were required to store their objects in a particular format that the Talisman software drivers understood, allowing it to be quickly picked up from memory during interrupts.

History

Introduction

The Talisman effort was Microsoft's attempt to commercialize concepts that had been experimented on for some time. In particular, the PixelFlow system developed at a Hewlett-Packard research lab at the University of North Carolina at Chapel Hill can be considered Talisman's direct parent.^[2]

When Talisman was first made widely public at the 1996 SIGGRAPH meeting, they promised a dramatic reduction in the cost of implementing a graphics subsystem.^[3] They planned on working with vendors to sell the concept of Talisman for inclusion into other companies' display systems. That is, Talisman was hoped to be a part of a larger media chip, as opposed to an entire 3D system that would stand alone in a system. Their basic system would support 20-30,000 polygons on a 1024 x 768 display at 32 bit/pixel, with a 40 Mpixel/s polygon rendering rate and 320 Mpixel/s image layer compositing rate.

Escalante

At the time, Microsoft was working with several vendors in order to develop a reference implementation known as Escalante. Samsung and 3DO were working together to design a single-chip DSP-like "Media Signal Processor" (MSP), combining Talisman functionality with additional media functionality. Cirrus Logic would provide a VLSI chip that would retrieve data placed in memory by the MSP, apply effects, and send it off for display. Known as the "Polygon Object Processor" (POP), this chip was periodically polled by another Cirrus Logic chip, the "Image Layer Compositor" (ILC), which was tied to the video circuitry. Additionally, Escalante intended to feature 4 MB of RDRAM on two 600 MHz 8-bit channels, offering 1.2 GB/s throughput. ^[4] Later Philips entered the fray with a planned new version of their TriMedia processor, which implemented most of Talisman in a single CPU, and Trident Microsystems, with similar plans.

It was in the midst of the Talisman project that the first person shooter genre started to come to the fore in gaming. This created market demand for accelerators that could be used with existing games with minimal changes. By the time the Escalante reference design was ready for production, the market forces had already resulted in a series of newer card designs with such improved performance that the Talisman cards simply couldn't compete. Cards with large amounts of RAM arranged to allow for extremely high speeds solved the bandwidth issue, simply brute forcing the problem instead of attempting to solve it through clever implementation.

Additionally, the Talisman concept required tight integration between the display system and the software using it. Unlike the new 3D cards coming to market at the time, Talisman systems would have to be able to ask the CPU to re-render portions of the image in order to update their chunks. This required the games to have a specific organization in memory in order to respond to these requests. In order to aid developers in this task, Direct3D was changed to more closely match the Talisman needs. However, for any game that had already been written, or those that didn't want to be tied to Talisman, this made the D3D system slower and considerably less interesting.

Disappearance

As a result of these changes, Talisman never became a commercial product. Cirrus Logic and Samsung both gave up on the system some time in 1997, leading Microsoft to abandon plans to release Escalante in 1997, and to external observers it appeared the entire project was dead.^[5]

There was a brief rebirth soon after, however, when Fujitsu claimed to be working on a single-chip implementation that would be available in 1998, with rumors of similar projects at S3 Graphics and ATI Technologies.^[6] None of these systems ever shipped and Talisman was quietly killed. This was much to the delight of the 3rd party graphics accelerator vendors, as well as the people within Microsoft that supported them in the market with DirectX.

Legacy

Nevertheless, several of the ideas pioneered in the Talisman system have since become common in most accelerators. In particular, texture compression is now widely used. On more recent cards, compression has also been used on the z-buffers to reduce memory demands while sorting the display. The idea of using "chunks" to sort the display has also been used in a small number of cards, referred to as tile based rendering, but like Talisman in general these have never become competitive in the desktop space due to the rapid changes in the market. However, many recent graphics processors specifically designed for mobile devices (such as cell phones) employ a tile-based approach. Only the one key idea of Talisman, asking for updates to geometry only "when needed", has not been attempted since.

References

1. ^ Allen Ballman, "What is Talisman?", Microsoft Research, SIGGRAPH 1996
2. ^ Combined Issue:Microsoft Talisman "Repackages" Chapel Hill Concept
3. ^ Jay Torborg and James Kajiy, "Talisman: Commodity Real-time 3D graphics for the PC", SIGGRAPH 1996
4. ^ Francis Vale, Intel MMX vs. Microsoft Talisman: Abbott and Costello Do Multimedia, 21st; The VXM Network, 1997
5. ^ Francis Vale, Talisman, Part II: Microsoft Still Doesn't Get the 3D picture, 21st; The VXM Network, 1997
6. ^ Mark Hachman, F"ujitsu To Bring Microsoft's Talisman To Life", Electronic Buyer's News, 16 September 1998

External links

• Chicken Crossing, short movie rendered in realtime using Talisman concepts, presented at SIGGRAPH '96