Microsoft Talisman
Encyclopedia
Talisman was a Microsoft
project to build a new 3D graphics architecture based on quickly compositing 2D "sub-images" onto the screen, an adaptation of tiled rendering
. In theory, this approach would dramatically reduce the amount of memory bandwidth
required for 3D games and thereby lead to lower-cost graphics accelerators. The project took place during the introduction of the first high-performance 3D accelerators, and these quickly surpassed Talisman in both performance and price. No Talisman-based systems were ever released commercially, and the project was eventually cancelled in the late 1990s.
Typical 3D accelerators of the mid-90s helped only with one step of this process, compositing the image. The cards had memory buffers for the textures that were loaded up only as required, and were then instructed to composite those textures onto geometry being handed to it by the CPU. The CPU was still responsible for all of the other work, mostly the conversion of the model objects coordinates into the world coordinates, and then sending the resulting models to the card as a series of polygons. By offloading the "easy part" of the operation to the card, the CPU could spend more time working on the models, allowing the complexity of the scenes to improve. As the textures were often the memory intensive part of the system, placing them semi-permanently on the card in high-speed memory allowed the compositing steps to be greatly sped up.
In this mode of operation every frame was created though what was essentially brute force. As soon as one frame had completed drawing, the CPU would start the process over, sending in a new set of polygons and asking the card to composite and render them into a framebuffer
for eventual display. There was no memory of the already rendered objects, so even objects that did not move in the display were nevertheless re-rendered from scratch for every frame. Although the CPU's workload was reduced, the bandwidth needed to send these more complex models to the cards was relatively high.
Much greater improvements in performance could be had if the graphics card also stored the polygons. However, these had to be re-created from scratch every time the scene changed or the user's viewpoint (the "camera") changed. This was common, changing almost every frame, whereas textures would generally change only when new models were introduced. Adding this functionality on the card was certainly possible, but it would require the graphics card to have considerable processing power, on par with the computer's CPU. Avoiding this complexity on the card meant that the number of polygons visible on the display had to be limited to what could be comfortably sent to the graphics card every time the frame updated.
A typical gaming display of the era using 640 x 480 resolution with 24-bit color, with basic 3D compositing with trilinear filtering
and no anti-aliasing
, would require 1,900 MB/s of memory bandwidth. The same rendering settings on a 1024 x 768 display would require 14,200 MB/s, and even basic anti-aliasing would be expected to roughly double that figure. For reference, SGI
's then-current RealityEngine2
machines featured a then-high memory bandwidth of about 10,000 MB/s, which was the reason these machines were widely used in 3D graphics. A typical PC of the era using AGP 2X could offer only 508 MB/s.
To produce the illusion of 3D, each of the individual tiles were manipulated through an affine transform before compositing, allowing the image to appear to "move" in space, at least in terms of perspective changes. The host CPU would only be asked to re-render a particular tile when the changes in perspective were great enough that the transform could no longer result in an artifact-free image. In that case the CPU would run its transformations again, handing the card a new model to be rendered back into the buffer. Microsoft calculated that each tile could be re-used for about four frames on average, thereby reducing load on the CPU by about four times.
The Talisman image buffers were broken down into 32 x 32 pixel "chunks" that were individually rendered using the 3D objects and textures provided by the CPU. The card would determine which objects overlapped which chunks, transforming those objects into 2D objects describing the chunk, which was then compressed and stored. Pointers to the chunks were then stored in a z-ordered (front to back) list for every 32 scan-lines on the display. One concern is that the chunks cannot be cleanly "stitched together", a problem that has sometimes been visible in various videogames using software rendering
. To avoid this, Talisman also stored a separate "edge buffer" for every chunk that stored an "overflow" area that would cover gaps in the mapping.
The system would respond by selecting the next strip and retrieving any chunks that were visible in that strip given the current camera location. In the typical case many of the chunks would be obscured by other chunks, and could be ignored during compositing, saving time. This is the reason for the z-sorting of the chunks, which allows them to be efficiently retrieved in "visibility order". If the chunks could be modified without distortion, the proper affine transform was called to update the chunk in-place. If it could not, say because the camera had moved too much since the last full update, the CPU was asked to provide new geometry for that chunk, which the card then rendered and placed back in storage.
Talisman had no analog of a framebuffer, rendering chunks on demand directly to the screen as the monitor's scan line progressed down the screen. This is an interesting analog with the Atari 2600
, which uses a similar system to render 2D images on the screen, a method known as "racing the beam". In both cases, this reduced the amount of memory needed, and the memory bandwidth being used between the display system and video hardware. In both cases this also required dramatically tighter integration between the video system and the programs running it. In the case of Talisman, the programs were required to store their objects in a particular format that the Talisman software drivers understood, allowing it to be quickly picked up from memory during interrupt
s.
research lab at the University of North Carolina at Chapel Hill
can be considered Talisman's direct parent.
When Talisman was first made widely public at the 1996 SIGGRAPH
meeting, they promised a dramatic reduction in the cost of implementing a graphics subsystem. They planned on working with vendors to sell the concept of Talisman for inclusion into other companies' display systems. That is, Talisman was hoped to be a part of a larger media chip, as opposed to an entire 3D system that would stand alone in a system. Their basic system would support 20-30,000 polygons on a 1024 x 768 display at 32 bit/pixel, with a 40 Mpixel/s polygon rendering rate and 320 Mpixel/s image layer compositing rate.
and 3DO
were working together to design a single-chip DSP
-like "Media Signal Processor" (MSP), combining Talisman functionality with additional media functionality. Cirrus Logic
would provide a VLSI chip that would retrieve data placed in memory by the MSP, apply effects, and send it off for display. Known as the "Polygon Object Processor" (POP), this chip was periodically polled by another Cirrus Logic chip, the "Image Layer Compositor" (ILC), which was tied to the video circuitry. Additionally, Escalante intended to feature 4 MB of RDRAM
on two 600 MHz 8-bit channels, offering 1.2 GB/s throughput. Later Philips
entered the fray with a planned new version of their TriMedia
processor, which implemented most of Talisman in a single CPU, and Trident Microsystems
, with similar plans.
It was in the midst of the Talisman project that the first person shooter genre started to come to the fore in gaming. This created market demand for accelerators that could be used with existing games with minimal changes. By the time the Escalante reference design was ready for production, the market forces had already resulted in a series of newer card designs with such improved performance that the Talisman cards simply couldn't compete. Cards with large amounts of RAM
arranged to allow for extremely high speeds solved the bandwidth issue, simply brute forcing the problem instead of attempting to solve it through clever implementation.
Additionally, the Talisman concept required tight integration between the display system and the software using it. Unlike the new 3D cards coming to market at the time, Talisman systems would have to be able to ask the CPU to re-render portions of the image in order to update their chunks. This required the games to have a specific organization in memory in order to respond to these requests. In order to aid developers in this task, Direct3D
was changed to more closely match the Talisman needs. However, for any game that had already been written, or those that didn't want to be tied to Talisman, this made the D3D system slower and considerably less interesting.
There was a brief rebirth soon after, however, when Fujitsu claimed to be working on a single-chip implementation that would be available in 1998, with rumors of similar projects at S3 Graphics
and ATI Technologies
. None of these systems ever shipped and Talisman was quietly killed. This was much to the delight of the 3rd party graphics accelerator vendors, as well as the people within Microsoft that supported them in the market with DirectX
.
, but like Talisman in general these have never become competitive in the desktop space due to the rapid changes in the market. However, many recent graphics processors specifically designed for mobile devices (such as cell phones) employ a tile-based approach. Only the one key idea of Talisman, asking for updates to geometry only "when needed", has not been attempted since.
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
project to build a new 3D graphics architecture based on quickly compositing 2D "sub-images" onto the screen, an adaptation of tiled rendering
Tiled rendering
Tiled rendering is the process of subdividing a computer graphics image by a regular grid in image space to exploit local spatial coherence in the scene and/or to facilitate the use of limited hardware rendering resources later in the graphics pipeline...
. In theory, this approach would dramatically reduce the amount of memory bandwidth
Memory bandwidth
Memory bandwidth is the rate at which data can be read from or stored into a semiconductor memory by a processor. Memory bandwidth is usually expressed in units of bytes/second, though this can vary for systems with natural data sizes that are not a multiple of the commonly used 8-bit bytes.Memory...
required for 3D games and thereby lead to lower-cost graphics accelerators. The project took place during the introduction of the first high-performance 3D accelerators, and these quickly surpassed Talisman in both performance and price. No Talisman-based systems were ever released commercially, and the project was eventually cancelled in the late 1990s.
Conventional 3D
Creating a 3D image for display consists of a series of steps. First, the objects to be displayed are loaded up into memory from individual "models". The display system then applies mathematical functions to transform the objects into a common coordinate system, the "world view". From this world view, a series of polygons (typically triangles) is created that approximates the original models as seen from a particular viewpoint, the "camera". Next, a compositing system produces an image by rendering the triangles and applying "textures" to the outside. Textures are small images that are painted onto the triangles to produce realism. The resulting image is them combined with various special effects, and moved into the display buffers. Of this process, the "display pipeline", only the geometry tends to change frequently, due to changes in the camera location, while the models and textures generally only change over much longer periods of time.Typical 3D accelerators of the mid-90s helped only with one step of this process, compositing the image. The cards had memory buffers for the textures that were loaded up only as required, and were then instructed to composite those textures onto geometry being handed to it by the CPU. The CPU was still responsible for all of the other work, mostly the conversion of the model objects coordinates into the world coordinates, and then sending the resulting models to the card as a series of polygons. By offloading the "easy part" of the operation to the card, the CPU could spend more time working on the models, allowing the complexity of the scenes to improve. As the textures were often the memory intensive part of the system, placing them semi-permanently on the card in high-speed memory allowed the compositing steps to be greatly sped up.
In this mode of operation every frame was created though what was essentially brute force. As soon as one frame had completed drawing, the CPU would start the process over, sending in a new set of polygons and asking the card to composite and render them into a framebuffer
Framebuffer
A framebuffer is a video output device that drives a video display from a memory buffer containing a complete frame of data.The information in the memory buffer typically consists of color values for every pixel on the screen...
for eventual display. There was no memory of the already rendered objects, so even objects that did not move in the display were nevertheless re-rendered from scratch for every frame. Although the CPU's workload was reduced, the bandwidth needed to send these more complex models to the cards was relatively high.
Much greater improvements in performance could be had if the graphics card also stored the polygons. However, these had to be re-created from scratch every time the scene changed or the user's viewpoint (the "camera") changed. This was common, changing almost every frame, whereas textures would generally change only when new models were introduced. Adding this functionality on the card was certainly possible, but it would require the graphics card to have considerable processing power, on par with the computer's CPU. Avoiding this complexity on the card meant that the number of polygons visible on the display had to be limited to what could be comfortably sent to the graphics card every time the frame updated.
A typical gaming display of the era using 640 x 480 resolution with 24-bit color, with basic 3D compositing with trilinear filtering
Trilinear filtering
Trilinear filtering is an extension of the bilinear texture filtering method, which also performs linear interpolation between mipmaps.Bilinear filtering has several weaknesses that make it an unattractive choice in many cases: using it on a full-detail texture when scaling to a very small size...
and no anti-aliasing
Anti-aliasing
In digital signal processing, spatial anti-aliasing is the technique of minimizing the distortion artifacts known as aliasing when representing a high-resolution image at a lower resolution...
, would require 1,900 MB/s of memory bandwidth. The same rendering settings on a 1024 x 768 display would require 14,200 MB/s, and even basic anti-aliasing would be expected to roughly double that figure. For reference, SGI
Silicon Graphics
Silicon Graphics, Inc. was a manufacturer of high-performance computing solutions, including computer hardware and software, founded in 1981 by Jim Clark...
's then-current RealityEngine2
RealityEngine
RealityEngine refers to a 3D graphics hardware architecture and a family of graphics systems that implemented the aforementioned hardware architecture that was developed and manufactured by Silicon Graphics during the early to mid 1990s...
machines featured a then-high memory bandwidth of about 10,000 MB/s, which was the reason these machines were widely used in 3D graphics. A typical PC of the era using AGP 2X could offer only 508 MB/s.
Tiled rendering
Talisman attempted to reduce the memory bandwidth requirements primarily by leaving unchanged post-rendered objects in memory to be re-composited without change. The system used a series of memory buffers and parallel compositing engines to quickly "assemble" a display that was made up of a number of 2D images, or "tiles". This technique is immediately suitable for 2D images, as any possible change in the display mapping can be made by individual mappings on the tiles. The same is not true from 3D, however, where rotations of the user's viewpoint requires objects in the display to rotate in 3D, something that can only be simulated by modifying a 2D image.To produce the illusion of 3D, each of the individual tiles were manipulated through an affine transform before compositing, allowing the image to appear to "move" in space, at least in terms of perspective changes. The host CPU would only be asked to re-render a particular tile when the changes in perspective were great enough that the transform could no longer result in an artifact-free image. In that case the CPU would run its transformations again, handing the card a new model to be rendered back into the buffer. Microsoft calculated that each tile could be re-used for about four frames on average, thereby reducing load on the CPU by about four times.
The Talisman image buffers were broken down into 32 x 32 pixel "chunks" that were individually rendered using the 3D objects and textures provided by the CPU. The card would determine which objects overlapped which chunks, transforming those objects into 2D objects describing the chunk, which was then compressed and stored. Pointers to the chunks were then stored in a z-ordered (front to back) list for every 32 scan-lines on the display. One concern is that the chunks cannot be cleanly "stitched together", a problem that has sometimes been visible in various videogames using software rendering
Software rendering
In the context of rendering , software rendering refers to a rendering process that is unaided by any specialized graphics hardware, such as a graphics card. The rendering takes place entirely in the CPU...
. To avoid this, Talisman also stored a separate "edge buffer" for every chunk that stored an "overflow" area that would cover gaps in the mapping.
Rendering pipeline
In a conventional 3D system, geometry is periodically generated, sent to the card for composition, composed into a framebuffer, and then eventually picked up by the video hardware for display. Talisman systems essentially reversed this process; the screen was divided into the same 32-high scan-line strips, and while the video hardware was drawing one of these strips, the hardware would call the Talisman side and tell it to prepare the details for the next strip.The system would respond by selecting the next strip and retrieving any chunks that were visible in that strip given the current camera location. In the typical case many of the chunks would be obscured by other chunks, and could be ignored during compositing, saving time. This is the reason for the z-sorting of the chunks, which allows them to be efficiently retrieved in "visibility order". If the chunks could be modified without distortion, the proper affine transform was called to update the chunk in-place. If it could not, say because the camera had moved too much since the last full update, the CPU was asked to provide new geometry for that chunk, which the card then rendered and placed back in storage.
Talisman had no analog of a framebuffer, rendering chunks on demand directly to the screen as the monitor's scan line progressed down the screen. This is an interesting analog with the Atari 2600
Atari 2600
The Atari 2600 is a video game console released in October 1977 by Atari, Inc. It is credited with popularizing the use of microprocessor-based hardware and cartridges containing game code, instead of having non-microprocessor dedicated hardware with all games built in...
, which uses a similar system to render 2D images on the screen, a method known as "racing the beam". In both cases, this reduced the amount of memory needed, and the memory bandwidth being used between the display system and video hardware. In both cases this also required dramatically tighter integration between the video system and the programs running it. In the case of Talisman, the programs were required to store their objects in a particular format that the Talisman software drivers understood, allowing it to be quickly picked up from memory during interrupt
Interrupt
In computing, an interrupt is an asynchronous signal indicating the need for attention or a synchronous event in software indicating the need for a change in execution....
s.
Introduction
The Talisman effort was Microsoft's attempt to commercialize concepts that had been experimented on for some time. In particular, the PixelFlow system developed at a Hewlett-PackardHewlett-Packard
Hewlett-Packard Company or HP is an American multinational information technology corporation headquartered in Palo Alto, California, USA that provides products, technologies, softwares, solutions and services to consumers, small- and medium-sized businesses and large enterprises, including...
research lab at the University of North Carolina at Chapel Hill
University of North Carolina at Chapel Hill
The University of North Carolina at Chapel Hill is a public research university located in Chapel Hill, North Carolina, United States...
can be considered Talisman's direct parent.
When Talisman was first made widely public at the 1996 SIGGRAPH
SIGGRAPH
SIGGRAPH is the name of the annual conference on computer graphics convened by the ACM SIGGRAPH organization. The first SIGGRAPH conference was in 1974. The conference is attended by tens of thousands of computer professionals...
meeting, they promised a dramatic reduction in the cost of implementing a graphics subsystem. They planned on working with vendors to sell the concept of Talisman for inclusion into other companies' display systems. That is, Talisman was hoped to be a part of a larger media chip, as opposed to an entire 3D system that would stand alone in a system. Their basic system would support 20-30,000 polygons on a 1024 x 768 display at 32 bit/pixel, with a 40 Mpixel/s polygon rendering rate and 320 Mpixel/s image layer compositing rate.
Escalante
At the time, Microsoft was working with several vendors in order to develop a reference implementation known as Escalante. SamsungSamsung
The Samsung Group is a South Korean multinational conglomerate corporation headquartered in Samsung Town, Seoul, South Korea...
and 3DO
The 3DO Company
The 3DO Company , also known as 3DO , was a video game company...
were working together to design a single-chip DSP
Digital signal processor
A digital signal processor is a specialized microprocessor with an architecture optimized for the fast operational needs of digital signal processing.-Typical characteristics:...
-like "Media Signal Processor" (MSP), combining Talisman functionality with additional media functionality. Cirrus Logic
Cirrus Logic
Cirrus Logic is a fabless semiconductor supplier specializing in analog, mixed-signal, and audio DSP integrated circuits . They are presently headquartered in Austin, Texas. Their audio processors and audio converters are found in many professional audio and consumer entertainment products,...
would provide a VLSI chip that would retrieve data placed in memory by the MSP, apply effects, and send it off for display. Known as the "Polygon Object Processor" (POP), this chip was periodically polled by another Cirrus Logic chip, the "Image Layer Compositor" (ILC), which was tied to the video circuitry. Additionally, Escalante intended to feature 4 MB of RDRAM
RDRAM
Direct Rambus DRAM or DRDRAM is a type of synchronous dynamic RAM. RDRAM was developed by Rambus inc., in the mid-1990s as a replacement for then-prevalent DIMM SDRAM memory architecture....
on two 600 MHz 8-bit channels, offering 1.2 GB/s throughput. Later Philips
Philips
Koninklijke Philips Electronics N.V. , more commonly known as Philips, is a multinational Dutch electronics company....
entered the fray with a planned new version of their TriMedia
TriMedia
TriMedia can refer to:* TriMedia , a media processor made by Philips/NXP Semiconductors* Trimedia International, a European public-relations agency* Tri-Media Productions, a Philipino TV production companySimilarly named pages:...
processor, which implemented most of Talisman in a single CPU, and Trident Microsystems
Trident Microsystems
Trident Microsystems is a supplier of display-processors for flat panel displays . At one time, Trident was also a supplier of PC graphics chipsets and sound controllers.- History :...
, with similar plans.
It was in the midst of the Talisman project that the first person shooter genre started to come to the fore in gaming. This created market demand for accelerators that could be used with existing games with minimal changes. By the time the Escalante reference design was ready for production, the market forces had already resulted in a series of newer card designs with such improved performance that the Talisman cards simply couldn't compete. Cards with large amounts of RAM
Ram
-Animals:*Ram, an uncastrated male sheep*Ram cichlid, a species of freshwater fish endemic to Colombia and Venezuela-Military:*Battering ram*Ramming, a military tactic in which one vehicle runs into another...
arranged to allow for extremely high speeds solved the bandwidth issue, simply brute forcing the problem instead of attempting to solve it through clever implementation.
Additionally, the Talisman concept required tight integration between the display system and the software using it. Unlike the new 3D cards coming to market at the time, Talisman systems would have to be able to ask the CPU to re-render portions of the image in order to update their chunks. This required the games to have a specific organization in memory in order to respond to these requests. In order to aid developers in this task, Direct3D
Direct3D
Direct3D is part of Microsoft's DirectX application programming interface . Direct3D is available for Microsoft Windows operating systems , and for other platforms through the open source software Wine. It is the base for the graphics API on the Xbox and Xbox 360 console systems...
was changed to more closely match the Talisman needs. However, for any game that had already been written, or those that didn't want to be tied to Talisman, this made the D3D system slower and considerably less interesting.
Disappearance
As a result of these changes, Talisman never became a commercial product. Cirrus Logic and Samsung both gave up on the system some time in 1997, leading Microsoft to abandon plans to release Escalante in 1997, and to external observers it appeared the entire project was dead.There was a brief rebirth soon after, however, when Fujitsu claimed to be working on a single-chip implementation that would be available in 1998, with rumors of similar projects at S3 Graphics
S3 Graphics
S3 Graphics, Ltd is an American company specializing in graphics chipsets. Although they do not have the large market share that they once had, they still produce graphics accelerators for home computers under the "S3 Chrome" brand name.-History:...
and ATI Technologies
ATI Technologies
ATI Technologies Inc. was a semiconductor technology corporation based in Markham, Ontario, Canada, that specialized in the development of graphics processing units and chipsets. Founded in 1985 as Array Technologies Inc., the company was listed publicly in 1993 and was acquired by Advanced Micro...
. None of these systems ever shipped and Talisman was quietly killed. This was much to the delight of the 3rd party graphics accelerator vendors, as well as the people within Microsoft that supported them in the market with DirectX
DirectX
Microsoft DirectX is a collection of application programming interfaces for handling tasks related to multimedia, especially game programming and video, on Microsoft platforms. Originally, the names of these APIs all began with Direct, such as Direct3D, DirectDraw, DirectMusic, DirectPlay,...
.
Legacy
Nevertheless, several of the ideas pioneered in the Talisman system have since become common in most accelerators. In particular, texture compression is now widely used. On more recent cards, compression has also been used on the z-buffers to reduce memory demands while sorting the display. The idea of using "chunks" to sort the display has also been used in a small number of cards, referred to as tile based renderingTiled rendering
Tiled rendering is the process of subdividing a computer graphics image by a regular grid in image space to exploit local spatial coherence in the scene and/or to facilitate the use of limited hardware rendering resources later in the graphics pipeline...
, but like Talisman in general these have never become competitive in the desktop space due to the rapid changes in the market. However, many recent graphics processors specifically designed for mobile devices (such as cell phones) employ a tile-based approach. Only the one key idea of Talisman, asking for updates to geometry only "when needed", has not been attempted since.
External links
- Chicken Crossing, short movie rendered in realtime using Talisman concepts, presented at SIGGRAPH '96