Xetal
Encyclopedia
Xetal is the name of a family of massively parallel
processors developed within Philips
Research and NXP Semiconductors.
(IC). With the image sensor and image processing combined on the same die
it is essentially possible to parallelize image processing up to the level where each pixel has its dedicated image processing logic. In such a design the image sensor would be in the upper layers of the IC while the image processing would be done in the lower layers, so image data would be transferred from one layer to the other, instead of through external pins or wires.
Additionally there is inherent parallelism
in image processing algorithms. Many algorithms do the same processing on every pixel. Image processing is therefore a suitable domain for a massively parallel approach using an SIMD
architecture. Although massive parallelism is not a new idea (earlier examples include ILLIAC IV
and Goodyear MPP
) the Xetal 1 was one of the first to apply this approach to image processing
.
with line-based A/D conversion. In this design, the analogue pixel values of the sensor were converted line by line (instead of pixel
by pixel). For every line there were 320 A/D converters. Each A/D converter is connected to a dedicated processing element (PE) to do image processing. This parallel design meant that a complete line of 320 pixels could essentially be processed in a single clock cycle. This parallelism was also applied to the memory architecture, where each processing element could access a pixel from a so-called /Line memory.
Simulations of this design showed that the digital (PE) part of the chip caused noise on the A/D converters. On top of that CMOS sensors at the time were produced using a 350-nm process using 3 metal layers. Few layers were used so as to limit height variations in the sensor surface which could cause artifacts. For discrete logic
the 180-nm process was more common. Also, more layers were used. Development of the CMOS sensor and the image processor therefore continued independently.
raw performance at this clock speed
is 5.7 GOPS (109 operations per second). As a result, combined with a CMOS image sensor at QVGA resolution running at 15 frames per second
the Xetal 1 could essentially perform 5000 operations per pixel.
During testing it turned out the Xetal 1 could even be clocked up to 38 MHz, more than double the original specification, resulting in a raw performance of over 12 GOPS. Moreover, it achieved this performance at very low power consumption (1-2 Watt
).
It was soon discovered that with these levels of performance it was possible to do much more than just image processing. The research team created demonstrations which showed that Xetal 1 was capable of running computer vision
algorithms such as object recognition and tracking, including a self-playing pinball machine
, air drumming
, and Robocup
robots.
Massively parallel
Massively parallel is a description which appears in computer science, life sciences, medical diagnostics, and other fields.A massively parallel computer is a distributed memory computer system which consists of many individual nodes, each of which is essentially an independent computer in itself,...
processors developed within Philips
Philips
Koninklijke Philips Electronics N.V. , more commonly known as Philips, is a multinational Dutch electronics company....
Research and NXP Semiconductors.
Background
The Xetal was conceived in 1999 at Philips Research when researchers investigated possibilities for combining a CMOS image sensor with powerful image processing logic. Since CMOS image sensors (contrary to CCD sensors) can be produced using the same manufacturing process as processors, both could be combined in a single integrated circuitIntegrated circuit
An integrated circuit or monolithic integrated circuit is an electronic circuit manufactured by the patterned diffusion of trace elements into the surface of a thin substrate of semiconductor material...
(IC). With the image sensor and image processing combined on the same die
Die (integrated circuit)
A die in the context of integrated circuits is a small block of semiconducting material, on which a given functional circuit is fabricated.Typically, integrated circuits are produced in large batches on a single wafer of electronic-grade silicon or other semiconductor through processes such as...
it is essentially possible to parallelize image processing up to the level where each pixel has its dedicated image processing logic. In such a design the image sensor would be in the upper layers of the IC while the image processing would be done in the lower layers, so image data would be transferred from one layer to the other, instead of through external pins or wires.
Additionally there is inherent parallelism
Parallel computing
Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently . There are several different forms of parallel computing: bit-level,...
in image processing algorithms. Many algorithms do the same processing on every pixel. Image processing is therefore a suitable domain for a massively parallel approach using an SIMD
SIMD
Single instruction, multiple data , is a class of parallel computers in Flynn's taxonomy. It describes computers with multiple processing elements that perform the same operation on multiple data simultaneously...
architecture. Although massive parallelism is not a new idea (earlier examples include ILLIAC IV
ILLIAC IV
The ILLIAC IV was one of the most infamous supercomputers ever built. One of a series of research machines, the ILLIACs from the University of Illinois, the ILLIAC IV design featured fairly high parallelism with up to 256 processors, used to allow the machine to work on large data sets in what...
and Goodyear MPP
Goodyear MPP
The Goodyear Massively Parallel Processor was amassively parallel processing supercomputer built by Goodyear Aerospacefor the NASA Goddard Space Flight Center.It was designed to deliver enormous computational power at lower cost than...
) the Xetal 1 was one of the first to apply this approach to image processing
Image processing
In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...
.
Initial design
The first design combined a QVGA image sensorImage sensor
An image sensor is a device that converts an optical image into an electronic signal. It is used mostly in digital cameras and other imaging devices...
with line-based A/D conversion. In this design, the analogue pixel values of the sensor were converted line by line (instead of pixel
Pixel
In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....
by pixel). For every line there were 320 A/D converters. Each A/D converter is connected to a dedicated processing element (PE) to do image processing. This parallel design meant that a complete line of 320 pixels could essentially be processed in a single clock cycle. This parallelism was also applied to the memory architecture, where each processing element could access a pixel from a so-called /Line memory.
Simulations of this design showed that the digital (PE) part of the chip caused noise on the A/D converters. On top of that CMOS sensors at the time were produced using a 350-nm process using 3 metal layers. Few layers were used so as to limit height variations in the sensor surface which could cause artifacts. For discrete logic
Logic gate
A logic gate is an idealized or physical device implementing a Boolean function, that is, it performs a logical operation on one or more logic inputs and produces a single logic output. Depending on the context, the term may refer to an ideal logic gate, one that has for instance zero rise time and...
the 180-nm process was more common. Also, more layers were used. Development of the CMOS sensor and the image processor therefore continued independently.
Xetal 1
The image processor resulting from this was the Xetal 1, first produced in 2001. It was manufactured using a 180-nm process and was designed to run at 18 MHz with 320 PEs and 16 line memories. Since each of the PEs can perform one operation per clock cycle theraw performance at this clock speed
Clock rate
The clock rate typically refers to the frequency that a CPU is running at.For example, a crystal oscillator frequency reference typically is synonymous with a fixed sinusoidal waveform, a clock rate is that frequency reference translated by electronic circuitry into a corresponding square wave...
is 5.7 GOPS (109 operations per second). As a result, combined with a CMOS image sensor at QVGA resolution running at 15 frames per second
Frame rate
Frame rate is the frequency at which an imaging device produces unique consecutive images called frames. The term applies equally well to computer graphics, video cameras, film cameras, and motion capture systems...
the Xetal 1 could essentially perform 5000 operations per pixel.
During testing it turned out the Xetal 1 could even be clocked up to 38 MHz, more than double the original specification, resulting in a raw performance of over 12 GOPS. Moreover, it achieved this performance at very low power consumption (1-2 Watt
Watt
The watt is a derived unit of power in the International System of Units , named after the Scottish engineer James Watt . The unit, defined as one joule per second, measures the rate of energy conversion.-Definition:...
).
It was soon discovered that with these levels of performance it was possible to do much more than just image processing. The research team created demonstrations which showed that Xetal 1 was capable of running computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
algorithms such as object recognition and tracking, including a self-playing pinball machine
Pinball
Pinball is a type of arcade game, usually coin-operated, where a player attempts to score points by manipulating one or more metal balls on a playfield inside a glass-covered case called a pinball machine. The primary objective of the game is to score as many points as possible...
, air drumming
Air guitar
Playing air guitar is a form of dance and movement in which the performer pretends to play rock or heavy metal-style electric guitar, including riffs, solos, etc. Playing an air guitar usually consists of exaggerated strumming and picking motions and is often coupled with loud singing or lip-synching...
, and Robocup
RoboCup
RoboCup is an international robotics competition founded in 1997. The aim is to develop autonomous soccer robots with the intention of promoting research and education in the field of artificial intelligence...
robots.