Subpixel rendering
Encyclopedia
Subpixel rendering is a way to increase the apparent resolution of a computer's liquid crystal display
(LCD) or Organic Light Emitting Diode display by rendering pixels to take into account the screen type's physical properties. It takes advantage of the fact that each pixel
on a colour LCD is actually composed of individual red, green, and blue or other color subpixels to anti-alias
text with greater detail or to increase the resolution of all image types on layouts which are specifically designed to be compatible with subpixel rendering.
A single pixel on a colour subpixelated display is made of several colour primaries, typically three coloured elements—ordered (on various displays) either as blue, green, and red (BGR), or as red, green, and blue (RGB). Some displays have more than three primaries, often called MultiPrimary, such as the combination of red, green, blue, and yellow (RGBY), or red, green, blue, and white (RGBW), or even red, green, blue, yellow, and cyan (RGBYC).
These pixel components, sometimes called sub-pixels, appear as a single colour to the human eye because of blurring by the optics and spatial integration by nerve cells in the eye. The components are easily visible, however, when viewed with a small magnifying glass, such as a loupe
. Over a certain resolution threshold the colours in the sub-pixels are not visible, but the relative intensity of the components shifts the apparent position or orientation of a line.
Methods that take this interaction between the display technology and the human visual system into account are called subpixel rendering algorithms. The resolution at which coloured sub-pixels go unnoticed differs, however, with each user—some users are distracted by the coloured "fringes" resulting from sub-pixel rendering.
Subpixel rendering is better suited to some display technologies than others. The technology is well-suited to LCDs and other technologies where each logical pixel corresponds directly to three or more independent coloured sub-pixels, but less so for CRT
s.
In a CRT the light from the pixel components often spreads across pixels, and the outputs of adjacent pixels are not perfectly independent. If a designer knew precisely about the display's electron beams and aperture grille
, subpixel rendering might have some advantage. But the properties of the CRT components, coupled with the alignment variations that are part of the production process, make subpixel rendering less effective for these displays.
The technique should have good application to organic light emitting diodes
and other display technologies that organize pixels the same way as LCDs.
in 1988, subpixel rendering was first brought to public attention by Microsoft
in Windows XP
as ClearType
, but it was not activated by default until Windows Vista
. Mac OS X
uses subpixel rendering as well, except in the "Best for CRT" setting, which applies traditional anti-aliasing. In stealth mode
from 1992 to 1999, Candice H. Brown Elliott was researching subpixel rendering and developing novel layouts, the PenTile Matrix family, that worked together with the subpixel rendering algorithms to increase the resolution of colour flat panel displays. In 2000, she co-founded Clairvoyante, Inc. to commercialize these layouts and subpixel rendering algorithms. In 2008, Samsung
purchased Clairvoyante and simultaneously funded a new company, Nouvoyance, Inc., retaining much of the technical staff, with Ms. Brown Elliott as CEO.
supported an early form of subpixel rendering in its high-resolution (280x192) graphics mode. However, the method Gibson describes can also be viewed as a limitation of the way the machine generates colour, rather than as a technique intentionally exploited by programmers to increase resolution.
The bytes that comprise the Apple II high-resolution screen buffer
contain seven visible bits (each corresponding directly to a pixel) and a flag bit used to select between purple/green or blue/orange colour sets. Each pixel, since it is represented by a single bit, is either on or off; there are no bits within the pixel itself for specifying colour or brightness. Colour is instead created as an artifact of the NTSC
colour encoding scheme, determined by horizontal position: pixels with even horizontal coordinates are always purple (or blue, if the flag bit is set), and odd pixels are always green (or orange). Two lit pixels next to each other are always white, regardless of whether the pair is even/odd or odd/even, and irrespective of the value of the flag bit. The foregoing is only an approximation of the true interplay of the digital and analog behavior of the Apple's video output circuits on one hand, and the properties of real NTSC monitors on the other hand. However, this approximation was what most programmers of the time would have in mind while working with the Apple's high-resolution mode.
In Gibson's example, then, the programmer is not necessarily placing purple and green pixels to increase the perceived resolution of a white line; he may also be seen as simply drawing a line two pixels wide so as to make it appear white, and this latter mental model is arguably the one that most programmers of the time used. If a diagonal line were only one pixel wide, it would appear alternately purple and green as it meandered down the screen between even and odd horizontal coordinates. While the quote from Apple II inventor Steve Wozniak
on Gibson's page seems to imply that Apple II graphics programmers routinely used subpixel rendering, it is difficult to make a case that many of them thought of what they were doing in such terms.
The flag bit in each byte affects colour by shifting pixels half a pixel-width to the right. This half-pixel shift was exploited by some graphics software, such as HRCG (High-Resolution Character Generator), an Apple utility that displayed text using the high-resolution graphics mode, to smooth diagonals. (Many Apple II users had monochrome displays, or turned down the saturation on their colour displays when running software that expected a monochrome display, so this technique was useful.) Although it did not provide a way to address subpixels individually, it did allow positioning of pixels at fractional pixel locations and can thus be considered a form of subpixel rendering. However, this technique is not related to LCD subpixel rendering as described in this article.
distortion is the primary benefit of subpixel rendered fonts on the conventional RGB Stripe panel.
Although subpixel rendering increases the number of reconstruction points on the display this does not always mean that higher resolution, higher spatial frequencies, more lines and spaces, may be displayed on a given arrangement of colour subpixels. A phenomenon occurs as the spatial frequency is increased past the whole pixel Nyquist limit from the Nyquist–Shannon sampling theorem
; Chromatic aliasing (colour fringes) may appear with higher spatial frequencies in a given orientation on the colour subpixel arrangement.
RGBRGBRGBRGBRGBRGB WWWWWWWWWWWWWWWWWW R = red
RGBRGBRGBRGBRGBRGB is WWWWWWWWWWWWWWWWWW G = green
RGBRGBRGBRGBRGBRGB perceived WWWWWWWWWWWWWWWWWW where B = blue
RGBRGBRGBRGBRGBRGB as WWWWWWWWWWWWWWWWWW W = white
RGBRGBRGBRGBRGBRGB WWWWWWWWWWWWWWWWWW
Shown below is an example of black and white lines at the Nyquist limit, but at a slanting angle, taking advantage of sub-pixel rendering to use a different phase each row:
RGB___RGB___RGB___ WWW___WWW___WWW___ R = red
_GBR___GBR___GBR__ is _WWW___WWW___WWW__ G = green
__BRG___BRG___BRG_ perceived __WWW___WWW___WWW_ where B = blue
___RGB___RGB___RGB as ___WWW___WWW___WWW _ = black
____GBR___GBR___GB ____WWW___WWW___WW W = white
Shown below is an example of chromatic aliasing when the traditional whole pixel Nyquist limit is exceeded:
RG__GB__BR__RG__GB YY__CC__MM__YY__CC R = red Y = yellow
RG__GB__BR__RG__GB is YY__CC__MM__YY__CC G = green C = cyan
RG__GB__BR__RG__GB perceived YY__CC__MM__YY__CC where B = blue M = magenta
RG__GB__BR__RG__GB as YY__CC__MM__YY__CC _ = black
RG__GB__BR__RG__GB YY__CC__MM__YY__CC
This case shows the result of attempting to place vertical black and white lines at four sub-pixels per cycle on the RGB Stripe architecture. One can visually see that the lines, instead of being white, are colored. Starting from the left, the first line is red combined with green to produce a yellow-colored line. The second line is green combined with blue to produce a pastel cyan-colored line. The third line is blue combined with red to produce a magenta-colored line. The colors then repeat: yellow, cyan, and magenta. This demonstrates that a spatial frequency of one cycle per four sub-pixels is too high. Attempts to go to a yet higher spatial frequency, such as one cycle per three sub-pixels, would result in a single solid color.
Some LCD compensate the inter-pixel color mix effect by having borders between pixels slightly larger than borders between sub-pixels. Then, in example above, viewer of such LCD would see that a blue line appears adjacent to a red line than to see a single magenta line.
RBGRBGRBGRBGRBGRBG
GBRGBRGBRGBRGBRGBR
RBGRBGRBGRBGRBGRBG
GBRGBRGBRGBRGBRGBR
RBGRBGRBGRBGRBGRBG
GBRGBRGBRGBRGBRGBR
In this case, the red and green order are interchanged every row to create a red & green checkerboard pattern with blue stripes. Note that the vertical subpixels could be split in half vertically to double the vertical resolution as well : the current LCD panels already typically use two colour LEDs (aligned vertically and displaying the same lightness, see the zoomed images below) to illuminate each vertical subpixel. This layout is one of the PenTile matrix family of layouts. When displaying the same number of black white lines, the blue subpixels are set at half brightness "b":
Rb_Rb_Rb_Rb_Rb_Rb_
Gb_Gb_Gb_Gb_Gb_Gb_
Rb_Rb_Rb_Rb_Rb_Rb_
Gb_Gb_Gb_Gb_Gb_Gb_
Rb_Rb_Rb_Rb_Rb_Rb_
Gb_Gb_Gb_Gb_Gb_Gb_
Notice that every column that turns on comprises red and green subpixels at full brightness and blue subpixels at half value to balance it to white. Now, one may display black and white lines at up to one cycle per three subpixels without chromatic aliasing, twice that of the RGB Stripe architecture.
) as members of the PenTile Matrix Family of layouts specifically designed for subpixel rendering efficiency.
For example, taking profit of the doubled visible horizontal resolution, one could double the vertical resolution to make the definition more isotropic. However this would reduce the aperture of pixels, producing lower contrasts. A better alternative uses the fact that the blue subpixels are those that contribute the least to the visible intensity, so that they are less precisely located by the eye. Blue subpixels are then rendered just as a diamond in the center of a pixel square, and the rest of the pixel surface is split in four parts as a checker board of red and green subpixels with smaller sizes. Rendering images with this variant can use the same technique as before, except that now there's a near-isotropic geometry that supports both the horizontal and the vertical with the same geometric properties, making the layout ideal for displaying the same image details when the LCD panel can be rotated.
The doubled vertical and horizontal visual resolution allows to reduce the subpixel density of about 33%, in order to increase their aperture also of about 33%, with the same separation distance between subpixels (for their electronic interconnection), And also to reduce the power dissipation of about 50% with a white/black contrast increased of about 50% and still a visual-pixel resolution enhanced by about 33% (i.e. about 125 dpi instead of 96 dpi), but with only half the total number of subpixels for the same displayed surface.
Pattern with a white subpixel, to increase the contrast and reduce the energy needed to illuminate white pixels (because colour filters in classic RGB striped panels absorb more than 65% of the total white light used to illuminate the panel. As each subpixel is a square instead of a thin rectangle, this also increases the aperture with the same average subpixel density, and same pixel density along both axis. As the horizontal density is reduced and the vertical density remains identical (for the same square pixel density), it becomes possible to increase the pixel density of about 33%, while maintaining the contrast comparable to classic RGB or BGR panels, taking profit of the more efficient use of light and lowered absorption levels by the colour filters.
It is not possible to use subpixel rendering to increase the resolution without creating colour fringes similar to those seen in classic RGB or BGR striped panels, but the increased resolution compensates it (in addition, their effective visible colour is reduced by the presence of "colour-neutral" white subpixels.
However, this layout allows a better rendering of greys (and more exact rendering of CMYK-based colours), at the price of a lower colour separation. But this is consistent with human vision and with modern image and video compression formats (like JPEG
and MPEG) used in modern HDTV transmissions and in Blu-ray Disc
s.
Yet another variant, a member for the PenTile Matrix Family of subpixel layouts, alternates between subpixel order RGBW / BWRG every other row, to allow subpixel rendering to increase the resolution, without chromatic aliasing. As before, the increased transmittance using the white subpixel allows higher subpixel density, but in this case, the displayed resolution is even higher due to the benefits of subpixel rendering:
RGBWRGBWRGBW
BWRGBWRGBWRG
RGBWRGBWRGBW
BWRGBWRGBWRG
RGB_RGB_RGB_
_W___W___W__
RGB_RGB_RGB_
_W___W___W__
However, such alternate layouts are still not compatible with subpixel rendering font algorithms used in Windows, Mac OS X and Linux, which currently support only the RGB or BGR striped subpixel layouts. However, the PenTile Matrix displays have a built-in subpixel rendering engine that allows conventional RGB data sets to be converted to the layouts, providing plug'n'play compatibility with conventional layout displays. New display models should be proposed in the future that will allow monitor drivers to specify : their visual resolution separately from the full pixel resolution and the relative position offsets of visible subpixels for each colour plane, as well as their respective contribution to white intensity. Such monitor drivers would allow renderers to correctly adjust their geometry transform matrixes in order to correctly compute the values of each colour plane, and take the best profit of subpixel rendering with the lowest chromatic aliasing.
in the United States on subpixel rendering technology for text rendering on RGB Stripe layouts. This has caused FreeType
, the library used by most current software on the X Window System
, to disable this functionality by default. Apple is able to use it in Mac OS X due to a patent cross-licensing agreement.
The patents 6,219,025, 6,239,783, 6,307,566, 6,225,973, 6,243,070, 6,393,145, 6,421,054, 6,282,327, 6,624,828 are filed between 1998-10-07 - 1999-10-07. And thus should expire in 2019-10-07.
Since May 2010, the patents US5155805, US5159668 and US5325479 have expired worldwide and thus FreeType has its bytecode interpreter enabled by default.
Clairvoyante, Inc. filed for approximately 100 patents on subpixel rendering and layouts specifically designed to be compatible with subpixel rendering (the PenTile Matrix Family). In March 2008, Samsung bought Clairvoyante and all of their patents. The co-founder of Clairvoyante, Candice Brown Elliott, then started Nouvoyance to continue development of the technology in partnership with Samsung.
Note that FreeType with the subpixel rendering functionality shown in these examples cannot be distributed in the United States due to the patents mentioned above.
The composite photographs below show three methods of font rendering for comparison. From top: Monochrome; Traditional (whole pixel) Anti-Aliasing
; Subpixel rendering.
Liquid crystal display
A liquid crystal display is a flat panel display, electronic visual display, or video display that uses the light modulating properties of liquid crystals . LCs do not emit light directly....
(LCD) or Organic Light Emitting Diode display by rendering pixels to take into account the screen type's physical properties. It takes advantage of the fact that each pixel
Pixel
In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....
on a colour LCD is actually composed of individual red, green, and blue or other color subpixels to anti-alias
Anti-aliasing
In digital signal processing, spatial anti-aliasing is the technique of minimizing the distortion artifacts known as aliasing when representing a high-resolution image at a lower resolution...
text with greater detail or to increase the resolution of all image types on layouts which are specifically designed to be compatible with subpixel rendering.
Background
A single pixel on a colour subpixelated display is made of several colour primaries, typically three coloured elements—ordered (on various displays) either as blue, green, and red (BGR), or as red, green, and blue (RGB). Some displays have more than three primaries, often called MultiPrimary, such as the combination of red, green, blue, and yellow (RGBY), or red, green, blue, and white (RGBW), or even red, green, blue, yellow, and cyan (RGBYC).
These pixel components, sometimes called sub-pixels, appear as a single colour to the human eye because of blurring by the optics and spatial integration by nerve cells in the eye. The components are easily visible, however, when viewed with a small magnifying glass, such as a loupe
Loupe
A loupe is a simple, small magnification device used to see small details more closely. Unlike a magnifying glass, a loupe does not have an attached handle, and its focusing lens are contained in an opaque cylinder or cone. Loupes are also called hand lenses .- Optics :Three basic types of loupes...
. Over a certain resolution threshold the colours in the sub-pixels are not visible, but the relative intensity of the components shifts the apparent position or orientation of a line.
Methods that take this interaction between the display technology and the human visual system into account are called subpixel rendering algorithms. The resolution at which coloured sub-pixels go unnoticed differs, however, with each user—some users are distracted by the coloured "fringes" resulting from sub-pixel rendering.
Subpixel rendering is better suited to some display technologies than others. The technology is well-suited to LCDs and other technologies where each logical pixel corresponds directly to three or more independent coloured sub-pixels, but less so for CRT
Cathode ray tube
The cathode ray tube is a vacuum tube containing an electron gun and a fluorescent screen used to view images. It has a means to accelerate and deflect the electron beam onto the fluorescent screen to create the images. The image may represent electrical waveforms , pictures , radar targets and...
s.
In a CRT the light from the pixel components often spreads across pixels, and the outputs of adjacent pixels are not perfectly independent. If a designer knew precisely about the display's electron beams and aperture grille
Aperture grille
An aperture grille is one of two major technologies used to manufacture color cathode ray tube televisions and computer displays; the other is shadow mask....
, subpixel rendering might have some advantage. But the properties of the CRT components, coupled with the alignment variations that are part of the production process, make subpixel rendering less effective for these displays.
The technique should have good application to organic light emitting diodes
Organic light-emitting diode
An OLED is a light-emitting diode in which the emissive electroluminescent layer is a film of organic compounds which emit light in response to an electric current. This layer of organic semiconductor material is situated between two electrodes...
and other display technologies that organize pixels the same way as LCDs.
History
Originally invented by IBMIBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
in 1988, subpixel rendering was first brought to public attention by Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
in Windows XP
Windows XP
Windows XP is an operating system produced by Microsoft for use on personal computers, including home and business desktops, laptops and media centers. First released to computer manufacturers on August 24, 2001, it is the second most popular version of Windows, based on installed user base...
as ClearType
ClearType
ClearType is a trademark for Microsoft's implementation of subpixel rendering technology. ClearType attempts to improve the appearance of text on certain types of computer display screens by sacrificing color fidelity for additional intensity variation. This trade-off is asserted to work well on...
, but it was not activated by default until Windows Vista
Windows Vista
Windows Vista is an operating system released in several variations developed by Microsoft for use on personal computers, including home and business desktops, laptops, tablet PCs, and media center PCs...
. Mac OS X
Mac OS X
Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
uses subpixel rendering as well, except in the "Best for CRT" setting, which applies traditional anti-aliasing. In stealth mode
Stealth mode
In business, stealth mode is a company's temporary state of secretiveness, usually undertaken in order to avoid alerting competitors to a pending product launch or other business initiative...
from 1992 to 1999, Candice H. Brown Elliott was researching subpixel rendering and developing novel layouts, the PenTile Matrix family, that worked together with the subpixel rendering algorithms to increase the resolution of colour flat panel displays. In 2000, she co-founded Clairvoyante, Inc. to commercialize these layouts and subpixel rendering algorithms. In 2008, Samsung
Samsung
The Samsung Group is a South Korean multinational conglomerate corporation headquartered in Samsung Town, Seoul, South Korea...
purchased Clairvoyante and simultaneously funded a new company, Nouvoyance, Inc., retaining much of the technical staff, with Ms. Brown Elliott as CEO.
Subpixel Rendering and the Apple II
It is sometimes claimed (e.g. by Steve Gibson) that the Apple IIApple II
The Apple II is an 8-bit home computer, one of the first highly successful mass-produced microcomputer products, designed primarily by Steve Wozniak, manufactured by Apple Computer and introduced in 1977...
supported an early form of subpixel rendering in its high-resolution (280x192) graphics mode. However, the method Gibson describes can also be viewed as a limitation of the way the machine generates colour, rather than as a technique intentionally exploited by programmers to increase resolution.
The bytes that comprise the Apple II high-resolution screen buffer
Screen buffer
In computing, screen buffer is a part of computer memory used by a computer application for the representation of the content to be shown on the computer display....
contain seven visible bits (each corresponding directly to a pixel) and a flag bit used to select between purple/green or blue/orange colour sets. Each pixel, since it is represented by a single bit, is either on or off; there are no bits within the pixel itself for specifying colour or brightness. Colour is instead created as an artifact of the NTSC
NTSC
NTSC, named for the National Television System Committee, is the analog television system that is used in most of North America, most of South America , Burma, South Korea, Taiwan, Japan, the Philippines, and some Pacific island nations and territories .Most countries using the NTSC standard, as...
colour encoding scheme, determined by horizontal position: pixels with even horizontal coordinates are always purple (or blue, if the flag bit is set), and odd pixels are always green (or orange). Two lit pixels next to each other are always white, regardless of whether the pair is even/odd or odd/even, and irrespective of the value of the flag bit. The foregoing is only an approximation of the true interplay of the digital and analog behavior of the Apple's video output circuits on one hand, and the properties of real NTSC monitors on the other hand. However, this approximation was what most programmers of the time would have in mind while working with the Apple's high-resolution mode.
In Gibson's example, then, the programmer is not necessarily placing purple and green pixels to increase the perceived resolution of a white line; he may also be seen as simply drawing a line two pixels wide so as to make it appear white, and this latter mental model is arguably the one that most programmers of the time used. If a diagonal line were only one pixel wide, it would appear alternately purple and green as it meandered down the screen between even and odd horizontal coordinates. While the quote from Apple II inventor Steve Wozniak
Steve Wozniak
Stephen Gary "Woz" Wozniak is an American computer engineer and programmer who founded Apple Computer, Co. with Steve Jobs and Ronald Wayne...
on Gibson's page seems to imply that Apple II graphics programmers routinely used subpixel rendering, it is difficult to make a case that many of them thought of what they were doing in such terms.
The flag bit in each byte affects colour by shifting pixels half a pixel-width to the right. This half-pixel shift was exploited by some graphics software, such as HRCG (High-Resolution Character Generator), an Apple utility that displayed text using the high-resolution graphics mode, to smooth diagonals. (Many Apple II users had monochrome displays, or turned down the saturation on their colour displays when running software that expected a monochrome display, so this technique was useful.) Although it did not provide a way to address subpixels individually, it did allow positioning of pixels at fractional pixel locations and can thus be considered a form of subpixel rendering. However, this technique is not related to LCD subpixel rendering as described in this article.
Addressability vs. resolution
With subpixel rendering technology, the number of points that may be independently addressed to reconstruct the image is increased. When the green subpixels are reconstructing the shoulders, the red subpixels are reconstructing near the peaks and vice versa. For text fonts, increasing the addressability allows the font designer to use spatial frequencies and phases that would have created noticeable distortions had it been whole pixel rendered. The improvement is most noted on italic fonts which exhibit different phases on each row. This reduction in moiréMoiré pattern
In physics, a moiré pattern is an interference pattern created, for example, when two grids are overlaid at an angle, or when they have slightly different mesh sizes.- Etymology :...
distortion is the primary benefit of subpixel rendered fonts on the conventional RGB Stripe panel.
Although subpixel rendering increases the number of reconstruction points on the display this does not always mean that higher resolution, higher spatial frequencies, more lines and spaces, may be displayed on a given arrangement of colour subpixels. A phenomenon occurs as the spatial frequency is increased past the whole pixel Nyquist limit from the Nyquist–Shannon sampling theorem
Nyquist–Shannon sampling theorem
The Nyquist–Shannon sampling theorem, after Harry Nyquist and Claude Shannon, is a fundamental result in the field of information theory, in particular telecommunications and signal processing. Sampling is the process of converting a signal into a numeric sequence...
; Chromatic aliasing (colour fringes) may appear with higher spatial frequencies in a given orientation on the colour subpixel arrangement.
Example with the common RGB stripes layout
For example, consider an RGB Stripe Panel:RGBRGBRGBRGBRGBRGB WWWWWWWWWWWWWWWWWW R = red
RGBRGBRGBRGBRGBRGB is WWWWWWWWWWWWWWWWWW G = green
RGBRGBRGBRGBRGBRGB perceived WWWWWWWWWWWWWWWWWW where B = blue
RGBRGBRGBRGBRGBRGB as WWWWWWWWWWWWWWWWWW W = white
RGBRGBRGBRGBRGBRGB WWWWWWWWWWWWWWWWWW
Shown below is an example of black and white lines at the Nyquist limit, but at a slanting angle, taking advantage of sub-pixel rendering to use a different phase each row:
RGB___RGB___RGB___ WWW___WWW___WWW___ R = red
_GBR___GBR___GBR__ is _WWW___WWW___WWW__ G = green
__BRG___BRG___BRG_ perceived __WWW___WWW___WWW_ where B = blue
___RGB___RGB___RGB as ___WWW___WWW___WWW _ = black
____GBR___GBR___GB ____WWW___WWW___WW W = white
Shown below is an example of chromatic aliasing when the traditional whole pixel Nyquist limit is exceeded:
RG__GB__BR__RG__GB YY__CC__MM__YY__CC R = red Y = yellow
RG__GB__BR__RG__GB is YY__CC__MM__YY__CC G = green C = cyan
RG__GB__BR__RG__GB perceived YY__CC__MM__YY__CC where B = blue M = magenta
RG__GB__BR__RG__GB as YY__CC__MM__YY__CC _ = black
RG__GB__BR__RG__GB YY__CC__MM__YY__CC
This case shows the result of attempting to place vertical black and white lines at four sub-pixels per cycle on the RGB Stripe architecture. One can visually see that the lines, instead of being white, are colored. Starting from the left, the first line is red combined with green to produce a yellow-colored line. The second line is green combined with blue to produce a pastel cyan-colored line. The third line is blue combined with red to produce a magenta-colored line. The colors then repeat: yellow, cyan, and magenta. This demonstrates that a spatial frequency of one cycle per four sub-pixels is too high. Attempts to go to a yet higher spatial frequency, such as one cycle per three sub-pixels, would result in a single solid color.
Some LCD compensate the inter-pixel color mix effect by having borders between pixels slightly larger than borders between sub-pixels. Then, in example above, viewer of such LCD would see that a blue line appears adjacent to a red line than to see a single magenta line.
Example with RBG-GBR alternated stripes layout
Novel subpixel layouts have been developed to allow higher real resolution without chromatic aliasing. Shown here is one of the member of the PenTile Matrix family of layouts. Shown below is an example of how a simple change to the arrangement of colour subpixels may allow a higher limit in the horizontal direction:RBGRBGRBGRBGRBGRBG
GBRGBRGBRGBRGBRGBR
RBGRBGRBGRBGRBGRBG
GBRGBRGBRGBRGBRGBR
RBGRBGRBGRBGRBGRBG
GBRGBRGBRGBRGBRGBR
In this case, the red and green order are interchanged every row to create a red & green checkerboard pattern with blue stripes. Note that the vertical subpixels could be split in half vertically to double the vertical resolution as well : the current LCD panels already typically use two colour LEDs (aligned vertically and displaying the same lightness, see the zoomed images below) to illuminate each vertical subpixel. This layout is one of the PenTile matrix family of layouts. When displaying the same number of black white lines, the blue subpixels are set at half brightness "b":
Rb_Rb_Rb_Rb_Rb_Rb_
Gb_Gb_Gb_Gb_Gb_Gb_
Rb_Rb_Rb_Rb_Rb_Rb_
Gb_Gb_Gb_Gb_Gb_Gb_
Rb_Rb_Rb_Rb_Rb_Rb_
Gb_Gb_Gb_Gb_Gb_Gb_
Notice that every column that turns on comprises red and green subpixels at full brightness and blue subpixels at half value to balance it to white. Now, one may display black and white lines at up to one cycle per three subpixels without chromatic aliasing, twice that of the RGB Stripe architecture.
Non-striped variants of the RBG-GBR alternated layout
Variants of the previous layout have been proposed by Clairvoyante/Nouvoyance (and demonstrated by SamsungSamsung
The Samsung Group is a South Korean multinational conglomerate corporation headquartered in Samsung Town, Seoul, South Korea...
) as members of the PenTile Matrix Family of layouts specifically designed for subpixel rendering efficiency.
For example, taking profit of the doubled visible horizontal resolution, one could double the vertical resolution to make the definition more isotropic. However this would reduce the aperture of pixels, producing lower contrasts. A better alternative uses the fact that the blue subpixels are those that contribute the least to the visible intensity, so that they are less precisely located by the eye. Blue subpixels are then rendered just as a diamond in the center of a pixel square, and the rest of the pixel surface is split in four parts as a checker board of red and green subpixels with smaller sizes. Rendering images with this variant can use the same technique as before, except that now there's a near-isotropic geometry that supports both the horizontal and the vertical with the same geometric properties, making the layout ideal for displaying the same image details when the LCD panel can be rotated.
The doubled vertical and horizontal visual resolution allows to reduce the subpixel density of about 33%, in order to increase their aperture also of about 33%, with the same separation distance between subpixels (for their electronic interconnection), And also to reduce the power dissipation of about 50% with a white/black contrast increased of about 50% and still a visual-pixel resolution enhanced by about 33% (i.e. about 125 dpi instead of 96 dpi), but with only half the total number of subpixels for the same displayed surface.
Checkered RG-BW layout
Another variant, called the RGBW Quad, uses a checkerboard with 4 subpixels per pixel, adding a white subpixel, or more specifically, replacing one of the green subpixels of Bayer filterBayer filter
A Bayer filter mosaic is a color filter array for arranging RGB color filters on a square grid of photosensors. Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image...
Pattern with a white subpixel, to increase the contrast and reduce the energy needed to illuminate white pixels (because colour filters in classic RGB striped panels absorb more than 65% of the total white light used to illuminate the panel. As each subpixel is a square instead of a thin rectangle, this also increases the aperture with the same average subpixel density, and same pixel density along both axis. As the horizontal density is reduced and the vertical density remains identical (for the same square pixel density), it becomes possible to increase the pixel density of about 33%, while maintaining the contrast comparable to classic RGB or BGR panels, taking profit of the more efficient use of light and lowered absorption levels by the colour filters.
It is not possible to use subpixel rendering to increase the resolution without creating colour fringes similar to those seen in classic RGB or BGR striped panels, but the increased resolution compensates it (in addition, their effective visible colour is reduced by the presence of "colour-neutral" white subpixels.
However, this layout allows a better rendering of greys (and more exact rendering of CMYK-based colours), at the price of a lower colour separation. But this is consistent with human vision and with modern image and video compression formats (like JPEG
JPEG
In computing, JPEG . The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. JPEG typically achieves 10:1 compression with little perceptible loss in image quality....
and MPEG) used in modern HDTV transmissions and in Blu-ray Disc
Blu-ray Disc
Blu-ray Disc is an optical disc storage medium designed to supersede the DVD format. The plastic disc is 120 mm in diameter and 1.2 mm thick, the same size as DVDs and CDs. Blu-ray Discs contain 25 GB per layer, with dual layer discs being the norm for feature-length video discs...
s.
Yet another variant, a member for the PenTile Matrix Family of subpixel layouts, alternates between subpixel order RGBW / BWRG every other row, to allow subpixel rendering to increase the resolution, without chromatic aliasing. As before, the increased transmittance using the white subpixel allows higher subpixel density, but in this case, the displayed resolution is even higher due to the benefits of subpixel rendering:
RGBWRGBWRGBW
BWRGBWRGBWRG
RGBWRGBWRGBW
BWRGBWRGBWRG
RGB_RGB_RGB_
_W___W___W__
RGB_RGB_RGB_
_W___W___W__
Visual resolution versus pixel resolution and software compatibility
Thus, not all layouts are created equal. Each particular layout may have a different “visual resolution”, Modulation Transfer Function Limit (MTFL), defined as the highest number of black and white lines that may be simultaneously rendered without visible chromatic aliasing.However, such alternate layouts are still not compatible with subpixel rendering font algorithms used in Windows, Mac OS X and Linux, which currently support only the RGB or BGR striped subpixel layouts. However, the PenTile Matrix displays have a built-in subpixel rendering engine that allows conventional RGB data sets to be converted to the layouts, providing plug'n'play compatibility with conventional layout displays. New display models should be proposed in the future that will allow monitor drivers to specify : their visual resolution separately from the full pixel resolution and the relative position offsets of visible subpixels for each colour plane, as well as their respective contribution to white intensity. Such monitor drivers would allow renderers to correctly adjust their geometry transform matrixes in order to correctly compute the values of each colour plane, and take the best profit of subpixel rendering with the lowest chromatic aliasing.
Patents
Microsoft has several patentsSoftware patent
Software patent does not have a universally accepted definition. One definition suggested by the Foundation for a Free Information Infrastructure is that a software patent is a "patent on any performance of a computer realised by means of a computer program".In 2005, the European Patent Office...
in the United States on subpixel rendering technology for text rendering on RGB Stripe layouts. This has caused FreeType
FreeType
FreeType is a software library written in C that implements a font rasterization engine. It is used to render text on to bitmaps and provides support for other font-related operations.-Details:...
, the library used by most current software on the X Window System
X Window System
The X window system is a computer software system and network protocol that provides a basis for graphical user interfaces and rich input device capability for networked computers...
, to disable this functionality by default. Apple is able to use it in Mac OS X due to a patent cross-licensing agreement.
The patents 6,219,025, 6,239,783, 6,307,566, 6,225,973, 6,243,070, 6,393,145, 6,421,054, 6,282,327, 6,624,828 are filed between 1998-10-07 - 1999-10-07. And thus should expire in 2019-10-07.
Since May 2010, the patents US5155805, US5159668 and US5325479 have expired worldwide and thus FreeType has its bytecode interpreter enabled by default.
Clairvoyante, Inc. filed for approximately 100 patents on subpixel rendering and layouts specifically designed to be compatible with subpixel rendering (the PenTile Matrix Family). In March 2008, Samsung bought Clairvoyante and all of their patents. The co-founder of Clairvoyante, Candice Brown Elliott, then started Nouvoyance to continue development of the technology in partnership with Samsung.
Examples
Photos were taken with a Canon PowerShot A470 digital camera using "Super Macro" mode and 4.0x digital zoom. The screen used was that integrated into a Lenovo G550 laptop. Note that the display has RGB pixels. Displays exist in all four patterns horizontal RGB/BGR and vertical RGB/BGR but horizontal RGB is the most common. In addition, several colour subpixel patterns have been developed specifically to take advantage of subpixel rendering. The best known of these is the PenTile matrix family of patterns.Note that FreeType with the subpixel rendering functionality shown in these examples cannot be distributed in the United States due to the patents mentioned above.
The composite photographs below show three methods of font rendering for comparison. From top: Monochrome; Traditional (whole pixel) Anti-Aliasing
Anti-aliasing
In digital signal processing, spatial anti-aliasing is the technique of minimizing the distortion artifacts known as aliasing when representing a high-resolution image at a lower resolution...
; Subpixel rendering.
See also
- CoolTypeCoolTypeCoolType is a software technology, designed by Adobe Systems to increase the legibility of text on color LCDs like laptop or TFT monitors, especially to make reading long text, like E-Books, easier...
- Font rasterizationFont rasterizationFont rasterization is the process of converting text from a vector description to a raster or bitmap description. This often involves some anti-aliasing on screen text to make it smoother and easier to read...
- Kell factorKell factorThe Kell factor, named after RCA engineer Raymond D. Kell, is a parameter used to limit the bandwidth of a sampled image signal to avoid the appearance of beat frequency patterns when displaying the image in a discrete display devices, usually taken to be 0.7. The number was first measured in 1934...
- PenTile Matrix FamilyPenTile matrix familyPenTile matrix family refers to a family of patented subpixel matrix schemes used in electronic device displays. PenTile is a trademark of Samsung....
- Sub-pixel resolutionSub-pixel resolutionIn Digital Image Processing, Sub-pixel resolution can be obtained in digital images containing well defined lines, points or edges that can be processed by an algorithm to reliably measure the position of the line, point or edge in the image with an accuracy exceeding the nominal pixel resolution...
External links
- Sub-Pixel Font Rendering Technology: History and Explanation by Steve Gibson, includes free downloadable Windows demo.
- SubLCD, a free non patented subpixel method.
- Pixel Borrowing, ClearType and Antialiasing
- Texts Rasterization Exposures Article from the Anti-Grain Geometry Project.