Virtual retinal display
Encyclopedia
A virtual retinal display (VRD), also known as a retinal scan display (RSD) or retinal projector (RP), is a display technology that draws a raster
display (like a television
) directly onto the retina
of the eye. The user sees what appears to be a conventional display floating in space in front of them. (However, the portion of the visual area where imagery appears must still intersect with optical elements of the display system. It is not possible to display an image over a solid angle from a point source unless the projection system can bypass the lenses within the eye.)
To create an image with the VRD a photon source (or three sources in the case of a color display) is used to generate a coherent beam of light. The use of a coherent source (such as a laser diode) allows the system to draw a diffraction limited spot on the retina. The light beam is intensity modulated to match the intensity of the image being rendered. The modulation can be accomplished after the beam is generated. If the source has enough modulation bandwidth, as in the case of a laser diode, the source can be modulated directly.
The resulting modulated beam is then scanned to place each image point, or pixel, at the proper position on the retina. A variety of scan patterns are possible. The scanner could be used in a calligraphic (vector) mode, in which the lines that form the image are drawn directly, or in a raster mode, much like standard computer monitors or television. Use of the raster method of image scanning and allows the VRD to be driven by standard video sources. To draw the raster, a horizontal scanner moves the beam to draw a row of pixels. The vertical scanner then moves the beam to the next line where another row of pixels is drawn.
After scanning, the optical beam must be properly projected into the eye. The goal is for the exit pupil of the VRD to be coplanar with the entrance pupil of the eye. The lens and cornea of the eye will then focus the beam on the retina, forming a spot. The position on the retina where the eye focuses the spot is determined by the angle at which light enters the eye. This angle is determined by the scanners and is continually varying in a raster pattern. The brightness of the focused spot is determined by the intensity modulation of the light beam. The intensity modulated moving spot, focused through the eye, draws an image on the retina. The eye's persistence allows the image to appear continuous and stable.
Finally, the drive electronics synchronize the scanners and intensity modulator with the incoming video signal in such a manner that a stable image is formed.
With a VRD, defects in the eye's optical system, such as damaged cornea and lens and reduced retinal sensitivity could be bypassed, as well as the problems of the display environment, such as ambient brightness, angle-of-view and display brightness. Additionally, the seen image could be augmented with other information.
Although the VRD is an output device, the technology lends itself to augmentation with eye tracking or eyegaze systems for input. Eye tracking is currently used in advanced still and video cameras for focusing on the object you wish to record.
This approach produces several advantages over conventional display devices:
A point source emits waves of light which radiate in ever-expanding circles about the point. The pupil of an eye, looking at the source, will see a small portion of the wavefront. The curvature of the wavefront as it enters the pupil is determined by the distance of the eye from the source. As the source moves farther away, less curvature is exhibited by the wavefronts. It is the wavefront curvature which determines where the eye must focus in order to create a sharp image.
If the eye is an infinite distance from the source, plane waves enter the pupil. The lens of the eye images the plane waves to a spot on the retina. The spot size is limited by the aberrations in the lens of the eye and by the diffraction of the light through the pupil. It is the angle at which the plane wave enters the eye that determines where on the retina the spot is formed. Two points focus to different spots on the retina because the wavefronts from the points are intersecting the pupil at different angles.
Neglecting the aberrations in the lens of the eye, one can determine the limit of the eye's resolution based on diffraction through the pupil. Using Rayleigh's criteria the minimum angular resolution is computed as follows:
Only recently a number of developments have made a true VRD system practical. In particular the development of high-brightness LED
s have made the displays bright enough to be used during the day, and adaptive optics
have allowed systems to dynamically correct for irregularities in the eye (although this is not always needed). The result is a high-resolution screenless
display with excellent color gamut and brightness, far better than the best television technologies.
The VRD was invented by Kazuo Yoshinaka of Nippon Electric Co. in 1986. Later work at the University of Washington
in the Human Interface Technology Lab resulted in a similar system in 1991. Most of the research into VRDs to date has been in combination with various virtual reality
systems. In this role VRDs have the potential advantage of being much smaller than existing television-based systems. They share some of the same disadvantages however, requiring some sort of optics to send the image into the eye, typically similar to the sunglasses system used with previous technologies. It also can be used as part of a wearable computer
system.
More recently, there has been some interest in VRDs as a display system for portable devices such as cell phones, PDAs
and various media players. In this role the device would be placed in front of the user, perhaps on a desk, and aimed in the general direction of the eyes. The system would then detect the eye using facial scanning techniques and keep the image in place using motion compensation
. In this role the VRD offers unique advantages, being able to replicate a full-sized monitor on a small device.
VRD system also can show an image in each eye with a very little angle difference for simulating three-dimensional scenes with high fidelity spectral colours. If applied to video games, for instance, gamers could have an enhanced sense of reality that liquid-crystal-display glasses could never provide, because the VRD can refocus dynamically to simulate near and distant objects with a far superior level of realism.
This system only generates essentially needed photons, and as such it is more efficient for mobile devices that are only designed to serve a single user. A VRD could potentially use tens or hundreds of times less power for Mobile Telephone and Netbook based applications.
Another important advantage is privacy: Only the intended user (in the usual case of single-user devices) is able to see the image displayed. This kind of device is also less vulnerable to TEMPEST
type side-channel leak of information.
To ensure that VRD device is safe, rigorous safety standards from the American National Standards Institute and the International Electrotechnical Commission were applied to the development of such systems. Optical damage caused by lasers comes from its tendency to concentrate its power in a very narrow area. This problem is overcome in VRD systems as they are scanned, constantly shifting from point to point with the beams focus.
If the laser stops scanning, permanent damage to the eye will result because the beam stays focused in one spot. This can be prevented by an emergency safety system to detect the situation and shut it off.
. However no VRD-based system has yet reached operational use and current military HMD development now appears focused on other technologies such as holographic waveguide optics.
or computed tomography
scans assist in surgical navigation.
Raster graphics
In computer graphics, a raster graphics image, or bitmap, is a data structure representing a generally rectangular grid of pixels, or points of color, viewable via a monitor, paper, or other display medium...
display (like a television
Television
Television is a telecommunication medium for transmitting and receiving moving images that can be monochrome or colored, with accompanying sound...
) directly onto the retina
Retina
The vertebrate retina is a light-sensitive tissue lining the inner surface of the eye. The optics of the eye create an image of the visual world on the retina, which serves much the same function as the film in a camera. Light striking the retina initiates a cascade of chemical and electrical...
of the eye. The user sees what appears to be a conventional display floating in space in front of them. (However, the portion of the visual area where imagery appears must still intersect with optical elements of the display system. It is not possible to display an image over a solid angle from a point source unless the projection system can bypass the lenses within the eye.)
The basic system
In a conventional display a real image is produced. The real image is either viewed directly or, as in the case with most head-mounted displays, projected through an optical system and the resulting virtual image is viewed. The projection moves the virtual image to a distance that allows the eye to focus comfortably. No real image is ever produced with the VRD. Rather, an image is formed directly on the retina of the user's eye. A block diagram of the VRD is shown in Figure.To create an image with the VRD a photon source (or three sources in the case of a color display) is used to generate a coherent beam of light. The use of a coherent source (such as a laser diode) allows the system to draw a diffraction limited spot on the retina. The light beam is intensity modulated to match the intensity of the image being rendered. The modulation can be accomplished after the beam is generated. If the source has enough modulation bandwidth, as in the case of a laser diode, the source can be modulated directly.
The resulting modulated beam is then scanned to place each image point, or pixel, at the proper position on the retina. A variety of scan patterns are possible. The scanner could be used in a calligraphic (vector) mode, in which the lines that form the image are drawn directly, or in a raster mode, much like standard computer monitors or television. Use of the raster method of image scanning and allows the VRD to be driven by standard video sources. To draw the raster, a horizontal scanner moves the beam to draw a row of pixels. The vertical scanner then moves the beam to the next line where another row of pixels is drawn.
After scanning, the optical beam must be properly projected into the eye. The goal is for the exit pupil of the VRD to be coplanar with the entrance pupil of the eye. The lens and cornea of the eye will then focus the beam on the retina, forming a spot. The position on the retina where the eye focuses the spot is determined by the angle at which light enters the eye. This angle is determined by the scanners and is continually varying in a raster pattern. The brightness of the focused spot is determined by the intensity modulation of the light beam. The intensity modulated moving spot, focused through the eye, draws an image on the retina. The eye's persistence allows the image to appear continuous and stable.
Finally, the drive electronics synchronize the scanners and intensity modulator with the incoming video signal in such a manner that a stable image is formed.
Why VRDs
Liquid crystal displays (LCDs) currently are the primary active display devices for the presentation of entertainment and information. An image that is generated electronically is viewed with the optical system of the eye. The image you see is subject not only to the quality of the optical system of the eye, but also to the quality of the display and the environment in which the display is located.With a VRD, defects in the eye's optical system, such as damaged cornea and lens and reduced retinal sensitivity could be bypassed, as well as the problems of the display environment, such as ambient brightness, angle-of-view and display brightness. Additionally, the seen image could be augmented with other information.
Although the VRD is an output device, the technology lends itself to augmentation with eye tracking or eyegaze systems for input. Eye tracking is currently used in advanced still and video cameras for focusing on the object you wish to record.
This approach produces several advantages over conventional display devices:
- Potentially very small and lightweight, glasses mountable
- Large field and angle of view, greater than 120 degrees
- High resolution, approaching that of human vision
- Full color with better potential color resolution than conventional displays
- Brightness and contrast ratio sufficient for outdoor use
- True stereo 3D display with depth modulation
- Bypasses many of the eye's optical and retinal defects
The eye
A brief review of how the eye forms an image will aid in understanding the VRD.A point source emits waves of light which radiate in ever-expanding circles about the point. The pupil of an eye, looking at the source, will see a small portion of the wavefront. The curvature of the wavefront as it enters the pupil is determined by the distance of the eye from the source. As the source moves farther away, less curvature is exhibited by the wavefronts. It is the wavefront curvature which determines where the eye must focus in order to create a sharp image.
If the eye is an infinite distance from the source, plane waves enter the pupil. The lens of the eye images the plane waves to a spot on the retina. The spot size is limited by the aberrations in the lens of the eye and by the diffraction of the light through the pupil. It is the angle at which the plane wave enters the eye that determines where on the retina the spot is formed. Two points focus to different spots on the retina because the wavefronts from the points are intersecting the pupil at different angles.
Neglecting the aberrations in the lens of the eye, one can determine the limit of the eye's resolution based on diffraction through the pupil. Using Rayleigh's criteria the minimum angular resolution is computed as follows:
| Where |
|---|
| D = diameter of the pupil |
| lambda = wavelength of light |
History
In the past similar systems have been made by projecting a defocused image directly in front of the user's eye on a small "screen", normally in the form of large glasses. The user focused their eyes on the background, where the screen appeared to be floating. The disadvantage of these systems was the limited area covered by the "screen", the high weight of the small televisions used to project the display, and the fact that the image would appear focused only if the user was focusing at a particular "depth". Limited brightness made them useful only in indoor settings as well.Only recently a number of developments have made a true VRD system practical. In particular the development of high-brightness LED
LEd
LEd is a TeX/LaTeX editing software working under Microsoft Windows. It is a freeware product....
s have made the displays bright enough to be used during the day, and adaptive optics
Adaptive optics
Adaptive optics is a technology used to improve the performance of optical systems by reducing the effect of wavefront distortions. It is used in astronomical telescopes and laser communication systems to remove the effects of atmospheric distortion, and in retinal imaging systems to reduce the...
have allowed systems to dynamically correct for irregularities in the eye (although this is not always needed). The result is a high-resolution screenless
Screenless
Screenless video describes systems for transmitting visual information from a video source without the use of a screen. Screenless computing systems can be divided into three groups: Visual Image, Retinal Direct, and Synaptic Interface.- Visual image :...
display with excellent color gamut and brightness, far better than the best television technologies.
The VRD was invented by Kazuo Yoshinaka of Nippon Electric Co. in 1986. Later work at the University of Washington
University of Washington
University of Washington is a public research university, founded in 1861 in Seattle, Washington, United States. The UW is the largest university in the Northwest and the oldest public university on the West Coast. The university has three campuses, with its largest campus in the University...
in the Human Interface Technology Lab resulted in a similar system in 1991. Most of the research into VRDs to date has been in combination with various virtual reality
Virtual reality
Virtual reality , also known as virtuality, is a term that applies to computer-simulated environments that can simulate physical presence in places in the real world, as well as in imaginary worlds...
systems. In this role VRDs have the potential advantage of being much smaller than existing television-based systems. They share some of the same disadvantages however, requiring some sort of optics to send the image into the eye, typically similar to the sunglasses system used with previous technologies. It also can be used as part of a wearable computer
Wearable computer
Wearable computers are miniature electronic devices that are worn by the bearer under, with or on top of clothing. This class of wearable technology has been developed for general or special purpose information technologies and media development...
system.
More recently, there has been some interest in VRDs as a display system for portable devices such as cell phones, PDAs
Personal digital assistant
A personal digital assistant , also known as a palmtop computer, or personal data assistant, is a mobile device that functions as a personal information manager. Current PDAs often have the ability to connect to the Internet...
and various media players. In this role the device would be placed in front of the user, perhaps on a desk, and aimed in the general direction of the eyes. The system would then detect the eye using facial scanning techniques and keep the image in place using motion compensation
Motion compensation
Motion compensation is an algorithmic technique employed in the encoding of video data for video compression, for example in the generation of MPEG-2 files. Motion compensation describes a picture in terms of the transformation of a reference picture to the current picture. The reference picture...
. In this role the VRD offers unique advantages, being able to replicate a full-sized monitor on a small device.
Advantages
Apart from the advantages mentioned before, the VRD system scanning light into only one eye allows images to be laid over one's view of real objects. For example, it could project an animated, X-ray-like image of a car's engine or the human body.VRD system also can show an image in each eye with a very little angle difference for simulating three-dimensional scenes with high fidelity spectral colours. If applied to video games, for instance, gamers could have an enhanced sense of reality that liquid-crystal-display glasses could never provide, because the VRD can refocus dynamically to simulate near and distant objects with a far superior level of realism.
This system only generates essentially needed photons, and as such it is more efficient for mobile devices that are only designed to serve a single user. A VRD could potentially use tens or hundreds of times less power for Mobile Telephone and Netbook based applications.
Another important advantage is privacy: Only the intended user (in the usual case of single-user devices) is able to see the image displayed. This kind of device is also less vulnerable to TEMPEST
TEMPEST
TEMPEST is a codename referring to investigations and studies of compromising emission . Compromising emanations are defined as unintentional intelligence-bearing signals which, if intercepted and analyzed, may disclose the information transmitted, received, handled, or otherwise processed by any...
type side-channel leak of information.
Safety
It is believed that VRD based Laser or LED displays are not harmful to the human eye, as they are of a far lower intensity than those that are deemed hazardous to vision, the beam is spread over a greater surface area, and does not rest on a single point for an extended period of time.To ensure that VRD device is safe, rigorous safety standards from the American National Standards Institute and the International Electrotechnical Commission were applied to the development of such systems. Optical damage caused by lasers comes from its tendency to concentrate its power in a very narrow area. This problem is overcome in VRD systems as they are scanned, constantly shifting from point to point with the beams focus.
If the laser stops scanning, permanent damage to the eye will result because the beam stays focused in one spot. This can be prevented by an emergency safety system to detect the situation and shut it off.
LED enhancements
Although the power required is low, light must be collected and focused down in a point. This is an inherent property with lasers, but not so simple with a LED. Advances in LED technology will be needed to further concentrate the light coming from these devices.Military utilities
VRDs have been investigated for military use as an alternative display system for Helmet Mounted DisplaysHelmet mounted display
A helmet mounted display is a device used in some modern aircraft, especially combat aircraft. HMDs project information similar to that of head-up displays on an aircrew’s visor or reticle, thereby allowing him to obtain situational awareness and/or cue weapons systems to the direction his head...
. However no VRD-based system has yet reached operational use and current military HMD development now appears focused on other technologies such as holographic waveguide optics.
Medical utilities
A system similar to car repair procedures can be used by doctors for complex operations. While a surgeon is operating, he or she can keep track of vital patient data, such as blood pressure or heart rate, on a VRD. For procedures such as the placement of a catheter stent, overlaid images prepared from previously obtained magnetic resonance imagingMagnetic resonance imaging
Magnetic resonance imaging , nuclear magnetic resonance imaging , or magnetic resonance tomography is a medical imaging technique used in radiology to visualize detailed internal structures...
or computed tomography
Computed tomography
X-ray computed tomography or Computer tomography , is a medical imaging method employing tomography created by computer processing...
scans assist in surgical navigation.
Cameras
In the mass-market of digital cameras, scanned-beam displays provide better image quality at lower power and cost than liquid-crystal-on-silicon and organic LED displays.Manufacturers and commercial uses
- Microvision from the United StatesUnited StatesThe United States of America is a federal constitutional republic comprising fifty states and a federal district...
commercialized a VRD system named Nomad in 2001. - Brother IndustriesBrother Industriesis a diversified Japanese company that produces or imports a wide variety of products including printers, sewing machines, large machine tools, label printers, and typewriters, fax machines, and other computer-related electronics. It markets its multifunction printers as Multi-Function Centers...
from JapanJapanJapan is an island nation in East Asia. Located in the Pacific Ocean, it lies to the east of the Sea of Japan, China, North Korea, South Korea and Russia, stretching from the Sea of Okhotsk in the north to the East China Sea and Taiwan in the south...
exhibited VRD system named AirScouter in September 2010. - Brother IndustriesBrother Industriesis a diversified Japanese company that produces or imports a wide variety of products including printers, sewing machines, large machine tools, label printers, and typewriters, fax machines, and other computer-related electronics. It markets its multifunction printers as Multi-Function Centers...
announced the commercialisation of the Airscouter in August 2011.
See also
- Augmented RealityAugmented realityAugmented reality is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is...
- Head-mounted displayHead-mounted displayA head-mounted display or helmet mounted display, both abbreviated HMD, is a display device, worn on the head or as part of a helmet, that has a small display optic in front of one or each eye .- Overview :...
- List of emerging technologies
- Visual prosthetic
- Physics of the FuturePhysics of the FuturePhysics of the Future: How Science Will Shape Human Destiny and Our Daily Lives by the Year 2100 is a book by theoretical physicist Michio Kaku, author of Hyperspace and Physics of the Impossible. It speculates on the possibilities of future technological development over the next 100 years...