Plenoptic camera
Encyclopedia
A light-field camera, also called a plenoptic camera, is a camera
that uses a microlens
array to capture 4D light field information about a scene. Such light field information can be used to improve the solution of computer graphics
and vision-related problems.
in stereo matching. To achieve this, an array of microlenses is placed at the focal plane of the camera main lens. The image sensor
is positioned slightly behind the microlenses. Using such images the displacement of image parts that are not in focus can be analyzed and depth information can be extracted. Potentially, this camera system can be used to refocus an image virtually on a computer after the picture has been taken, as explained by Ng et al. The drawback of such a system is the low resolution
that the final images have. As one microlens samples the light directions at one spatial point an increase in the number of image pixels can only be done by increasing the number of microlenses by the same amount. To overcome this limitation, Lumsdaine and Georgiev describe a new design of a plenoptic camera, called focused plenoptic camera where the microlens array is positioned before or behind the focal plane of the main lens. This modification samples the light field in a different way that allows for a higher spatial resolution by having a lower angular resolution
at the same time. With this design images can be refocused with a much higher spatial resolution. However, the low angular resolution can introduce some unwanted aliasing artifacts. A different design using low cost printed film (mask) instead of microlens array was proposed by researchers at MERL in 2007. This design overcomes several limitations of microlens array in terms of chromatic aberration
s, loss of boundary pixels, and allows higher spatial resolution photos to be captured. However, a mask based design reduces light compared to microlens arrays.
Computer Graphics Laboratory used a 16 megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.
has sold several models of plenoptic camera for industrial and scientific applications, with resolutions starting from 1 megapixel.
Lytro
was founded by Stanford University Computer Graphics Laboratory alumnus Ren Ng
to commercialize the light field camera he developed as a graduate student there. Lytro has developed a consumer light-field "digital camera capable of images using a plenoptic technique. Orders were being taken in late 2011 for cameras to be delivered early in 2012.
light-field camera is a prototype 100-megapixel camera that takes a three-dimensional
photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2 megapixel photo of the entire scene around the camera and each image can be focused later in any way.
The CAFADIS camera is a plenoptic camera developed by the University of La Laguna
(Spain). CAFADIS stands (in Spanish) for phase-distance camera, since it can be used for distance and wavefront estimation. From a single shot it can produce several images refocused at different distances, depth maps, all-in-focus images and stereo pairs. A similar optical design can also be used in adaptive optics
in astrophysics
, in order to correct the aberrations caused by atmospheric turbulence in telescope
images. In order to perform these tasks different algorithms, running on GPU and FPGA, operate on the raw image captured by the camera.
Mitsubishi
Electric Research Labs's (MERL) light field camera is based on the principle of optical heterodyning and uses a printed film (mask) placed close to the sensor. Any hand-held camera can be converted into a light field camera using this technology by simply inserting a low-cost film on top of the sensor. A mask-based design avoids the problem of loss of resolution, since a high-resolution photo can be generated for the focused parts of the scene.
Eclipse transmitted light microscope/wide-field fluorescence microscope
and standard CCD cameras. Light-field capturing ability is obtained by a module containing a microlens array and other optical components placed in the light path between the objective lens and camera, with the final multifocused image rendered using deconvolution
. A later version of the prototype added a light-field illumination system consisting of a video projector (allowing computational control of illumination) and a second microlens array in the illumination light path of the microscope. The addition of a light-field illumination system both allowed for additional types of illumination (such as oblique illumination and quasi-dark-field) and correction for optical aberrations.
Camera
A camera is a device that records and stores images. These images may be still photographs or moving images such as videos or movies. The term camera comes from the camera obscura , an early mechanism for projecting images...
that uses a microlens
Microlens
A microlens is a small lens, generally with a diameter less than a millimetre and often as small as 10 micrometres . The small sizes of the lenses means that a simple design can give good optical quality but sometimes unwanted effects arise due to optical diffraction at the small features...
array to capture 4D light field information about a scene. Such light field information can be used to improve the solution of computer graphics
Computer graphics
Computer graphics are graphics created using computers and, more generally, the representation and manipulation of image data by a computer with help from specialized software and hardware....
and vision-related problems.
Technology
In 1992, Adelson and Wang proposed the design of a plenoptic camera that can be used to significantly reduce the correspondence problemCorrespondence problem
The correspondence problem tries to figure out which parts of an image correspond to which parts of another image, after the camera has moved, time has elapsed, and/or the objects have moved around.-Overview:...
in stereo matching. To achieve this, an array of microlenses is placed at the focal plane of the camera main lens. The image sensor
Image sensor
An image sensor is a device that converts an optical image into an electronic signal. It is used mostly in digital cameras and other imaging devices...
is positioned slightly behind the microlenses. Using such images the displacement of image parts that are not in focus can be analyzed and depth information can be extracted. Potentially, this camera system can be used to refocus an image virtually on a computer after the picture has been taken, as explained by Ng et al. The drawback of such a system is the low resolution
Optical resolution
Optical resolution describes the ability of an imaging system to resolve detail in the object that is being imaged.An imaging system may have many individual components including a lens and recording and display components...
that the final images have. As one microlens samples the light directions at one spatial point an increase in the number of image pixels can only be done by increasing the number of microlenses by the same amount. To overcome this limitation, Lumsdaine and Georgiev describe a new design of a plenoptic camera, called focused plenoptic camera where the microlens array is positioned before or behind the focal plane of the main lens. This modification samples the light field in a different way that allows for a higher spatial resolution by having a lower angular resolution
Angular resolution
Angular resolution, or spatial resolution, describes the ability of any image-forming device such as an optical or radio telescope, a microscope, a camera, or an eye, to distinguish small details of an object...
at the same time. With this design images can be refocused with a much higher spatial resolution. However, the low angular resolution can introduce some unwanted aliasing artifacts. A different design using low cost printed film (mask) instead of microlens array was proposed by researchers at MERL in 2007. This design overcomes several limitations of microlens array in terms of chromatic aberration
Chromatic aberration
In optics, chromatic aberration is a type of distortion in which there is a failure of a lens to focus all colors to the same convergence point. It occurs because lenses have a different refractive index for different wavelengths of light...
s, loss of boundary pixels, and allows higher spatial resolution photos to be captured. However, a mask based design reduces light compared to microlens arrays.
Image refocusing
A team at Stanford UniversityStanford University
The Leland Stanford Junior University, commonly referred to as Stanford University or Stanford, is a private research university on an campus located near Palo Alto, California. It is situated in the northwestern Santa Clara Valley on the San Francisco Peninsula, approximately northwest of San...
Computer Graphics Laboratory used a 16 megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.
Stereo with plenoptic cameras
Plenoptic cameras are good for following moving objects. A recording from a security camera based upon this technology could be used to produce an accurate 3D model of a suspect.Cameras available for purchase
From 2010 RaytrixRaytrix
Raytrix is a German company founded by Christian Perwass and Lennart Wietzke that was the first to create and market commercial plenoptic cameras.The R5 camera produces images of 1 megapixel resolution, while the R11 produces 3 megapixel images....
has sold several models of plenoptic camera for industrial and scientific applications, with resolutions starting from 1 megapixel.
Lytro
Lytro
Lytro is a startup company founded in 2006 by Ren Ng, Ph.D, a light-field photography researcher at Stanford University that aims to deliver a light field camera for consumers by the end of 2011....
was founded by Stanford University Computer Graphics Laboratory alumnus Ren Ng
Ren Ng
Dr. Ren Ng is the founder and the chief executive officer of a Mountain View, California-based start-up, Lytro, Inc. . The company is developing light field technology for digital cameras and has received $50 million in venture capital funding..Lytro unveiled its camera design on Oct...
to commercialize the light field camera he developed as a graduate student there. Lytro has developed a consumer light-field "digital camera capable of images using a plenoptic technique. Orders were being taken in late 2011 for cameras to be delivered early in 2012.
Other cameras
The AdobeAdobe Systems
Adobe Systems Incorporated is an American computer software company founded in 1982 and headquartered in San Jose, California, United States...
light-field camera is a prototype 100-megapixel camera that takes a three-dimensional
Stereo camera
A stereo camera is a type of camera with two or more lenses with a separate image sensor or film frame for each lens. This allows the camera to simulate human binocular vision, and therefore gives it the ability to capture three-dimensional images, a process known as stereo photography. Stereo...
photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2 megapixel photo of the entire scene around the camera and each image can be focused later in any way.
The CAFADIS camera is a plenoptic camera developed by the University of La Laguna
University of La Laguna
The University of San Fernando de La Laguna, also known as the ULL is situated in San Cristóbal de La Laguna, on the island of Tenerife. It is the oldest university in the Canary Islands, and has the highest student population of any university in these islands. It is also considered the most...
(Spain). CAFADIS stands (in Spanish) for phase-distance camera, since it can be used for distance and wavefront estimation. From a single shot it can produce several images refocused at different distances, depth maps, all-in-focus images and stereo pairs. A similar optical design can also be used in adaptive optics
Adaptive optics
Adaptive optics is a technology used to improve the performance of optical systems by reducing the effect of wavefront distortions. It is used in astronomical telescopes and laser communication systems to remove the effects of atmospheric distortion, and in retinal imaging systems to reduce the...
in astrophysics
Astrophysics
Astrophysics is the branch of astronomy that deals with the physics of the universe, including the physical properties of celestial objects, as well as their interactions and behavior...
, in order to correct the aberrations caused by atmospheric turbulence in telescope
Telescope
A telescope is an instrument that aids in the observation of remote objects by collecting electromagnetic radiation . The first known practical telescopes were invented in the Netherlands at the beginning of the 1600s , using glass lenses...
images. In order to perform these tasks different algorithms, running on GPU and FPGA, operate on the raw image captured by the camera.
Mitsubishi
Mitsubishi
The Mitsubishi Group , Mitsubishi Group of Companies, or Mitsubishi Companies is a Japanese multinational conglomerate company that consists of a range of autonomous businesses which share the Mitsubishi brand, trademark and legacy...
Electric Research Labs's (MERL) light field camera is based on the principle of optical heterodyning and uses a printed film (mask) placed close to the sensor. Any hand-held camera can be converted into a light field camera using this technology by simply inserting a low-cost film on top of the sensor. A mask-based design avoids the problem of loss of resolution, since a high-resolution photo can be generated for the focused parts of the scene.
Light-field microscope
Stanford University Computer Graphics Laboratory has developed a light-field microscope using a microlens array similar to the one used in the light-field camera developed by the lab. The prototype is built around a NikonNikon
, also known as just Nikon, is a multinational corporation headquartered in Tokyo, Japan, specializing in optics and imaging. Its products include cameras, binoculars, microscopes, measurement instruments, and the steppers used in the photolithography steps of semiconductor fabrication, of which...
Eclipse transmitted light microscope/wide-field fluorescence microscope
Fluorescence microscope
A fluorescence microscope is an optical microscope used to study properties of organic or inorganic substances using the phenomena of fluorescence and phosphorescence instead of, or in addition to, reflection and absorption...
and standard CCD cameras. Light-field capturing ability is obtained by a module containing a microlens array and other optical components placed in the light path between the objective lens and camera, with the final multifocused image rendered using deconvolution
Deconvolution
In mathematics, deconvolution is an algorithm-based process used to reverse the effects of convolution on recorded data. The concept of deconvolution is widely used in the techniques of signal processing and image processing...
. A later version of the prototype added a light-field illumination system consisting of a video projector (allowing computational control of illumination) and a second microlens array in the illumination light path of the microscope. The addition of a light-field illumination system both allowed for additional types of illumination (such as oblique illumination and quasi-dark-field) and correction for optical aberrations.
External links
- Article by Ren Ng of Stanford (now at Lytro)
- Lytro site (previously Refocus Imaging), a Stanford spinoff.
- Raytrix GmbH Raytrix english homepage.
- Say Sayonara to Blurry Pics. Wired.
- Fourier slice photography
- Rebellion Photonics
- Light Field Microscopy video by Stanford Computer Graphics Laboratory.