Image registration
Encyclopedia
Image registration is the process of transforming different sets of data into one coordinate system. Data may be multiple photographs, data from different sensors, from different times, or from different viewpoints. It is used in computer vision
, medical imaging
, military automatic target recognition
, and compiling and analyzing images and data from satellites. Registration is necessary in order to be able to compare or integrate the data obtained from these different measurements.
s, which include translation, rotation, scaling, and other affine transforms. Linear transformation
s are global in nature, thus, they cannot model local geometric differences between images.
The second category of transformations allow 'elastic' or 'nonrigid' transformations. These transformations are capable of locally warping the target image to align with the reference image. Nonrigid transformations include radial basis functions (thin-plate or surface splines, multiquadrics, and compactly-supported transformations), physical continuum models (viscous fluids), and large deformation models (diffeomorphism
s).
(CPs) in images. When the number of control points exceeds the minimum required to define the appropriate transformation model, iterative algorithms like RANSAC
can be used to robustly estimate the parameters of a particular transformation type (e.g. affine) for registration of the images.
Frequency-domain methods find the transformation parameters for registration of the images while working in the transform domain. Such methods work for simple transformations, such as translation, rotation, and scaling. Applying the Phase correlation
method to a pair of images produces a third image which contains a single peak. The location of this peak corresponds to the relative translation between the images. Unlike many spatial-domain algorithms, the phase correlation method is resilient to noise, occlusions, and other defects typical of medical or satellite images. Additionally, the phase correlation uses the fast Fourier transform
to compute the cross-correlation between the two images, generally resulting in large performance gains. The method can be extended to determine rotation and scaling differences between two images by first converting the images to log-polar
coordinates. Due to properties of the Fourier transform
, the rotation and scaling parameters can be determined in a manner invariant to translation.
Multi-modality registration methods are often used in medical imaging
as images of a subject are frequently obtained from different scanners. Examples include registration of brain CT
/MRI images or whole body PET
/CT
images for tumor localization, registration of contrast-enhanced CT
images against non-contrast-enhanced CT
images for segmentation of specific parts of the anatomy, and registration of ultrasound
and CT
images for prostate
localization in radiotherapy.
. An image similarity measure quantifies the degree of similarity between intensity patterns in two images. The choice of an image similarity measure depends on the modality of the images to be registered. Common examples of image similarity measures include cross-correlation
, mutual information
, sum of squared intensity differences, and ratio image uniformity. Mutual information and normalized mutual information are the most popular image similarity measures for registration of multimodality images. Cross-correlation, sum of squared intensity differences and ratio image uniformity are commonly used for registration of images in the same modality.
associated with registering images that have any spatio-temporal differences. A confident registration with a measure of uncertainty is critical for many change detection
applications such as medical diagnostics.
In remote sensing
applications where a digital image pixel may represent several kilometers of spatial distance (such as NASA's LANDSAT imagery), an uncertain image registration can mean that a solution could be several kilometers from ground truth. Several notable papers have attempted to quantify uncertainty in image registration in order to compare results. However, many approaches to quantifying uncertainty or estimating deformations are computationally intensive or are only applicable to limited sets of spatial transformations.
Medical image
registration (for data of the same patient taken at different points in time such as change detection or tumor monitoring) often additionally involves elastic (also known as nonrigid) registration to cope with deformation of the subject (due to breathing, anatomical changes, and so forth). Nonrigid registration of medical images can also be used to register a patient's data to an anatomical atlas, such as the Talairach
atlas for neuroimaging.
It is also used in astrophotography
to align images taken of space. Using control points (automatically or manually entered), the computer performs transformations on one image to make major features align with a second image.
Image registration is essential part of panoramic image creation. There are many different techniques that can be implemented in real time and run on embedded devices like cameras and camera-phones.
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
, medical imaging
Medical imaging
Medical imaging is the technique and process used to create images of the human body for clinical purposes or medical science...
, military automatic target recognition
Automatic Target Recognition
Automatic target recognition , is the ability for an algorithm or device to recognize targets or objects based on data obtained from sensors....
, and compiling and analyzing images and data from satellites. Registration is necessary in order to be able to compare or integrate the data obtained from these different measurements.
Intensity-based vs feature-based
Image registration or image alignment algorithms can be classified into intensity-based and feature-based. One of the images is referred to as the reference or source and the second image is referred to as the target or sensed. Image registration involves spatially transforming the target image to align with the reference image. Intensity-based methods compare intensity patterns in images via correlation metrics, while feature-based methods find correspondence between image features such as points, lines, and contours. Intensity-based methods register entire images or subimages. If subimages are registered, centers of corresponding subimages are treated as corresponding feature points. Feature-based method established correspondence between a number of points in images. Knowing the correspondence between a number of points in images, a transformation is then determined to map the target image to the reference images, thereby establishing point-by-point correspondence between the reference and target images.Transformation models
Image registration algorithms can also be classified according to the transformation models they use to relate the target image space to the reference image space. The first broad category of transformation models includes linear transformationLinear transformation
In mathematics, a linear map, linear mapping, linear transformation, or linear operator is a function between two vector spaces that preserves the operations of vector addition and scalar multiplication. As a result, it always maps straight lines to straight lines or 0...
s, which include translation, rotation, scaling, and other affine transforms. Linear transformation
Linear transformation
In mathematics, a linear map, linear mapping, linear transformation, or linear operator is a function between two vector spaces that preserves the operations of vector addition and scalar multiplication. As a result, it always maps straight lines to straight lines or 0...
s are global in nature, thus, they cannot model local geometric differences between images.
The second category of transformations allow 'elastic' or 'nonrigid' transformations. These transformations are capable of locally warping the target image to align with the reference image. Nonrigid transformations include radial basis functions (thin-plate or surface splines, multiquadrics, and compactly-supported transformations), physical continuum models (viscous fluids), and large deformation models (diffeomorphism
Diffeomorphism
In mathematics, a diffeomorphism is an isomorphism in the category of smooth manifolds. It is an invertible function that maps one differentiable manifold to another, such that both the function and its inverse are smooth.- Definition :...
s).
Spatial vs. frequency domain methods
Spatial methods operate in the image domain, matching intensity patterns or features in images. Some of the feature matching algorithms are outgrowths of traditional techniques for performing manual image registration, in which an operator chooses corresponding control pointsFeature (Computer vision)
In computer vision and image processing the concept of feature is used to denote a piece of information which is relevant for solving the computational task related to a certain application...
(CPs) in images. When the number of control points exceeds the minimum required to define the appropriate transformation model, iterative algorithms like RANSAC
RANSAC
RANSAC is an abbreviation for "RANdom SAmple Consensus". It is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers. It is a non-deterministic algorithm in the sense that it produces a reasonable result only with a certain...
can be used to robustly estimate the parameters of a particular transformation type (e.g. affine) for registration of the images.
Frequency-domain methods find the transformation parameters for registration of the images while working in the transform domain. Such methods work for simple transformations, such as translation, rotation, and scaling. Applying the Phase correlation
Phase correlation
In image processing, phase correlation is a method of image registration, and uses a fast frequency-domain approach to estimate the relative translative offset between two similar images.- Example :...
method to a pair of images produces a third image which contains a single peak. The location of this peak corresponds to the relative translation between the images. Unlike many spatial-domain algorithms, the phase correlation method is resilient to noise, occlusions, and other defects typical of medical or satellite images. Additionally, the phase correlation uses the fast Fourier transform
Fast Fourier transform
A fast Fourier transform is an efficient algorithm to compute the discrete Fourier transform and its inverse. "The FFT has been called the most important numerical algorithm of our lifetime ." There are many distinct FFT algorithms involving a wide range of mathematics, from simple...
to compute the cross-correlation between the two images, generally resulting in large performance gains. The method can be extended to determine rotation and scaling differences between two images by first converting the images to log-polar
Log-polar coordinates
Log-polar coordinates is a coordinate system in two dimensions, where a point is identified by two numbers, one for the logarithm of the distance to a certain point, and one for an angle. Log-polar coordinates are closely connected to polar coordinates, which are usually used to describe domains...
coordinates. Due to properties of the Fourier transform
Fourier transform
In mathematics, Fourier analysis is a subject area which grew from the study of Fourier series. The subject began with the study of the way general functions may be represented by sums of simpler trigonometric functions...
, the rotation and scaling parameters can be determined in a manner invariant to translation.
Single- vs. multi-modality methods
Another classification can be made between single-modality and multi-modality methods. Single-modality methods tend to register images in the same modality acquired by the same scanner/sensor type, while multi-modality registration methods tended to register images acquired by different scanner/sensor types.Multi-modality registration methods are often used in medical imaging
Medical imaging
Medical imaging is the technique and process used to create images of the human body for clinical purposes or medical science...
as images of a subject are frequently obtained from different scanners. Examples include registration of brain CT
Computed tomography
X-ray computed tomography or Computer tomography , is a medical imaging method employing tomography created by computer processing...
/MRI images or whole body PET
Positron emission tomography
Positron emission tomography is nuclear medicine imaging technique that produces a three-dimensional image or picture of functional processes in the body. The system detects pairs of gamma rays emitted indirectly by a positron-emitting radionuclide , which is introduced into the body on a...
/CT
Computed tomography
X-ray computed tomography or Computer tomography , is a medical imaging method employing tomography created by computer processing...
images for tumor localization, registration of contrast-enhanced CT
Computed tomography
X-ray computed tomography or Computer tomography , is a medical imaging method employing tomography created by computer processing...
images against non-contrast-enhanced CT
Computed tomography
X-ray computed tomography or Computer tomography , is a medical imaging method employing tomography created by computer processing...
images for segmentation of specific parts of the anatomy, and registration of ultrasound
Ultrasound
Ultrasound is cyclic sound pressure with a frequency greater than the upper limit of human hearing. Ultrasound is thus not separated from "normal" sound based on differences in physical properties, only the fact that humans cannot hear it. Although this limit varies from person to person, it is...
and CT
Computed tomography
X-ray computed tomography or Computer tomography , is a medical imaging method employing tomography created by computer processing...
images for prostate
Prostate
The prostate is a compound tubuloalveolar exocrine gland of the male reproductive system in most mammals....
localization in radiotherapy.
Automatic vs. interactive methods
Registration methods may be classified based on the level of automation they provide. Manual, interactive, semi-automatic, and automatic methods have been developed. Manual methods provide tools to align the images manually. Interactive methods reduce user bias by performing certain key operations automatically while still relying on the user to guide the registration. Semi-automatic methods perform more of the registration steps automatically but depend on the user to verify the correctness of a registration. Automatic methods do not allow any user interaction and perform all registration steps automatically.Similarity measures for image registration
Image similarities are broadly used in medical imagingMedical imaging
Medical imaging is the technique and process used to create images of the human body for clinical purposes or medical science...
. An image similarity measure quantifies the degree of similarity between intensity patterns in two images. The choice of an image similarity measure depends on the modality of the images to be registered. Common examples of image similarity measures include cross-correlation
Cross-correlation
In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them. This is also known as a sliding dot product or sliding inner-product. It is commonly used for searching a long-duration signal for a shorter, known feature...
, mutual information
Mutual information
In probability theory and information theory, the mutual information of two random variables is a quantity that measures the mutual dependence of the two random variables...
, sum of squared intensity differences, and ratio image uniformity. Mutual information and normalized mutual information are the most popular image similarity measures for registration of multimodality images. Cross-correlation, sum of squared intensity differences and ratio image uniformity are commonly used for registration of images in the same modality.
Uncertainty
There is a level of uncertaintyUncertainty
Uncertainty is a term used in subtly different ways in a number of fields, including physics, philosophy, statistics, economics, finance, insurance, psychology, sociology, engineering, and information science...
associated with registering images that have any spatio-temporal differences. A confident registration with a measure of uncertainty is critical for many change detection
Change detection
In statistical analysis, change detection tries to identify changes in the probability distribution of a stochastic process or time series. In general the problem concerns both detecting whether or not a change has occurred, or whether several changes might have occurred, and identifying the times...
applications such as medical diagnostics.
In remote sensing
Remote sensing
Remote sensing is the acquisition of information about an object or phenomenon, without making physical contact with the object. In modern usage, the term generally refers to the use of aerial sensor technologies to detect and classify objects on Earth by means of propagated signals Remote sensing...
applications where a digital image pixel may represent several kilometers of spatial distance (such as NASA's LANDSAT imagery), an uncertain image registration can mean that a solution could be several kilometers from ground truth. Several notable papers have attempted to quantify uncertainty in image registration in order to compare results. However, many approaches to quantifying uncertainty or estimating deformations are computationally intensive or are only applicable to limited sets of spatial transformations.
Applications
Image registration has applications in remote sensing (cartography updating), and computer vision. Due to the vast applications to which image registration can be applied, it is impossible to develop a general method that is optimized for all uses.Medical image
Medical imaging
Medical imaging is the technique and process used to create images of the human body for clinical purposes or medical science...
registration (for data of the same patient taken at different points in time such as change detection or tumor monitoring) often additionally involves elastic (also known as nonrigid) registration to cope with deformation of the subject (due to breathing, anatomical changes, and so forth). Nonrigid registration of medical images can also be used to register a patient's data to an anatomical atlas, such as the Talairach
Jean Talairach
Jean Talairach was a neurosurgeon who practiced at the Centre Hospitalier Ste. Anne in Paris.-Talairach coordinates:...
atlas for neuroimaging.
It is also used in astrophotography
Astrophotography
Astrophotography is a specialized type of photography that entails recording images of astronomical objects and large areas of the night sky. The first photographs of an astronomical object were taken in the 1840s, but it was not until the late 19th century that advances in technology allowed for...
to align images taken of space. Using control points (automatically or manually entered), the computer performs transformations on one image to make major features align with a second image.
Image registration is essential part of panoramic image creation. There are many different techniques that can be implemented in real time and run on embedded devices like cameras and camera-phones.
External links
- Richard Szeliski, Image Alignment and Stitching: A Tutorial. Foundations and Trends in Computer Graphics and Computer Vision, 2:1-104, 2006.
- B. Fischer, J. Modersitzki: Ill-posed medicine – an introduction to image registration. Inverse Problems, 24:1–19, 2008
- Barbara Zitová, Jan Flusser: Image registration methods: a survey. Image Vision Comput. 21(11): 977-1000 (2003).
- How to register two images using MatlabMATLABMATLAB is a numerical computing environment and fourth-generation programming language. Developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages,...