Edge detection
Encyclopedia
Edge detection is a fundamental tool in image processing
Image processing
In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...

 and computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...

, particularly in the areas of feature detection and feature extraction
Feature extraction
In pattern recognition and in image processing, feature extraction is a special form of dimensionality reduction.When the input data to an algorithm is too large to be processed and it is suspected to be notoriously redundant then the input data will be transformed into a reduced representation...

, which aim at identifying points in a digital image
Digital image
A digital image is a numeric representation of a two-dimensional image. Depending on whether or not the image resolution is fixed, it may be of vector or raster type...

 at which the image brightness
Luminous intensity
In photometry, luminous intensity is a measure of the wavelength-weighted power emitted by a light source in a particular direction per unit solid angle, based on the luminosity function, a standardized model of the sensitivity of the human eye...

 changes sharply or, more formally, has discontinuities. The same problem of finding discontinuities in 1D signals is known as step detection
Step detection
In statistics and signal processing, step detection is the process of finding abrupt changes in the mean level of a time series or signal. It is usually considered as a special kind of statistical method known as change point detection...

.

Motivations

The purpose of detecting sharp changes in image brightness is to capture important events and changes in properties of the world.
It can be shown that under rather general assumptions for an image formation model, discontinuities in image brightness are likely to correspond to:
  • discontinuities in depth,
  • discontinuities in surface orientation,
  • changes in material properties and
  • variations in scene illumination.

In the ideal case, the result of applying an edge detector to an image may lead to a set of connected curves that indicate the boundaries of objects, the boundaries of surface markings as well as curves that correspond to discontinuities in surface orientation.
Thus, applying an edge detection algorithm to an image may significantly reduce the amount of data to be processed and may therefore filter out information that may be regarded as less relevant, while preserving the important structural properties of an image.
If the edge detection step is successful, the subsequent task of interpreting the information contents in the original image may therefore be substantially simplified.
However, it is not always possible to obtain such ideal edges from real life images of moderate complexity.
Edges extracted from non-trivial images are often hampered by fragmentation, meaning that the edge curves are not connected, missing edge segments as well as false edges not corresponding to interesting phenomena in the image – thus complicating the subsequent task of interpreting the image data.

Edge detection is one of the fundamental steps in image processing, image analysis, image pattern recognition, and computer vision techniques. During recent years, however, substantial (and successful) research has also been made on computer vision methods that do not explicitly rely on edge detection as a pre-processing step.

Edge properties

The edges extracted from a two-dimensional image of a three-dimensional scene can be classified as either viewpoint dependent or viewpoint independent.
A viewpoint independent edge typically reflects inherent properties of the three-dimensional objects, such as surface markings and surface shape.
A viewpoint dependent edge may change as the viewpoint changes, and typically reflects the geometry of the scene, such as objects occluding one another.

A typical edge might for instance be the border between a block of red color and a block of yellow. In contrast a line
Line (mathematics)
The notion of line or straight line was introduced by the ancient mathematicians to represent straight objects with negligible width and depth. Lines are an idealization of such objects...

(as can be extracted by a ridge detector
Ridge detection
The ridges of a smooth function of two variables is a set of curves whose points are, in one or more ways to be made precise below, local maxima of the function in at least one dimension. For a function of N variables, its ridges are a set of curves whose points are local maxima in N-1 dimensions...

) can be a small number of pixels of a different color on an otherwise unchanging background. For a line, there may therefore usually be one edge on each side of the line.

A simple edge model

Although certain literature has considered the detection of ideal step edges, the edges obtained from natural images are usually not at all ideal step edges. Instead they are normally affected by one or several of the following effects:
  • focal blur caused by a finite depth-of-field and finite point spread function
    Point spread function
    The point spread function describes the response of an imaging system to a point source or point object. A more general term for the PSF is a system's impulse response, the PSF being the impulse response of a focused optical system. The PSF in many contexts can be thought of as the extended blob...

    .
  • penumbral blur caused by shadows created by light sources of non-zero radius.
  • shading
    Shading
    Shading refers to depicting depth perception in 3D models or illustrations by varying levels of darkness.-Drawing:Shading is a process used in drawing for depicting levels of darkness on paper by applying media more densely or with a darker shade for darker areas, and less densely or with a lighter...

     at a smooth object

A number of researchers have used a Gaussian smoothed step edge (an error function) as the simplest extension of the ideal step edge model for modeling the effects of edge blur in practical applications.
Thus, a one-dimensional image which has exactly one edge placed at may be modeled as:


At the left side of the edge, the intensity is , and right of the edge it is
. The scale parameter is called the blur scale of the edge.

Why edge detection is a non-trivial task

To illustrate why edge detection is not a trivial task, consider the problem of detecting edges in the following one-dimensional signal. Here, we may intuitively say that there should be an edge between the 4th and 5th pixels.




















5 7 6 4 152 148 149



If the intensity difference were smaller between the 4th and the 5th pixels and if the intensity differences between the adjacent neighboring pixels were higher, it would not be as easy to say that there should be an edge in the corresponding region. Moreover, one could argue that this case is one in which there are several edges.





















5 7 6 41 113 148 149



Hence, to firmly state a specific threshold on how large the intensity change between two neighbouring pixels must be for us to say that there should be an edge between these pixels is not always simple. Indeed, this is one of the reasons why edge detection may be a non-trivial problem unless the objects in the scene are particularly simple and the illumination conditions can be well controlled (see for example, the edges extracted from the image with the girl above).

Approaches

There are many methods for edge detection, but most of them can be grouped into two categories, search-based and zero-crossing
Zero crossing
Zero-crossing is a commonly used term in electronics, mathematics, and image processing. In mathematical terms, a "zero-crossing" is a point where the sign of a function changes Zero-crossing is a commonly used term in electronics, mathematics, and image processing. In mathematical terms, a...

 based.
The search-based methods detect edges by first computing a measure of edge strength, usually a first-order derivative expression such as the gradient magnitude, and then searching for local directional maxima of the gradient magnitude using a computed estimate of the local orientation of the edge, usually the gradient direction.
The zero-crossing based methods search for zero crossings in a second-order derivative expression computed from the image in order to find edges, usually the zero-crossings of the Laplacian or the zero-crossings of a non-linear differential expression. As a pre-processing step to edge detection, a smoothing stage, typically Gaussian smoothing, is almost always applied (see also noise reduction
Noise reduction
Noise reduction is the process of removing noise from a signal.All recording devices, both analogue or digital, have traits which make them susceptible to noise...

).

The edge detection methods that have been published mainly differ in the types of smoothing filters that are applied and the way the measures of edge strength are computed. As many edge detection methods rely on the computation of image gradients, they also differ in the types of filters used for computing gradient estimates in the x- and y-directions.

A survey of a number of different edge detection methods can be found in (Ziou and Tabbone 1998); see also the encyclopedia articles on edge detection in Encyclopedia of Mathematics and Encyclopedia of Computer Science and Engineering.

Canny edge detection

John Canny
John Canny
John F. Canny is an Australian computer scientist, and Paul and Stacy Jacobs Distinguished Professor of Engineering in the Computer Science Department of the University of California, Berkeley...

 considered the mathematical problem of deriving an optimal smoothing filter given the criteria of detection, localization and minimizing multiple responses to a single edge. He showed that the optimal filter given these assumptions is a sum of four exponential terms. He also showed that this filter can be well approximated by first-order derivatives of Gaussians.
Canny also introduced the notion of non-maximum suppression, which means that given the presmoothing filters, edge points are defined as points where the gradient magnitude assumes a local maximum in the gradient direction.
Looking for the zero crossing of the 2nd derivative along the gradient direction was first proposed by Haralick .
It took less than two decades to find a modern geometric variational meaning for that operator that links it to the Marr–Hildreth
Marr-Hildreth algorithm
In computer vision, the Marr–Hildreth algorithm is a method of detecting edges in digital images, that is continuous curves where there are strong and rapid variations in image brightness. The Marr–Hildreth edge detection method is simple and operates by convolving the image with the Laplacian of...

 (zero crossing of the Laplacian) edge detector.
That observation was presented by Ron Kimmel
Ron Kimmel
Ron Kimmel is a professor of Computer Science atthe Technion Israel Institute of Technology.He holds a D.Sc. degree in electrical engineering from the Technion,and spent a post-doctoral position at UC Berkeley and Berkeley Labs, and a visiting...

 and Alfred Bruckstein.

Although his work was done in the early days of computer vision, the Canny edge detector (including its variations) is still a state-of-the-art edge detector. Unless the preconditions are particularly suitable, it is hard to find an edge detector that performs significantly better than the Canny edge detector.

The Canny-Deriche detector was derived from similar mathematical criteria as the Canny edge detector, although starting from a discrete viewpoint and then leading to a set of recursive filters for image smoothing instead of exponential filters or Gaussian filters.

The differential edge detector described below can be seen as a reformulation of Canny's method from the viewpoint of differential invariants computed from a scale-space representation leading to a number of advantages in terms of both theoretical analysis and sub-pixel implementation.

Other first-order methods

For estimating image gradients from the input image or a smoothed version of it, different gradient operators can be applied. The simplest approach is to use central differences:


corresponding to the application of the following filter masks to the image data:


The well-known and earlier Sobel operator is based on the following filters:


Given such estimates of first- order derivatives, the gradient magnitude is then computed as:


while the gradient orientation can be estimated as


Other first-order difference operators for estimating image gradient have been proposed in the Prewitt
Prewitt
The Prewitt operator is used in image processing, particularly within edge detection algorithms. Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Prewitt operator is...

 operator and Roberts cross
Roberts Cross
The Roberts' Cross operator is used in image processing and computer vision for edge detection. It was one of the first edge detectors and was initially proposed by Lawrence Roberts in 1963...

.

Thresholding and linking

Once we have computed a measure of edge strength (typically the gradient magnitude), the next stage is to apply a threshold, to decide whether edges are present or not at an image point. The lower the threshold, the more edges will be detected, and the result will be increasingly susceptible to noise
Image noise
Image noise is random variation of brightness or color information in images, and is usually an aspect of electronic noise. It can be produced by the sensor and circuitry of a scanner or digital camera...

 and detecting edges of irrelevant features in the image. Conversely a high threshold may miss subtle edges, or result in fragmented edges.

If the edge thresholding is applied to just the gradient magnitude image, the resulting edges will in general be thick and some type of edge thinning post-processing is necessary. For edges detected with non-maximum suppression however, the edge curves are thin by definition and the edge pixels can be linked into edge polygon by an edge linking (edge tracking) procedure. On a discrete grid, the non-maximum suppression stage can be implemented by estimating the gradient direction using first-order derivatives, then rounding off the gradient direction to multiples of 45 degrees, and finally comparing the values of the gradient magnitude in the estimated gradient direction.

A commonly used approach to handle the problem of appropriate thresholds for thresholding is by using thresholding with hysteresis
Hysteresis
Hysteresis is the dependence of a system not just on its current environment but also on its past. This dependence arises because the system can be in more than one internal state. To predict its future evolution, either its internal state or its history must be known. If a given input alternately...

. This method uses multiple thresholds to find edges. We begin by using the upper threshold to find the start of an edge. Once we have a start point, we then trace the path of the edge through the image pixel by pixel, marking an edge whenever we are above the lower threshold. We stop marking our edge only when the value falls below our lower threshold. This approach makes the assumption that edges are likely to be in continuous curves, and allows us to follow a faint section of an edge we have previously seen, without meaning that every noisy pixel in the image is marked down as an edge. Still, however, we have the problem of choosing appropriate thresholding parameters, and suitable thresholding values may vary over the image.

Edge thinning

Edge thinning is a technique used to remove the unwanted spurious points on the edge of an image. This technique is employed after the image has been filtered for noise (using median, Gaussian filter etc.), the edge operator has been applied (like the ones described above) to detect the edges and after the edges have been smoothed using an appropriate threshold value.
This removes all the unwanted points and if applied carefully, results in one pixel thick edge elements.

Advantages:
  1. Sharp and thin edges lead to greater efficiency in object recognition.
  2. If Hough transform
    Hough transform
    The Hough transform is a feature extraction technique used in image analysis, computer vision, and digital image processing. The purpose of the technique is to find imperfect instances of objects within a certain class of shapes by a voting procedure...

    s are used to detect lines and ellipses, then thinning could give much better results.
  3. If the edge happens to be boundary of a region, then thinning could easily give the image parameters like perimeter without much algebra.


There are many popular algorithms used to do this, one such is described below:

1) Choose a type of connectivity, like 8, 6 or 4.

2) 8 connectivity is preferred, where all the immediate pixels surrounding a particular pixel are considered.

3) Remove points from North, south, east and west.

4) Do this in multiple passes, i.e. after the north pass, use the same semi processed image in the other passes and so on.

5) Remove a point if:
The point has no neighbors in the North (if you are in the north pass, and
respective directions for other passes.)
The point is not the end of a line.
The point is isolated.
Removing the points will not cause to disconnect its neighbors in any way.
6) Else keep the point.
The number of passes across direction should be chosen according to the level of accuracy desired.

Second-order approaches to edge detection

Some edge-detection operators are instead based upon second-order derivatives of the intensity. This essentially captures the rate of change
Rate of change
Rate of change may refer to:* Derivative, rate of change in a mathematical function* Difference quotient, the difference between two output values divided by the difference between the corresponding input values...

 in the intensity gradient. Thus, in the ideal continuous case, detection of zero-crossings in the second derivative captures local maxima in the gradient.

The early Marr-Hildreth
Marr-Hildreth algorithm
In computer vision, the Marr–Hildreth algorithm is a method of detecting edges in digital images, that is continuous curves where there are strong and rapid variations in image brightness. The Marr–Hildreth edge detection method is simple and operates by convolving the image with the Laplacian of...

 operator is based on the detection of zero-crossings of the Laplacian operator applied to a Gaussian-smoothed image. It can be shown, however, that this operator will also return false edges corresponding to local minima of the gradient magnitude. Moreover, this operator will give poor localization at curved edges. Hence, this operator is today mainly of historical interest.

Differential edge detection

A more refined second-order edge detection approach which automatically detects edges with sub-pixel accuracy, uses the following differential approach of detecting zero-crossings of the second-order directional derivative in the gradient direction:

Following the differential geometric way of expressing the requirement of non-maximum suppression proposed by Lindeberg, let us introduce at every image point a local coordinate system , with the -direction parallel to the gradient direction. Assuming that the image has been pre-smoothed by Gaussian smoothing and a scale-space representation at scale has been computed, we can require that the gradient magnitude of the scale-space representation, which is equal to the first-order directional derivative in the -direction , should have its first order directional derivative in the -direction equal to zero
while the second-order directional derivative in the -direction of should be negative, i.e.,
Written out as an explicit expression in terms of local partial derivatives , ... , this edge definition can be expressed as the zero-crossing curves of the differential invariant
that satisfy a sign-condition on the following differential invariant
where , ... denote partial derivatives computed from
a scale-space representation obtained by smoothing the original image with a Gaussian kernel.
In this way, the edges will be automatically obtained as continuous curves with sub-pixel accuracy.
Hysteresis thresholding can also be applied to these differential and subpixel edge segments.

In practice, first-order derivative approximations can be computed by central differences as described above, while second-order derivatives can be computed from the scale-space representation according to:

corresponding to the following filter masks:


Higher-order derivatives for the third-order sign condition can be obtained in an analogous fashion.

Phase congruency-based edge detection

A recent development in edge detection techniques takes a frequency domain approach to finding edge locations. Phase congruency
Phase congruency
Phase congruency is a measure of feature significance in computer images, a method of edge detection that is particularly robust against changes in illumination and contrast.-Foundations:...

 (also known as phase coherence) methods attempt to find locations in an image where all sinusoids in the frequency domain are in phase. These locations will generally correspond to the location of a perceived edge, regardless of whether the edge is represented by a large change in intensity in the spatial domain. A key benefit of this technique is that it responds strongly to Mach bands
Mach bands
Mach bands is an optical illusion named after the physicist Ernst Mach. The illusion consists of light or dark stripes that are perceived next to the boundary between two regions of an image that have different lightness gradients .-Explanation:The Mach bands effect is due to the spatial...

, and avoids false positives typically found around roof edges. A roof edge, is a discontinuity in the first order derivative of a grey-level profile.

See also

  • Canny edge detector
  • Hough transform
    Hough transform
    The Hough transform is a feature extraction technique used in image analysis, computer vision, and digital image processing. The purpose of the technique is to find imperfect instances of objects within a certain class of shapes by a voting procedure...

     for detecting straight lines, circles or ellipses from edge points
  • Feature detection (computer vision) for other low-level feature detectors
  • Image noise reduction
  • Prewitt
    Prewitt
    The Prewitt operator is used in image processing, particularly within edge detection algorithms. Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Prewitt operator is...

  • Ridge detection
    Ridge detection
    The ridges of a smooth function of two variables is a set of curves whose points are, in one or more ways to be made precise below, local maxima of the function in at least one dimension. For a function of N variables, its ridges are a set of curves whose points are local maxima in N-1 dimensions...

     for relations between edge detectors and ridge detectors
  • Scale-space for theory of Gaussian image smoothing and multi-scale feature detection
  • Sobel operator
  • Convolution#Applications

Further reading

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK