Detection theory
Encyclopedia
Detection theory, or signal detection theory, is a means to quantify the ability to discern between information-bearing energy patterns (called stimulus
in humans, signal in machines) and random energy patterns that distract from the information (called noise
, consisting of background stimuli and random activity of the detection machine and of the nervous system of the operator).
According to the theory, there are a number of determiners of how a detecting system will detect a signal, and where its threshold levels will be. The theory can explain how changing the threshold will affect the ability to discern, often exposing how adapted the system is to the task, purpose or goal at which it is aimed.
When the detecting system is a human being, experience, expectations, physiological state (e.g. fatigue) and other factors can affect the threshold applied. For instance, a sentry in wartime will likely detect fainter stimuli than the same sentry in peacetime.
Much of the early work in detection theory was done by radar
researchers. By 1954 the theory was fully developed on the theoretical side as described by Peterson
, Birdsall and Fox and the foundation for the psychological theory was made by Wilson P. Tanner, David M. Green, and John A. Swets
, also in 1954.
Detection theory was used in 1966 by John A. Swets
and David M. Green for psychophysics
. Green and Swets
criticized the traditional methods of psychophysics for their inability to discriminate between the real sensitivity of subjects and their (potential) response bias
es.
Detection theory has applications in many fields such as diagnostics of any kind, quality control
, telecommunications, and psychology
. The concept is similar to the signal to noise ratio used in the sciences and confusion matrices
used in artificial intelligence
. It is also usable in alarm management
, where it is important to separate important events from background
noise.
).
To apply signal detection theory to a data set where stimuli were either present or absent, and the observer categorized each trial as having the stimulus present or absent, the trials are sorted into one of four categories:
Based on the proportions of these types of trials, numerical estimates of sensitivity can be obtained with statistics like the sensitivity index d'
and A', and response bias can be estimated with statistics like β.
Signal detection theory can also be applied to memory experiments, where items are presented on a study list for later testing. A test list is created by combining these 'old' items with novel, 'new' items that did not appear on the study list. On each test trial the subject will respond 'yes, this was on the study list' or 'no, this was not on the study list'. Items presented on the study list are called Targets, and new items are called Distractors. Saying 'Yes' to a target constitutes a Hit, while saying 'Yes' to a distractor constitutes a False Alarm.
. Topics include memory
, stimulus characterists of schedules of reinforcement, etc.
, or d'
. There are also non-parametric measures, such as the area under the ROC-curve
.
, H1, absent, and H2, present, in the event of a particular observation
, y, a classical approach is to choose H1 when p(H1|y) > p(H2|y) and H2 in the reverse case. In the event that the two a posteriori
probabilities
are equal, one typically defaults to a single choice, say H2. One could also flip a coin although the expected number of errors would be the same.
When taking this approach, usually what one knows are the conditional probabilities, p(y|H1) and p(y|H2), and the a priori probabilities
and 
. In this case,
,
where p(y) is the total probability of event y,
.
H2 is chosen in case
and H1 otherwise.
Often, the ratio
is called 
and 
is called 
, the likelihood ratio.
Using this terminology, H2 is chosen in case
. This is called MAP testing, where MAP stands for "maximum a posteriori").
Taking this approach minimizes the expected number of errors one will make.
, it is much more important to shoot down the bomber if it is there, than it is not to send a fighter squadron to inspect a false alarm (assuming a large supply of fighter squadrons). The Bayes
criterion is an approach suitable for such cases.
Here a utility
is associated with each of four situations:
As is shown below, what is important are the differences,
and 
.
Similarly, there are four probabilities,
, 
, etc., for each of the cases (which are dependent on one's decision strategy).
The Bayes criterion approach is to maximize the expected utility:
Effectively, one may maximize the sum,
,
and make the following substitutions:
where
and 
are the a priori probabilities, 
and 
, and 
is the region of observation events, y, that are responded to as though H1 is true.
and thus 
are maximized by extending 
over the region where
This is accomplished by deciding H2 in case
and H1 otherwise, where L(y) is the so-defined likelihood ratio.
Stimulus (psychology)
In psychology, stimuli are energy patterns which are registered by the senses. In behaviorism and related stimulus–response theories, stimuli constitute the basis for behavior, whereas in perceptual psychology they constitute the basis for perception.In the second half of the 19th century, the...
in humans, signal in machines) and random energy patterns that distract from the information (called noise
Noise (electronics)
Electronic noise is a random fluctuation in an electrical signal, a characteristic of all electronic circuits. Noise generated by electronic devices varies greatly, as it can be produced by several different effects...
, consisting of background stimuli and random activity of the detection machine and of the nervous system of the operator).
According to the theory, there are a number of determiners of how a detecting system will detect a signal, and where its threshold levels will be. The theory can explain how changing the threshold will affect the ability to discern, often exposing how adapted the system is to the task, purpose or goal at which it is aimed.
When the detecting system is a human being, experience, expectations, physiological state (e.g. fatigue) and other factors can affect the threshold applied. For instance, a sentry in wartime will likely detect fainter stimuli than the same sentry in peacetime.
Much of the early work in detection theory was done by radar
Radar
Radar is an object-detection system which uses radio waves to determine the range, altitude, direction, or speed of objects. It can be used to detect aircraft, ships, spacecraft, guided missiles, motor vehicles, weather formations, and terrain. The radar dish or antenna transmits pulses of radio...
researchers. By 1954 the theory was fully developed on the theoretical side as described by Peterson
W. Wesley Peterson
William Wesley Peterson was an American mathematician and computer scientist. He was best known for inventing the Cyclic Redundancy Check , for which research he was awarded the Japan Prize in 1999....
, Birdsall and Fox and the foundation for the psychological theory was made by Wilson P. Tanner, David M. Green, and John A. Swets
John A. Swets
John A. Swets is a psychologist. He has played a key role in adaptation of signal detection theory to first the psychology of perception and later as a central tool in medical diagnostics.- Further reading :...
, also in 1954.
Detection theory was used in 1966 by John A. Swets
John A. Swets
John A. Swets is a psychologist. He has played a key role in adaptation of signal detection theory to first the psychology of perception and later as a central tool in medical diagnostics.- Further reading :...
and David M. Green for psychophysics
Psychophysics
Psychophysics quantitatively investigates the relationship between physical stimuli and the sensations and perceptions they effect. Psychophysics has been described as "the scientific study of the relation between stimulus and sensation" or, more completely, as "the analysis of perceptual...
. Green and Swets
John A. Swets
John A. Swets is a psychologist. He has played a key role in adaptation of signal detection theory to first the psychology of perception and later as a central tool in medical diagnostics.- Further reading :...
criticized the traditional methods of psychophysics for their inability to discriminate between the real sensitivity of subjects and their (potential) response bias
Response bias
Response bias is a type of cognitive bias which can affect the results of a statistical survey if respondents answer questions in the way they think the questioner wants them to answer rather than according to their true beliefs...
es.
Detection theory has applications in many fields such as diagnostics of any kind, quality control
Quality control
Quality control, or QC for short, is a process by which entities review the quality of all factors involved in production. This approach places an emphasis on three aspects:...
, telecommunications, and psychology
Psychology
Psychology is the study of the mind and behavior. Its immediate goal is to understand individuals and groups by both establishing general principles and researching specific cases. For many, the ultimate goal of psychology is to benefit society...
. The concept is similar to the signal to noise ratio used in the sciences and confusion matrices
Confusion matrix
In the field of artificial intelligence, a confusion matrix is a specific table layout that allows visualization of the performance of an algorithm, typically a supervised learning one . Each column of the matrix represents the instances in a predicted class, while each row represents the...
used in artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
. It is also usable in alarm management
Alarm management
Alarm management is the application of human factors along with instrumentation engineering and systems thinking to manage the design of an alarm system to increase its usability...
, where it is important to separate important events from background
Background noise
In acoustics and specifically in acoustical engineering, background noise or ambient noise is any sound other than the sound being monitored. Background noise is a form of noise pollution or interference. Background noise is an important concept in setting noise regulations...
noise.
Psychology
Signal detection theory (SDT) is used when psychologists want to measure the way we make decisions under conditions of uncertainty, such as how we would perceive distances in foggy conditions. SDT assumes that the decision maker is not a passive receiver of information, but an active decision-maker who makes difficult perceptual judgements under conditions of uncertainty. In foggy circumstances, we are forced to decide how far away from us an object is, based solely upon visual stimulus which is impaired by the fog. Since the brightness of the object, such as a traffic light, is used by the brain to discriminate the distance of an object, and the fog reduces the brightness of objects, we perceive the object to be much farther away than it actually is (see also decision theoryDecision theory
Decision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
).
To apply signal detection theory to a data set where stimuli were either present or absent, and the observer categorized each trial as having the stimulus present or absent, the trials are sorted into one of four categories:
| Respond "Absent" | Respond "Present" | |
|---|---|---|
| Stimulus Present | Miss | Hit |
| Stimulus Absent | Correct Rejection | False Alarm |
Based on the proportions of these types of trials, numerical estimates of sensitivity can be obtained with statistics like the sensitivity index d'
D'
The sensitivity index or d' is a statistic used in signal detection theory. It provides the separation between the means of the signal and the noise distributions, in units of the standard deviation of the noise distribution....
and A', and response bias can be estimated with statistics like β.
Signal detection theory can also be applied to memory experiments, where items are presented on a study list for later testing. A test list is created by combining these 'old' items with novel, 'new' items that did not appear on the study list. On each test trial the subject will respond 'yes, this was on the study list' or 'no, this was not on the study list'. Items presented on the study list are called Targets, and new items are called Distractors. Saying 'Yes' to a target constitutes a Hit, while saying 'Yes' to a distractor constitutes a False Alarm.
| Respond "No" | Respond "Yes" | |
|---|---|---|
| Target | Miss | Hit |
| Distractor | Correct Rejection | False Alarm |
Applications
Signal Detection Theory has wide application, both in humans and other animalsComparative psychology
Comparative psychology generally refers to the scientific study of the behavior and mental processes of non-human animals. However, scientists from different disciplines do not always agree on this definition...
. Topics include memory
Memory
In psychology, memory is an organism's ability to store, retain, and recall information and experiences. Traditional studies of memory began in the fields of philosophy, including techniques of artificially enhancing memory....
, stimulus characterists of schedules of reinforcement, etc.
Sensitivity or discriminability
Conceptually, sensitivity refers to how hard or easy it is to detect that a target stimulus is present from background events. For example, in a recognition memory paradigm, having longer to study to-be-remembered words makes it easier to recognize previously seen or heard words. In contrast, having to remember 30 words rather than 5 makes the discrimination harder. One of the most commonly used statistics for computing sensitivity is the so-called sensitivity indexD'
The sensitivity index or d' is a statistic used in signal detection theory. It provides the separation between the means of the signal and the noise distributions, in units of the standard deviation of the noise distribution....
, or d'
D'
The sensitivity index or d' is a statistic used in signal detection theory. It provides the separation between the means of the signal and the noise distributions, in units of the standard deviation of the noise distribution....
. There are also non-parametric measures, such as the area under the ROC-curve
Receiver operating characteristic
In signal detection theory, a receiver operating characteristic , or simply ROC curve, is a graphical plot of the sensitivity, or true positive rate, vs. false positive rate , for a binary classifier system as its discrimination threshold is varied...
.
Bias
Bias is the extent to which one response is more probable than another. That is, a receiver may be more likely to respond that a stimulus is present or more likely to respond that a stimulus is not present. Bias is independent of sensitivity. For example, if there is a penalty for either false alarms or misses, this may influence bias. If the stimulus is a bomber, then a miss (failing to detect the plane) may increase deaths, so a liberal bias is likely. In contrast, crying wolf (a false alarm) too often may make people less likely to respond, grounds for a conservative bias.P(H1|y) > P(H2|y) / MAP Testing
In the case of making a decision between two hypothesesHypothesis
A hypothesis is a proposed explanation for a phenomenon. The term derives from the Greek, ὑποτιθέναι – hypotithenai meaning "to put under" or "to suppose". For a hypothesis to be put forward as a scientific hypothesis, the scientific method requires that one can test it...
, H1, absent, and H2, present, in the event of a particular observation
Observation
Observation is either an activity of a living being, such as a human, consisting of receiving knowledge of the outside world through the senses, or the recording of data using scientific instruments. The term may also refer to any data collected during this activity...
, y, a classical approach is to choose H1 when p(H1|y) > p(H2|y) and H2 in the reverse case. In the event that the two a posteriori
A Posteriori
Apart from the album, some additional remixes were released exclusively through the iTunes Store. They are:*"Eppur si muove" – 6:39*"Dreaming of Andromeda" Apart from the album, some additional remixes were released exclusively through the iTunes Store. They are:*"Eppur si muove" (Tocadisco...
probabilities
Probability
Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...
are equal, one typically defaults to a single choice, say H2. One could also flip a coin although the expected number of errors would be the same.
When taking this approach, usually what one knows are the conditional probabilities, p(y|H1) and p(y|H2), and the a priori probabilities
and . In this case,,where p(y) is the total probability of event y,
.H2 is chosen in case
and H1 otherwise.
Often, the ratio
is called and is called , the likelihood ratio.Using this terminology, H2 is chosen in case
. This is called MAP testing, where MAP stands for "maximum a posteriori").Taking this approach minimizes the expected number of errors one will make.
Bayes Criterion
In some cases, it is far more important to respond appropriately to H1 than it is to respond appropriately to H2. For example, if one is trying to detect an incoming bomber known to be carrying a nuclear weaponNuclear weapon
A nuclear weapon is an explosive device that derives its destructive force from nuclear reactions, either fission or a combination of fission and fusion. Both reactions release vast quantities of energy from relatively small amounts of matter. The first fission bomb test released the same amount...
, it is much more important to shoot down the bomber if it is there, than it is not to send a fighter squadron to inspect a false alarm (assuming a large supply of fighter squadrons). The Bayes
Thomas Bayes
Thomas Bayes was an English mathematician and Presbyterian minister, known for having formulated a specific case of the theorem that bears his name: Bayes' theorem...
criterion is an approach suitable for such cases.
Here a utility
Utility
In economics, utility is a measure of customer satisfaction, referring to the total satisfaction received by a consumer from consuming a good or service....
is associated with each of four situations:
: One responds with behavior appropriate to H1 and H1 is true: fighters destroy bomber, incurring fuel, maintenance, and weapons costs, take risk of some being shot down;
: One responds with behavior appropriate to H1 and H2 is true: fighters sent out, incurring fuel and maintenance costs, bomber location remains unknown;
: One responds with behavior appropriate to H2 and H1 is true: city destroyed;
: One responds with behavior appropriate to H2 and H2 is true: fighters stay home, bomber location remains unknown;
As is shown below, what is important are the differences,
and .Similarly, there are four probabilities,
, , etc., for each of the cases (which are dependent on one's decision strategy).The Bayes criterion approach is to maximize the expected utility:
Effectively, one may maximize the sum,
,and make the following substitutions:
where
and are the a priori probabilities, and , and is the region of observation events, y, that are responded to as though H1 is true. and thus are maximized by extending over the region whereThis is accomplished by deciding H2 in case
and H1 otherwise, where L(y) is the so-defined likelihood ratio.
See also
- Constant false alarm rateConstant false alarm rateConstant false alarm rate detection refers to a common form of adaptive algorithm used in radar systems to detect target returns against a background of noise, clutter and interference.Other detection algorithms are not adaptive...
- DemodulationDemodulationDemodulation is the act of extracting the original information-bearing signal from a modulated carrier wave.A demodulator is an electronic circuit that is used to recover the information content from the modulated carrier wave.These terms are traditionally used in connection with radio receivers,...
- Detector (radio)Detector (radio)A detector is a device that recovers information of interest contained in a modulated wave. The term dates from the early days of radio when all transmissions were in Morse code, and it was only necessary to detect the presence of a radio wave using a device such as a coherer without necessarily...
- Decision theoryDecision theoryDecision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
- Estimation theoryEstimation theoryEstimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component. The parameters describe an underlying physical setting in such a way that their value affects the distribution of the...
- Likelihood-ratio testLikelihood-ratio testIn statistics, a likelihood ratio test is a statistical test used to compare the fit of two models, one of which is a special case of the other . The test is based on the likelihood ratio, which expresses how many times more likely the data are under one model than the other...
- ModulationModulationIn electronics and telecommunications, modulation is the process of varying one or more properties of a high-frequency periodic waveform, called the carrier signal, with a modulating signal which typically contains information to be transmitted...
- Neyman-Pearson lemma
- Receiver operating characteristicReceiver operating characteristicIn signal detection theory, a receiver operating characteristic , or simply ROC curve, is a graphical plot of the sensitivity, or true positive rate, vs. false positive rate , for a binary classifier system as its discrimination threshold is varied...
- Statistical signal processingStatistical signal processingStatistical signal processing is an area of Applied Mathematics and Signal Processing that treats signals as stochastic processes, dealing with their statistical properties...
- Just noticeable differenceJust noticeable differenceIn psychophysics, a just noticeable difference, customarily abbreviated with lowercase letters as jnd, is the smallest detectable difference between a starting and secondary level of a particular sensory stimulus...
- Psychometric functionPsychometric functionA psychometric function describes the relationship between a parameter of a physical stimulus and the responses of a person who has to decide about a certain aspect of that stimulus. Usually these sensory decisions take the form of a two-alternative forced choice . The psychometric function usually...
- Type I and type II errorsType I and type II errorsIn statistical test theory the notion of statistical error is an integral part of hypothesis testing. The test requires an unambiguous statement of a null hypothesis, which usually corresponds to a default "state of nature", for example "this person is healthy", "this accused is not guilty" or...
External links
- A Description of Signal Detection Theory
- An application of SDT to safety
- Signal Detection Theory by Garrett Neske, The Wolfram Demonstrations Project