Linear time-invariant system theory, commonly known as LTI system theory, comes from
applied mathematicsApplied mathematics is a branch of mathematics that concerns itself with mathematical methods that are typically used in science, engineering, business, and industry. Thus, "applied mathematics" is a mathematical science with specialized knowledge...
and has direct applications in
NMR spectroscopyNuclear magnetic resonance spectroscopy, most commonly known as NMR spectroscopy, is a research technique that exploits the magnetic properties of certain atomic nuclei to determine physical and chemical properties of atoms or the molecules in which they are contained...
,
seismologySeismology is the scientific study of earthquakes and the propagation of elastic waves through the Earth or through other planet-like bodies. The field also includes studies of earthquake effects, such as tsunamis as well as diverse seismic sources such as volcanic, tectonic, oceanic,...
,
circuitAn electrical network is an interconnection of electrical elements such as resistors, inductors, capacitors, transmission lines, voltage sources, current sources and switches. An electrical circuit is a special type of network, one that has a closed loop giving a return path for the current...
s,
signal processingSignal processing is an area of systems engineering, electrical engineering and applied mathematics that deals with operations on or analysis of signals, in either discrete or continuous time...
,
control theoryControl theory is an interdisciplinary branch of engineering and mathematics that deals with the behavior of dynamical systems. The desired output of a system is called the reference...
, and other technical areas. It investigates the response of a
linearA linear system is a mathematical model of a system based on the use of a linear operator.Linear systems typically exhibit features and properties that are much simpler than the general, nonlinear case....
and
time-invariant systemA time-invariant system is one whose output does not depend explicitly on time.This property can be satisfied if the transfer function of the system is not a function of time except expressed by the input and output....
to an arbitrary input signal. Trajectories of these systems are commonly measured and tracked as they move through time (e.g., an acoustic waveform), but in applications like
image processingIn electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...
and
field theoryA classical field theory is a physical theory that describes the study of how one or more physical fields interact with matter. The word 'classical' is used in contrast to those field theories that incorporate quantum mechanics ....
, the LTI systems also have trajectories in spatial dimensions. Thus these systems are also called linear translation-invariant to give the theory the most general reach. In the case of generic discrete-time (i.e., sampled) systems, linear shift-invariant is the corresponding term. A good example of LTI systems are electrical circuits that can be made up of resistors, capacitors and inductors.
Overview
The defining properties of any LTI system are linearity and time invariance.
-
- Input
produces output 
- In particular,
- where
and 
are scalars and inputs that vary over a continuum indexed by 
. Thus if an input function can be represented by a continuum of input functions, combined "linearly", as shown, then the corresponding output function can be represented by the corresponding continuum of output functions, scaled and summed in the same way.
- Time invariance means that whether we apply an input to the system now or T seconds from now, the output will be identical except for a time delay of the T seconds. That is, if the output due to input
is 
, then the output due to input 
is 
. Hence, the system is time invariant because the output does not depend on the particular time the input is applied.
The fundamental result in LTI system theory is that any LTI system can be characterized entirely by a single function called the system's
impulse responseIn signal processing, the impulse response, or impulse response function , of a dynamic system is its output when presented with a brief input signal, called an impulse. More generally, an impulse response refers to the reaction of any dynamic system in response to some external change...
. The output of the system is simply the
convolutionIn mathematics and, in particular, functional analysis, convolution is a mathematical operation on two functions f and g, producing a third function that is typically viewed as a modified version of one of the original functions. Convolution is similar to cross-correlation...
of the input to the system with the system's impulse response. This method of analysis is often called the
time domainTime domain is a term used to describe the analysis of mathematical functions, physical signals or time series of economic or environmental data, with respect to time. In the time domain, the signal or function's value is known for all real numbers, for the case of continuous time, or at various...
point-of-view. The same result is true of discrete-time linear shift-invariant systems in which signals are discrete-time samples, and convolution is defined on sequences.
Equivalently, any LTI system can be characterized in the
frequency domainIn electronics, control systems engineering, and statistics, frequency domain is a term used to describe the domain for analysis of mathematical functions or signals with respect to frequency, rather than time....
by the system's
transfer functionA transfer function is a mathematical representation, in terms of spatial or temporal frequency, of the relation between the input and output of a linear time-invariant system. With optical imaging devices, for example, it is the Fourier transform of the point spread function i.e...
, which is the
Laplace transform of the system's impulse response (or Z transform in the case of discrete-time systems). As a result of the properties of these transforms, the output of the system in the frequency domain is the product of the transfer function and the transform of the input. In other words, convolution in the time domain is equivalent to multiplication in the frequency domain.
For all LTI systems, the
eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
s, and the basis functions of the transforms, are
complexA complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...
exponentialsIn mathematics, the exponential function is the function ex, where e is the number such that the function ex is its own derivative. The exponential function is used to model a relationship in which a constant change in the independent variable gives the same proportional change In mathematics,...
. This is, if the input to a system is the complex waveform
for some complex amplitude
and complex frequency
, the output will be some complex constant times the input, say
for some new complex amplitude
. The ratio
is the transfer function at frequency
.
Because
sinusoidsThe sine wave or sinusoid is a mathematical function that describes a smooth repetitive oscillation. It occurs often in pure mathematics, as well as physics, signal processing, electrical engineering and many other fields...
are a sum of complex exponentials with complex-conjugate frequencies, if the input to the system is a sinusoid, then the output of the system will also be a sinusoid, perhaps with a different
amplitudeAmplitude is the magnitude of change in the oscillating variable with each oscillation within an oscillating system. For example, sound waves in air are oscillations in atmospheric pressure and their amplitudes are proportional to the change in pressure during one oscillation...
and a different
phasePhase in waves is the fraction of a wave cycle which has elapsed relative to an arbitrary point.-Formula:The phase of an oscillation or wave refers to a sinusoidal function such as the following:...
, but always with the same frequency. LTI systems cannot produce frequency components that are not in the input.
LTI system theory is good at describing many important systems. Most LTI systems are considered "easy" to analyze, at least compared to the time-varying and/or nonlinear case. Any system that can be modeled as a linear homogeneous
differential equationA differential equation is a mathematical equation for an unknown function of one or several variables that relates the values of the function itself and its derivatives of various orders...
with constant coefficients is an LTI system. Examples of such systems are
electrical circuitsAn electrical network is an interconnection of electrical elements such as resistors, inductors, capacitors, transmission lines, voltage sources, current sources and switches. An electrical circuit is a special type of network, one that has a closed loop giving a return path for the current...
made up of
resistorA linear resistor is a linear, passive two-terminal electrical component that implements electrical resistance as a circuit element.The current through a resistor is in direct proportion to the voltage across the resistor's terminals. Thus, the ratio of the voltage applied across a resistor's...
s,
inductorAn inductor is a passive two-terminal electrical component used to store energy in a magnetic field. An inductor's ability to store magnetic energy is measured by its inductance, in units of henries...
s, and
capacitorA capacitor is a passive two-terminal electrical component used to store energy in an electric field. The forms of practical capacitors vary widely, but all contain at least two electrical conductors separated by a dielectric ; for example, one common construction consists of metal foils separated...
s (RLC circuits). Ideal spring–mass–damper systems are also LTI systems, and are mathematically equivalent to RLC circuits.
Most LTI system concepts are similar between the continuous-time and discrete-time (linear shift-invariant) cases. In image processing, the time variable is replaced with two space variables, and the notion of time invariance is replaced by two-dimensional shift invariance. When analyzing
filter bankIn signal processing, a filter bank is an array of band-pass filters that separates the input signal into multiple components, each one carrying a single frequency subband of the original signal. One application of a filter bank is a graphic equalizer, which can attenuate the components...
s and MIMO systems, it is often useful to consider
vectorsIn computer science, an array data structure or simply array is a data structure consisting of a collection of elements , each identified by at least one index...
of signals.
A linear system that is not time-invariant can be solved using other approaches such as the
Green functionGreen function might refer to:*Green's function of a differential operator.*Deligne–Lusztig theory in the representation theory of finite groups of Lie type.*Green's function in many-body theory....
method. The same method must be used when the initial conditions of the problem are not null.
Impulse response and convolution
The behavior of a linear, continuous-time, time-invariant system with input signal x(t) and output signal y(t) is described by the convolution integral, :
where
is the system's response to an
impulseThe Dirac delta function, or δ function, is a generalized function depending on a real parameter such that it is zero for all values of the parameter except when the parameter is zero, and its integral over the parameter from −∞ to ∞ is equal to one. It was introduced by theoretical...
:
is therefore proportional to a weighted average of the input function
The weighting function is
simply shifted by amount
As
changes, the weighting function emphasizes different parts of the input function. When
is zero for all negative
depends only on values of
prior to time
and the system is said to be
causalA causal system is a system where the output depends on past/current inputs but not future inputs i.e...
.
To understand why the convolution produces the output of an LTI system, let the notation
represent the function
with variable
and constant
And let the shorter notation
represent
Then a continuous-time system transforms an input function,
into an output function,
And in general, every value of the output can depend on every value of the input. This concept is represented by:
where
is the transformation operator for time
In a typical system,
depends most heavily on the values of
that occurred near time
Unless the transform itself changes with
the output function is just constant, and the system is uninteresting.
For a linear system,
must satisfy :
And the time-invariance requirement is:
In this notation, we can write the impulse response as
Similarly:
Substituting this result into the convolution integral:
which has the form of the right side of for the case
and
then allows this continuation:
In summary, the input function,
can be represented by a continuum of time-shifted impulse functions, combined "linearly", as shown at . The system's linearity property allows the system's response to be represented by the corresponding continuum of impulse
responses, combined in the same way. And the time-invariance property allows that combination to be represented by the convolution integral.
The mathematical operations above have a simple graphical simulation.
Exponentials as eigenfunctions
An
eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
is a function for which the output of the operator is a scaled version of the same function. That is,
,
where f is the eigenfunction and
is the eigenvalue, a constant.
The
exponential functionIn mathematics, the exponential function is the function ex, where e is the number such that the function ex is its own derivative. The exponential function is used to model a relationship in which a constant change in the independent variable gives the same proportional change In mathematics,...
s
, where
, are
eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
s of a
linearIn mathematics, a linear map or function f is a function which satisfies the following two properties:* Additivity : f = f + f...
, time-invariant operator. A simple proof illustrates this concept. Suppose the input is
. The output of the system with impulse response
is then
which, by the commutative property of
convolutionIn mathematics and, in particular, functional analysis, convolution is a mathematical operation on two functions f and g, producing a third function that is typically viewed as a modified version of one of the original functions. Convolution is similar to cross-correlation...
, is equivalent to
where the scalar
is dependent only on the parameter s.
So the system's response is a scaled version of the input. In particular, for any
, the system output is the product of the input
and the constant
. Hence,
is an
eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
of an LTI system, and the corresponding eigenvalue is
.
Fourier and Laplace transforms
The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The
Laplace transform
is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids (i.e., exponential functions of the form
where
and
). These are generally called complex exponentials even though the argument is purely imaginary. The
Fourier transformIn mathematics, Fourier analysis is a subject area which grew from the study of Fourier series. The subject began with the study of the way general functions may be represented by sums of simpler trigonometric functions...
gives the eigenvalues for pure complex sinusoids. Both of
and
are called the system function, system response, or transfer function.
The Laplace transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality, with the transform integral being taken from zero to infinity (the transform shown above with lower limit of integration of negative infinity is formally known as the bilateral Laplace transform).
The Fourier transform is used for analyzing systems that process signals that are infinite in extent, such as modulated sinusoids, even though it cannot be directly applied to input and output signals that are not square integrable. The Laplace transform actually works directly for these signals if they are zero before a start time, even if they are not square integrable, for stable systems. The Fourier transform is often applied to spectra of infinite signals via the
Wiener–Khinchin theoremThe Wiener–Khinchin theorem states that the power spectral density of a wide–sense stationary random process is the Fourier transform of the corresponding autocorrelation function.-History:Norbert Wiener first published the result in...
even when Fourier transforms of the signals do not exist.
Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain, given signals for which the transforms exist
Not only is it often easier to do the transforms, multiplication, and inverse transform than the original convolution, but one can also gain insight into the behavior of the system from the system response. One can look at the modulus of the system function |H(s)| to see whether the input
is passed (let through) the system or rejected or attenuated by the system (not let through).
Examples
- A simple example of an LTI operator is the derivative
In calculus, a branch of mathematics, the derivative is a measure of how a function changes as its input changes. Loosely speaking, a derivative can be thought of as how much one quantity is changing in response to changes in some other quantity; for example, the derivative of the position of a...
.
-
(i.e., it is linear)
-
(i.e., it is time invariant)
- When the Laplace transform of the derivative is taken, it transforms to a simple multiplication by the Laplace variable s.
- That the derivative has such a simple Laplace transform partly explains the utility of the transform.
- Another simple LTI operator is an averaging operator
-
- By the linearity of integration,
- it is linear. Additionally, because
- it is time invariant. In fact,
can be written as a convolution with the boxcar functionIn mathematics, a boxcar function is any function which is zero over the entirereal line except for a single interval where it is equal to a constant, A; it is a simple step function...
. That is,
- where the boxcar function
Important system properties
Some of the most important properties of a system are causality and stability. Causality is a necessity if the independent variable is time, but not all systems have time as an independent variable. For example, a system that processes still images does not need to be causal. Non-causal systems can be built and can be useful in many circumstances. Even non-real systems can be built and are very useful in many contexts.
Causality
A system is causal if the output depends only on present and past inputs. A necessary and sufficient condition for causality is
where 
is the impulse response. It is not possible in general to determine causality from the Laplace transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined.
Stability
A system is bounded-input, bounded-output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if every input satisfying
leads to an output satisfying
(that is, a finite maximum absolute value of 
implies a finite maximum absolute value of 
), then the system is stable. A necessary and sufficient condition is that 
, the impulse response, is in L1In mathematics, the Lp spaces are function spaces defined using a natural generalization of the p-norm for finite-dimensional vector spaces...
(has a finite L1 norm):
In the frequency domain, the region of convergence must contain the imaginary axis 
.
As an example, the ideal low-pass filterA low-pass filter is an electronic filter that passes low-frequency signals but attenuates signals with frequencies higher than the cutoff frequency. The actual amount of attenuation for each frequency varies from filter to filter. It is sometimes called a high-cut filter, or treble cut filter...
with impulse response equal to a sinc function is not BIBO stable, because the sinc function does not have a finite L1 norm. Thus, for some bounded input, the output of the ideal low-pass filter is unbounded. In particular, if the input is zero for 
and equal to a sinusoid at the cut-off frequency for 
, then the output will be unbounded for all times other than the zero crossings.
Discrete-time systems
Almost everything in continuous-time systems has a counterpart in discrete-time systems.
Discrete-time systems from continuous-time systems
In many contexts, a discrete time (DT) system is really part of a larger continuous time (CT) system. For example, a digital recording system takes an analog sound, digitizes it, possibly processes the digital signals, and plays back an analog sound for people to listen to.
Formally, the DT signals studied are almost always uniformly sampled versions of CT signals. If 
is a CT signal, then an analog to digital converter will transform it to the DT signal:
where T is the sampling period. It is very important to limit the range of frequencies in the input signal for faithful representation in the DT signal, since then the sampling theorem guarantees that no information about the CT signal is lost. A DT signal can only contain a frequency range of 
; other frequencies are aliasedIn signal processing and related disciplines, aliasing refers to an effect that causes different signals to become indistinguishable when sampled...
to the same range.
Impulse response and convolution
Let 
represent the sequence 
.
And let the shorter notation 
represent 
A discrete system transforms an input sequence, 
into an output sequence, 
In general, every element of the output can depend on every element of the input. Representing the transformation operator by 
, we can write:
Note that unless the transform itself changes with n, the output sequence is just constant, and the system is uninteresting. (Thus the subscript, n.) In a typical system, y[n] depends most heavily on the elements of x whose indices are near n.
For the special case of the Kronecker delta function, 
the output sequence is the impulse response:
For a linear system, 
must satisfy:
And the time-invariance requirement is:
In such a system, the impulse response, 
characterizes the system completely. I.e., for any input sequence, the output sequence can be calculated in terms of the input and the impulse response. To see how that is done, consider the identity:
which expresses 
in terms of a sum of weighted delta functions.
Therefore:
where we have invoked for the case 
and 
And because of , we may write:
Therefore:
which is the familiar discrete convolution formula. The operator 
can therefore be interpreted as proportional to a weighted average of the function x[k].
The weighting function is h[-k], simply shifted by amount n. As n changes, the weighting function emphasizes different parts of the input function. Equivalently, the system's response to an impulse at n=0 is a "time" reversed copy of the unshifted weighting function. When h[k] is zero for all negative k, the system is said to be causalA causal system is a system where the output depends on past/current inputs but not future inputs i.e...
.
Exponentials as eigenfunctions
An eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
is a function for which the output of the operator is the same function, just scaled by some amount. In symbols,
,
where f is the eigenfunction and 
is the eigenvalue, a constant.
The exponential functionIn mathematics, the exponential function is the function ex, where e is the number such that the function ex is its own derivative. The exponential function is used to model a relationship in which a constant change in the independent variable gives the same proportional change In mathematics,...
s 
, where 
, are eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
s of a linearIn mathematics, a linear map or function f is a function which satisfies the following two properties:* Additivity : f = f + f...
, time-invariant operator. 
is the sampling interval, and 
. A simple proof illustrates this concept.
Suppose the input is 
. The output of the system with impulse response 
is then
which is equivalent to the following by the commutative property of convolutionIn mathematics and, in particular, functional analysis, convolution is a mathematical operation on two functions f and g, producing a third function that is typically viewed as a modified version of one of the original functions. Convolution is similar to cross-correlation...
where
is dependent only on the parameter z.
So 
is an eigenfunctionIn mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
of an LTI system because the system response is the same as the input times the constant 
.
Z and discrete-time Fourier transforms
The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Z transform
is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids, i.e. exponentials of the form 
, where 
. These can also be written as 
with 
. These are generally called complex exponentials even though the argument is purely imaginary.
The Discrete-time Fourier transformIn mathematics, the discrete-time Fourier transform is one of the specific forms of Fourier analysis. As such, it transforms one function into another, which is called the frequency domain representation, or simply the "DTFT", of the original function . But the DTFT requires an input function...
(DTFT) 
gives the eigenvalues of pure sinusoids. Both of 
and 
are called the system function, system response, or transfer function.
The Z transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality. The Fourier transform is used for analyzing signals that are infinite in extent.
Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain. That is,
Just as with the Laplace transform transfer function in continuous-time system analysis, the Z transform makes it easier to analyze systems and gain insight into their behavior. One can look at the modulus of the system function |H(z)| to see whether the input 
is passed (let through) by the system, or rejected or attenuated by the system (not let through).
Examples
- A simple example of an LTI operator is the delay operator
.
(i.e., it is linear)
(i.e., it is time invariant)
- The Z transform of the delay operator is a simple multiplication by z-1. That is,
- Another simple LTI operator is the averaging operator
-
.
- Because of the linearity of sums,
- and so it is linear. Because,
- it is also time invariant.
Important system properties
The input-output characteristics of discrete-time LTI system are completely described by its impulse response 
.
Some of the most important properties of a system are causality and stability. Unlike CT systems, non-causal DT systems can be realized. It is trivial to make an acausal FIRA finite impulse response filter is a type of a signal processing filter whose impulse response is of finite duration, because it settles to zero in finite time. This is in contrast to infinite impulse response filters, which have internal feedback and may continue to respond indefinitely...
system causal by adding delays. It is even possible to make acausal IIRIIR may refer to* IIR Holdings, a human capital development company acquired by Informa* Indo-Iranian languages* Infinite impulse response...
systems. Non-stable systems can be built and can be useful in many circumstances. Even non-real systems can be built and are very useful in many contexts.
Causality
A discrete-time LTI system is causal if the current value of the output depends on only the current value and past values of the input., A necessary and sufficient condition for causality is
where 
is the impulse response. It is not possible in general to determine causality from the Z transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined.
Stability
A system is bounded input, bounded output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if
implies that
(that is, if bounded input implies bounded output, in the sense that the maximum absolute values of 
and 
are finite), then the system is stable. A necessary and sufficient condition is that 
, the impulse response, satisfies
In the frequency domain, the region of convergence must contain the unit circleIn mathematics, a unit circle is a circle with a radius of one. Frequently, especially in trigonometry, "the" unit circle is the circle of radius one centered at the origin in the Cartesian coordinate system in the Euclidean plane...
(i.e., the locusIn geometry, a locus is a collection of points which share a property. For example a circle may be defined as the locus of points in a plane at a fixed distance from a given point....
satisfying 
for complex z).
See also
- Circulant matrix
In linear algebra, a circulant matrix is a special kind of Toeplitz matrix where each row vector is rotated one element to the right relative to the preceding row vector. In numerical analysis, circulant matrices are important because they are diagonalized by a discrete Fourier transform, and hence...
- Frequency response
Frequency response is the quantitative measure of the output spectrum of a system or device in response to a stimulus, and is used to characterize the dynamics of the system. It is a measure of magnitude and phase of the output as a function of frequency, in comparison to the input...
- Impulse response
In signal processing, the impulse response, or impulse response function , of a dynamic system is its output when presented with a brief input signal, called an impulse. More generally, an impulse response refers to the reaction of any dynamic system in response to some external change...
- System analysis
System analysis in the field of electrical engineering characterizes electrical systems and their properties. System Analysis can be used to represent almost anything from population growth to audio speakers, electrical engineers often use it because of its direct relevance to many areas of their...
- Green function
Green function might refer to:*Green's function of a differential operator.*Deligne–Lusztig theory in the representation theory of finite groups of Lie type.*Green's function in many-body theory....
External links