Trace (linear algebra)

In linear algebra

Linear algebra

Linear algebra is a branch of mathematics that studies vector spaces, also called linear spaces, along with linear functions that input one vector and output another. Such functions are called linear maps and can be represented by matrices if a basis is given. Thus matrix theory is often...

, the trace of an n-by-n square matrix A is defined to be the sum of the elements on the main diagonal (the diagonal from the upper left to the lower right) of A, i.e.,

where a_ii represents the entry on the ith row and ith column of A. The trace of a matrix is the sum of the (complex) eigenvalues, and it is an invariant

Invariants of tensors

In mathematics, in the fields of multilinear algebra and representation theory, invariants of tensors are coefficients of the characteristic polynomial of the tensor A:\ p:=\det ,...

 with respect to a change of basis

Change of basis

In linear algebra, change of basis refers to the conversion of vectors and linear transformations between matrix representations which have different bases.-Expression of a basis:...

. This characterization can be used to define the trace for a linear operator in general. Note that the trace is only defined for a square matrix (i.e. n×n).

Geometrically, the trace can be interpreted as the infinitesimal change in volume (as the derivative of the determinant

Determinant

In linear algebra, the determinant is a value associated with a square matrix. It can be computed from the entries of the matrix by a specific arithmetic expression, while other ways to determine its value exist as well...

), which is made precise in Jacobi's formula

Jacobi's formula

In matrix calculus, Jacobi's formula expresses the derivative of the determinant of a matrix A in terms of the adjugate of A and the derivative of A...

.

The term trace is a calque

Calque

In linguistics, a calque or loan translation is a word or phrase borrowed from another language by literal, word-for-word or root-for-root translation.-Calque:...

 from the German Spur (cognate

Cognate

In linguistics, cognates are words that have a common etymological origin. This learned term derives from the Latin cognatus . Cognates within the same language are called doublets. Strictly speaking, loanwords from another language are usually not meant by the term, e.g...

 with the English spoor), which, as a function in mathematics, is often abbreviated to "Sp".

Example

Let T be a linear operator represented by the matrix


Then .

The trace of the identity matrix

Identity matrix

In linear algebra, the identity matrix or unit matrix of size n is the n×n square matrix with ones on the main diagonal and zeros elsewhere. It is denoted by In, or simply by I if the size is immaterial or can be trivially determined by the context...

 is the dimension of the space; this leads to generalizations of dimension using trace. The trace of a projection (i.e., P² = P) is the rank

Rank (linear algebra)

The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...

 of the projection. The trace of a nilpotent matrix

Nilpotent matrix

In linear algebra, a nilpotent matrix is a square matrix N such thatN^k = 0\,for some positive integer k. The smallest such k is sometimes called the degree of N....

 is zero. The product of a symmetric matrix and a skew-symmetric matrix

Skew-symmetric matrix

In mathematics, and in particular linear algebra, a skew-symmetric matrix is a square matrix A whose transpose is also its negative; that is, it satisfies the equation If the entry in the and is aij, i.e...

 has zero trace.

More generally, if is the characteristic polynomial

Characteristic polynomial

In linear algebra, one associates a polynomial to every square matrix: its characteristic polynomial. This polynomial encodes several important properties of the matrix, most notably its eigenvalues, its determinant and its trace....

 of a matrix A, then

If A and B are positive semi-definite matrices of the same order then

Properties

The trace is a linear map. That is,



for all square matrices A and B, and all scalar

Scalar (mathematics)

In linear algebra, real numbers are called scalars and relate to vectors in a vector space through the operation of scalar multiplication, in which a vector can be multiplied by a number to produce another vector....

s c.

If A is an m×n matrix and B is an n×m matrix, then

Conversely, the above properties characterize the trace completely in the sense as follows. Let be a linear functional

Linear functional

In linear algebra, a linear functional or linear form is a linear map from a vector space to its field of scalars.  In Rn, if vectors are represented as column vectors, then linear functionals are represented as row vectors, and their action on vectors is given by the dot product, or the...

 on the space of square matrices satisfying . Then and tr are proportional.

The trace is similarity-invariant, which means that A and P⁻¹AP have the same trace. This is because


A matrix and its transpose

Transpose

In linear algebra, the transpose of a matrix A is another matrix AT created by any one of the following equivalent actions:...

 have the same trace:


Let A be a symmetric matrix, and B an anti-symmetric matrix. Then


When both A and B are n by n, the trace of the (ring-theoretic) commutator

Commutator

In mathematics, the commutator gives an indication of the extent to which a certain binary operation fails to be commutative. There are different definitions used in group theory and ring theory.-Group theory:...

 of A and B vanishes: tr([A, B]) = 0; one can state this as "the trace is a map of Lie algebras from operators to scalars", as the commutator of scalars is trivial (it is an abelian Lie algebra). In particular, using similarity invariance, it follows that the identity matrix is never similar to the commutator of any pair of matrices.

Conversely, any square matrix with zero trace is the commutator of some pair of matrices. Moreover, any square matrix with zero trace is unitarily equivalent

Unitary equivalence

*For unitary equivalence of bounded operators in Hilbert space, see self-adjoint operator.*For unitary equivalence of unitary representations see that page....

 to a square matrix with diagonal consisting of all zeros.

The trace of any power of a nilpotent matrix

Nilpotent matrix

In linear algebra, a nilpotent matrix is a square matrix N such thatN^k = 0\,for some positive integer k. The smallest such k is sometimes called the degree of N....

 is zero. When the characteristic of the base field is zero, the converse also holds: if for all , then is nilpotent.

Note that order does matter in taking traces: in general,

In other words, we can only interchange the two halves of the expression, albeit repeatedly. This means that the trace is invariant under cyclic permutations, i.e.,


However, if products of three symmetric matrices are considered, any permutation is allowed. (Proof: tr(ABC) = tr(A^T B^T C^T) = tr((CBA)^T) = tr(CBA).) For more than three factors this is not true. This is known as the cyclic property.

Unlike the determinant

Determinant

In linear algebra, the determinant is a value associated with a square matrix. It can be computed from the entries of the matrix by a specific arithmetic expression, while other ways to determine its value exist as well...

, the trace of the product is not the product of traces. What is true is that the trace of the tensor product

Tensor product

In mathematics, the tensor product, denoted by ⊗, may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras, topological vector spaces, and modules, among many other structures or objects. In each case the significance of the symbol is the same: the most general...

 of two matrices is the product of their traces:


The trace of a product can be rewritten as the sum of all elements from a Hadamard product (entry-wise product):


This should be more computationally efficient, since the matrix product of an matrix with an one (first and last dimensions must match to give a square matrix for the trace) has multiplications and additions, whereas the computation of the Hadamard version (entry-wise product) requires only multiplications followed by additions.

• The trace of a projection matrix is the dimension of the target space. If

then

Exponential trace

Expressions like exp(tr(A)), where A is a square matrix, occur so often in some fields (e.g. multivariate statistical theory), that a shorthand notation has become common:


This is sometimes referred to as the exponential trace function.

Trace of a linear operator

Given some linear map f : V → V (V is a finite-dimensional vector space

Vector space

A vector space is a mathematical structure formed by a collection of vectors: objects that may be added together and multiplied by numbers, called scalars in this context. Scalars are often taken to be real numbers, but one may also consider vector spaces with scalar multiplication by complex...

) generally, we can define the trace of this map by considering the trace of matrix representation

Representation theory

Representation theory is a branch of mathematics that studies abstract algebraic structures by representing their elements as linear transformations of vector spaces, and studiesmodules over these abstract algebraic structures...

 of f, that is, choosing a basis

Basis (linear algebra)

In linear algebra, a basis is a set of linearly independent vectors that, in a linear combination, can represent every vector in a given vector space or free module, or, more simply put, which define a "coordinate system"...

 for V and describing f as a matrix relative to this basis, and taking the trace of this square matrix. The result will not depend on the basis chosen, since different bases will give rise to similar matrices, allowing for the possibility of a basis-independent definition for the trace of a linear map.

Such a definition can be given using the canonical isomorphism between the space End(V) of linear maps on V and V⊗V^*, where V^* is the dual space

Dual space

In mathematics, any vector space, V, has a corresponding dual vector space consisting of all linear functionals on V. Dual vector spaces defined on finite-dimensional vector spaces can be used for defining tensors which are studied in tensor algebra...

 of V. Let v be in V and let f be in V^*. Then the trace of the decomposable element v⊗f is defined to be f(v); the trace of a general element is defined by linearity. Using an explicit basis for V and the corresponding dual basis for V^*, one can show that this gives the same definition of the trace as given above.

Eigenvalue relationships

If A is a square n-by-n matrix with real

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

 or complex

Complex number

A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...

 entries and if λ₁,...,λ_n are the eigenvalues of A (listed according to their algebraic multiplicities), then


This follows from the fact that A is always similar to its Jordan form, an upper triangular matrix

Triangular matrix

In the mathematical discipline of linear algebra, a triangular matrix is a special kind of square matrix where either all the entries below or all the entries above the main diagonal are zero...

 having λ₁,...,λ_n on the main diagonal. In contrast, the determinant

Determinant

In linear algebra, the determinant is a value associated with a square matrix. It can be computed from the entries of the matrix by a specific arithmetic expression, while other ways to determine its value exist as well...

 of is the product of its eigenvalues; i.e.,


More generally,

Derivatives

The trace is the derivative of the determinant: it is the Lie algebra

Lie algebra

In mathematics, a Lie algebra is an algebraic structure whose main use is in studying geometric objects such as Lie groups and differentiable manifolds. Lie algebras were introduced to study the concept of infinitesimal transformations. The term "Lie algebra" was introduced by Hermann Weyl in the...

 analog of the (Lie group

Lie group

In mathematics, a Lie group is a group which is also a differentiable manifold, with the property that the group operations are compatible with the smooth structure...

) map of the determinant. This is made precise in Jacobi's formula

Jacobi's formula

In matrix calculus, Jacobi's formula expresses the derivative of the determinant of a matrix A in terms of the adjugate of A and the derivative of A...

 for the derivative

Derivative

In calculus, a branch of mathematics, the derivative is a measure of how a function changes as its input changes. Loosely speaking, a derivative can be thought of as how much one quantity is changing in response to changes in some other quantity; for example, the derivative of the position of a...

 of the determinant (see under determinant

Determinant

In linear algebra, the determinant is a value associated with a square matrix. It can be computed from the entries of the matrix by a specific arithmetic expression, while other ways to determine its value exist as well...

). As a particular case, :
the trace is the derivative of the determinant at the identity. From this (or from the connection between the trace and the eigenvalues), one can derive a connection between the trace function, the exponential map between a Lie algebra and its Lie group (or concretely, the matrix exponential

Matrix exponential

In mathematics, the matrix exponential is a matrix function on square matrices analogous to the ordinary exponential function. Abstractly, the matrix exponential gives the connection between a matrix Lie algebra and the corresponding Lie group....

 function), and the determinant

Determinant

In linear algebra, the determinant is a value associated with a square matrix. It can be computed from the entries of the matrix by a specific arithmetic expression, while other ways to determine its value exist as well...

:

For example, consider the one-parameter family of linear transformations given by rotation through angle θ,


These transformations all have determinant 1, so they preserve area. The derivative of this family at θ = 0 is the antisymmetric matrix


which clearly has trace zero, indicating that this matrix represents an infinitesimal transformation which preserves area.

A related characterization of the trace applies to linear vector fields. Given a matrix A, define a vector field F on Rⁿ by F(x) = Ax. The components of this vector field are linear functions (given by the rows of A). The divergence

Divergence

In vector calculus, divergence is a vector operator that measures the magnitude of a vector field's source or sink at a given point, in terms of a signed scalar. More technically, the divergence represents the volume density of the outward flux of a vector field from an infinitesimal volume around...

 div F is a constant function, whose value is equal to tr(A).
By the divergence theorem

Divergence theorem

In vector calculus, the divergence theorem, also known as Gauss' theorem , Ostrogradsky's theorem , or Gauss–Ostrogradsky theorem is a result that relates the flow of a vector field through a surface to the behavior of the vector field inside the surface.More precisely, the divergence theorem...

, one can interpret this in terms of flows: if F(x) represents the velocity of a fluid at the location x, and U is a region in Rⁿ, the net flow of the fluid out of U is given by tr(A)· vol(U), where vol(U) is the volume

Volume

Volume is the quantity of three-dimensional space enclosed by some closed boundary, for example, the space that a substance or shape occupies or contains....

 of U.

The trace is a linear operator, hence it commutes with the derivative:

Applications

The trace is used to define characters

Character (mathematics)

In mathematics, a character is a special kind of function from a group to a field . There are at least two distinct, but overlapping meanings...

 of group representation

Group representation

In the mathematical field of representation theory, group representations describe abstract groups in terms of linear transformations of vector spaces; in particular, they can be used to represent group elements as matrices so that the group operation can be represented by matrix multiplication...

s. Two representations of a group are equivalent (up to change of basis on ) if for all .

The trace also plays a central role in the distribution of quadratic forms

Quadratic form (statistics)

If \epsilon is a vector of n random variables, and \Lambda is an n-dimensional symmetric matrix, then the scalar quantity\epsilon^T\Lambda\epsilonis known as a quadratic form in \epsilon.-Expectation:It can be shown that...

.

Lie algebra

The trace is a map of Lie algebras from the Lie algebra gl_n of operators on a n-dimensional space ( matrices) to the Lie algebra k of scalars; as k is abelian (the Lie bracket vanishes), the fact that this is a map of Lie algebras is exactly the statement that the trace of a bracket vanishes:

The kernel of this map, a matrix whose trace is zero

0 (number)

0 is both a numberand the numerical digit used to represent that number in numerals.It fulfills a central role in mathematics as the additive identity of the integers, real numbers, and many other algebraic structures. As a digit, 0 is used as a placeholder in place value systems...

, is often said to be or , and these matrices form the simple Lie algebra sl_n, which is the Lie algebra

Lie algebra

In mathematics, a Lie algebra is an algebraic structure whose main use is in studying geometric objects such as Lie groups and differentiable manifolds. Lie algebras were introduced to study the concept of infinitesimal transformations. The term "Lie algebra" was introduced by Hermann Weyl in the...

 of the special linear group

Special linear group

In mathematics, the special linear group of degree n over a field F is the set of n×n matrices with determinant 1, with the group operations of ordinary matrix multiplication and matrix inversion....

 of matrices with determinant 1. The special linear group consists of the matrices which do not change volume, while the special linear algebra is the matrices which infinitesimally do not change volume.

In fact, there is a internal direct sum decomposition of operators/matrices into traceless operators/matrices and scalars operators/matrices. The projection map onto scalar operators can be expressed in terms of the trace, concretely as:
Formally, one can compose the trace (the counit map) with the unit map of "inclusion of scalars" to obtain a map mapping onto scalars, and multiplying by n. Dividing by n makes this a projection, yielding the formula above.

In terms of short exact sequences, one has
which is analogous to
for Lie groups. However, the trace splits naturally (via times scalars) so but the splitting of the determinant would be as the nth root times scalars, and this does not in general define a function, so the determinant does not split and the general linear group does not decompose:

Bilinear forms

The bilinear form


is called the Killing form

Killing form

In mathematics, the Killing form, named after Wilhelm Killing, is a symmetric bilinear form that plays a basic role in the theories of Lie groups and Lie algebras...

, which is used for the classification of Lie algebras.

The trace defines a bilinear form:


(x, y square matrices).

The form is symmetric, non-degenerate and associative in the sense that:


In a simple Lie algebra (e.g., ), every such bilinear form is proportional to each other; in particular, to the Killing form.

Two matrices x and y are said to be trace orthogonal if

Inner product

For an m-by-n matrix A with complex (or real) entries and ^* being the conjugate transpose, we have


with equality if and only if A = 0. The assignment


yields an inner product on the space of all complex (or real) m-by-n matrices.

The norm

Norm (mathematics)

In linear algebra, functional analysis and related areas of mathematics, a norm is a function that assigns a strictly positive length or size to all vectors in a vector space, other than the zero vector...

 induced by the above inner product is called the Frobenius norm. Indeed it is simply the Euclidean norm if the matrix is considered as a vector of length mn.

Generalization

The concept of trace of a matrix is generalised to the trace class

Trace class

In mathematics, a trace class operator is a compact operator for which a trace may be defined, such that the trace is finite and independent of the choice of basis....

 of compact operator

Compact operator

In functional analysis, a branch of mathematics, a compact operator is a linear operator L from a Banach space X to another Banach space Y, such that the image under L of any bounded subset of X is a relatively compact subset of Y...

s on Hilbert space

Hilbert space

The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...

s, and the analog of the Frobenius norm is called the Hilbert–Schmidt norm.

The partial trace

Partial trace

In linear algebra and functional analysis, the partial trace is a generalization of the trace. Whereas the trace is a scalar valued function on operators, the partial trace is an operator-valued function...

 is another generalization of the trace that is operator-valued.

If A is a general associative algebra

Associative algebra

In mathematics, an associative algebra A is an associative ring that has a compatible structure of a vector space over a certain field K or, more generally, of a module over a commutative ring R...

 over a field k, then a trace on A is often defined to be any map tr: A → k which vanishes on commutators: tr([a, b]) = 0 for all a, b in A. Such a trace is not uniquely defined; it can always at least be modified by multiplication by a nonzero scalar.

A supertrace

Supertrace

In the theory of superalgebras, if A is a commutative superalgebra, V is a free right A-supermodule and T is an endomorphism from V to itself, then the supertrace of T, str is defined by the following trace diagram:...

 is the generalization of a trace to the setting of superalgebra

Superalgebra

In mathematics and theoretical physics, a superalgebra is a Z2-graded algebra. That is, it is an algebra over a commutative ring or field with a decomposition into "even" and "odd" pieces and a multiplication operator that respects the grading....

s.

The operation of tensor contraction

Tensor contraction

In multilinear algebra, a tensor contraction is an operation on one or more tensors that arises from the natural pairing of a finite-dimensional vector space and its dual. In components, it is expressed as a sum of products of scalar components of the tensor caused by applying the summation...

 generalizes the trace to arbitrary tensors.

Coordinate-free definition

We can identify the space of linear operators on a vector space with the space , where . We also have a canonical bilinear function that consists of applying an element of to an element of to get an element of , in symbols . This induces a linear function on the tensor product

Tensor product

In mathematics, the tensor product, denoted by ⊗, may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras, topological vector spaces, and modules, among many other structures or objects. In each case the significance of the symbol is the same: the most general...

 (by its universal property) which, as it turns out, when that tensor product is viewed as the space of operators, is equal to the trace.

This also clarifies why and why , as composition of operators (multiplication of matrices) and trace can be interpreted as the same pairing. Viewing , one may interpret the composition map as
coming from the pairing on the middle terms. Taking the trace of the product then comes from pairing on the outer terms, while taking the product in the opposite order and then taking the trace just switches which pairing is applied first. On the other hand, taking the trace of and the trace of corresponds to applying the pairing on the left terms and on the right terms (rather than on inner and outer), and is thus different.

In coordinates, this corresponds to indexes: multiplication is given by , so and which is the same, while , which is different.

For finite-dimensional, with basis and dual basis , then is the entry of the matrix of the operator with respect to that basis. Any operator is therefore a sum of the form . With defined as above, . The latter, however, is just the Kronecker delta, being 1 if i=j and 0 otherwise. This shows that is simply the sum of the coefficients along the diagonal. This method, however, makes coordinate invariance an immediate consequence of the definition.

Dual

Further, one may dualize this map, obtaining a map . This map is precisely the inclusion of scalars, sending to the identity matrix: "trace is dual to scalars". In the language of bialgebra

Bialgebra

In mathematics, a bialgebra over a field K is a vector space over K which is both a unital associative algebra and a coalgebra, such that these structures are compatible....

s, scalars are the unit, while trace is the counit.

One can then compose these, , which yields multiplication by , as the trace of the identity is the dimension of the vector space.

See also

• Trace class
  Trace class
  In mathematics, a trace class operator is a compact operator for which a trace may be defined, such that the trace is finite and independent of the choice of basis....
  
  
• Field trace
  Field trace
  In mathematics, the field trace is a function defined with respect to a finite field extension L/K. It is a K-linear map from L to K...
  
  
• Golden–Thompson inequality
• Characteristic function
• Specht's theorem
  Specht's theorem
  In mathematics, Specht's theorem gives a necessary and sufficient condition for two matrices to be unitarily equivalent. It is named after Wilhelm Specht, who proved the theorem in 1940....
  
  
• von Neumann's trace inequality