Matrix norm

# Matrix norm

Discussion

Encyclopedia
In mathematics
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...

, a matrix norm is a natural extension of the notion of a vector norm to matrices
Matrix (mathematics)
In mathematics, a matrix is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements isMatrices of the same size can be added or subtracted element by element...

.

## Definition

In what follows, will denote the field
Field (mathematics)
In abstract algebra, a field is a commutative ring whose nonzero elements form a group under multiplication. As such it is an algebraic structure with notions of addition, subtraction, multiplication, and division, satisfying certain axioms...

of real
Real number
In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

or complex number
Complex number
A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...

s. Let denote the vector space
Vector space
A vector space is a mathematical structure formed by a collection of vectors: objects that may be added together and multiplied by numbers, called scalars in this context. Scalars are often taken to be real numbers, but one may also consider vector spaces with scalar multiplication by complex...

containing all matrices with rows and columns with entries in . Throughout the article denotes the conjugate transpose
Conjugate transpose
In mathematics, the conjugate transpose, Hermitian transpose, Hermitian conjugate, or adjoint matrix of an m-by-n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking the transpose and then taking the complex conjugate of each entry...

of matrix .

A matrix norm is a vector norm on . That is, if denotes the norm of the matrix , then,
• if and iff
IFF
IFF, Iff or iff may refer to:Technology/Science:* Identification friend or foe, an electronic radio-based identification system using transponders...

• for all in and all matrices in
• for all matrices and in

Additionally, in the case of square matrices (thus, m = n), some (but not all) matrix norms satisfy the following condition, which is related to the fact that matrices are more than just vectors:
• for all matrices and in

A matrix norm that satisfies this additional property is called a sub-multiplicative norm ( in some books, the terminology matrix norm is used only for those norms which are sub-multiplicative). The set of all n-by-n matrices, together with such a sub-multiplicative norm, is an example of a Banach algebra
Banach algebra
In mathematics, especially functional analysis, a Banach algebra, named after Stefan Banach, is an associative algebra A over the real or complex numbers which at the same time is also a Banach space...

.

## Induced norm

If vector norms on Km and Kn are given (K is field of real
Real number
In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

or complex number
Complex number
A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...

s), then one defines the corresponding induced norm or operator norm
Operator norm
In mathematics, the operator norm is a means to measure the "size" of certain linear operators. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces.- Introduction and definition :...

on the space of m-by-n matrices as the following maxima:
These are different from the entrywise p-norms and the Schatten p-norms for matrices treated below, which are also usually denoted
by

If m = n and one uses the same norm on the domain and the range, then the induced operator norm is a sub-multiplicative matrix norm.

The operator norm corresponding to the p-norm for vectors is:

In the case of and , the norms can be computed as:
which is simply the maximum absolute column sum of the matrix.
which is simply the maximum absolute row sum of the matrix

For example, if the matrix A is defined by
then we have ||A||1 = max(5,13,19) = 19. and ||A|| = max(15,12,10) = 15. Consider another example
where we add all the entries in each column and determine the greatest value, which results in ||A||1 = max (6,13,11,8) = 13.

We can do the same for the rows and get ||A|| = max(9,11,9,9) = 11. Thus 11 is our max.

In the special case of p = 2 (the Euclidean norm) and m = n (square matrices), the induced matrix norm is the spectral norm. The spectral norm of a matrix A is the largest singular value of A i.e. the square root of the largest eigenvalue of the positive-semidefinite matrix A*A:
where A* denotes the conjugate transpose
Conjugate transpose
In mathematics, the conjugate transpose, Hermitian transpose, Hermitian conjugate, or adjoint matrix of an m-by-n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking the transpose and then taking the complex conjugate of each entry...

of A.

More generally, one can define the subordinate matrix norm on induced by
on , and on as:

Subordinate norms are consistent with the norms that induce them, giving

Any induced norm satisfies the inequality
where ρ(A) is the spectral radius
In mathematics, the spectral radius of a square matrix or a bounded linear operator is the supremum among the absolute values of the elements in its spectrum, which is sometimes denoted by ρ.-Matrices:...

of A. For a symmetric or hermitian matrix , we have equality for the 2-norm, since in this case the 2-norm is the spectral radius of . For an arbitrary matrix, we may not have equality for any . Take
the spectral radius of is 0, but is not the zero matrix, and so none of the induced norms are equal to the spectral radius of .

Furthermore,
for square matrices
we have the spectral radius formula:

## "Entrywise" norms

These vector norms treat an matrix as a vector of size , and
use one of the familiar vector norms.

For example, using the p-norm for vectors, we get:

This is a different norm from the induced p-norm (see above) and the Schatten p-norm (see below), but the notation is the same.

The special case p = 2 is the Frobenius norm, and p = ∞ yields the maximum norm.

### Frobenius norm

For p = 2, this is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is often reserved for operators on Hilbert space
Hilbert space
The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...

. This norm can be defined in various ways:

where A* denotes the conjugate transpose
Conjugate transpose
In mathematics, the conjugate transpose, Hermitian transpose, Hermitian conjugate, or adjoint matrix of an m-by-n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking the transpose and then taking the complex conjugate of each entry...

of A, σi are the singular values of A, and the trace function is used. The Frobenius norm is very similar to the Euclidean norm on Kn and comes from an inner product on the space of all matrices.

The Frobenius norm is sub-multiplicative and is very useful for numerical linear algebra
Numerical linear algebra
Numerical linear algebra is the study of algorithms for performing linear algebra computations, most notably matrix operations, on computers. It is often a fundamental part of engineering and computational science problems, such as image and signal processing, Telecommunication, computational...

. This norm is often easier to compute than induced norms.

### Max norm

The max norm is the elementwise norm with p = ∞:
This norm is not sub-multiplicative.

## Schatten norms

The Schatten p-norms arise when applying the p-norm to the vector of singular values
Singular value decomposition
In linear algebra, the singular value decomposition is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics....

of a matrix. If the singular values are denoted by σi, then the Schatten p-norm is defined by
These norms again share the notation with the induced and entrywise p-norms, but they are different.

All Schatten norms are sub-multiplicative. They are also unitarily invariant, which means that ||A|| = ||UAV|| for all matrices A and all unitary matrices U and V.

The most familiar cases are p = 1, 2, ∞. The case p = 2 yields the Frobenius norm, introduced before. The case p = ∞ yields the spectral norm, which is the matrix norm induced by the vector 2-norm (see above). Finally, p = 1 yields the nuclear norm (also known as the trace norm, or the Ky Fan 'n'-norm), defined as

(Here denotes a matrix such that . More precisely, since is a positive semidefinite matrix, its square root
Square root of a matrix
In mathematics, the square root of a matrix extends the notion of square root from numbers to matrices. A matrix B is said to be a square root of A if the matrix product B · B is equal to A.-Properties:...

is well-defined.)

## Consistent norms

A matrix norm on is called consistent with a vector norm on and a vector norm on if:
for all . All induced norms are consistent by definition.

## Compatible norms

A matrix norm on is called compatible with a vector norm on if:
for all . Induced norms are compatible by definition.

## Equivalence of norms

For any two vector norms ||·||α and ||·||β, we have

for some positive numbers r and s, for all matrices A in . In other words, they are equivalent norms; they induce the same topology
Topology
Topology is a major area of mathematics concerned with properties that are preserved under continuous deformations of objects, such as deformations that involve stretching, but no tearing or gluing...

on . This is a special case of the equivalence of norms in finite-dimensional Normed vector spaces.

Moreover, for every vector norm on , there exists a unique positive real number such that is a sub-multiplicative matrix norm for every .

A matrix norm ||·||α is said to be minimal if there exists no other matrix norm ||·||β satisfying ||·||β ≤ ||·||α.

### Examples of norm equivalence

For matrix the following inequalities hold:
• , where is the rank
Rank (linear algebra)
The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...

of
• , where is the rank
Rank (linear algebra)
The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...

of

Here, ||·||p refers to the matrix norm induced by the vector p-norm.

Another useful inequality between matrix norms is