In linear algebra

Linear algebra

Linear algebra is a branch of mathematics that studies vector spaces, also called linear spaces, along with linear functions that input one vector and output another. Such functions are called linear maps and can be represented by matrices if a basis is given. Thus matrix theory is often...

, the column space of a matrix

Matrix (mathematics)

In mathematics, a matrix is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements isMatrices of the same size can be added or subtracted element by element...

 (sometimes called the range of a matrix) is the set of all possible linear combination

Linear combination

In mathematics, a linear combination is an expression constructed from a set of terms by multiplying each term by a constant and adding the results...

s of its column vectors

Vector (spatial)

In elementary mathematics, physics, and engineering, a Euclidean vector is a geometric object that has both a magnitude and direction...

. The column space of an m × n matrix is a subspace

Euclidean subspace

In linear algebra, a Euclidean subspace is a set of vectors that is closed under addition and scalar multiplication. Geometrically, a subspace is a flat in n-dimensional Euclidean space that passes through the origin...

 of m-dimensional Euclidean space

Euclidean space

In mathematics, Euclidean space is the Euclidean plane and three-dimensional space of Euclidean geometry, as well as the generalizations of these notions to higher dimensions...

. The dimension of the column space is called the rank

Rank (linear algebra)

The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...

 of the matrix.

The column space of a matrix is the image

Image (mathematics)

In mathematics, an image is the subset of a function's codomain which is the output of the function on a subset of its domain. Precisely, evaluating the function at each element of a subset X of the domain produces a set called the image of X under or through the function...

 or range

Range (mathematics)

In mathematics, the range of a function refers to either the codomain or the image of the function, depending upon usage. This ambiguity is illustrated by the function f that maps real numbers to real numbers with f = x^2. Some books say that range of this function is its codomain, the set of all...

 of the corresponding matrix transformation.

Definition

Let A be an m × n matrix, with column vectors v₁, v₂, ..., v_n. A linear combination

Linear combination

In mathematics, a linear combination is an expression constructed from a set of terms by multiplying each term by a constant and adding the results...

 of these vectors is any vector of the form
where c₁, c₂, ..., c_n are scalars. The set of all possible linear combinations of v₁,...,v_n is called the column space of A. That is, the column space of A is the span

Linear span

In the mathematical subfield of linear algebra, the linear span of a set of vectors in a vector space is the intersection of all subspaces containing that set...

 of the vectors v₁,...,v_n.

Example

If , then the column vectors are v₁ = (1, 0, 2)^T and v₂ = (0, 1, 0)^T.

A linear combination of v₁ and v₂ is any vector of the form

The set of all such vectors is the column space of A. In this case, the column space is precisely the set of vectors (x, y, z) ∈ R³ satisfying the equation z = 2x (using Cartesian coordinates, this set is a plane

Plane (mathematics)

In mathematics, a plane is a flat, two-dimensional surface. A plane is the two dimensional analogue of a point , a line and a space...

 through the origin in three-dimensional space

Three-dimensional space

Three-dimensional space is a geometric 3-parameters model of the physical universe in which we live. These three dimensions are commonly called length, width, and depth , although any three directions can be chosen, provided that they do not lie in the same plane.In physics and mathematics, a...

).

Any linear combination of the column vectors of a matrix A can be written as the product of A with a column vector:
Therefore, the column space of A consists of all possible products Ax, for x ∈ Rⁿ. This is the same as the image

Image (mathematics)

In mathematics, an image is the subset of a function's codomain which is the output of the function on a subset of its domain. Precisely, evaluating the function at each element of a subset X of the domain produces a set called the image of X under or through the function...

 (or range

Range (mathematics)

In mathematics, the range of a function refers to either the codomain or the image of the function, depending upon usage. This ambiguity is illustrated by the function f that maps real numbers to real numbers with f = x^2. Some books say that range of this function is its codomain, the set of all...

) of the corresponding matrix transformation.

Basis

The columns of A span the column space, but they may not form a basis

Basis (linear algebra)

In linear algebra, a basis is a set of linearly independent vectors that, in a linear combination, can represent every vector in a given vector space or free module, or, more simply put, which define a "coordinate system"...

 if the column vectors are not linearly independent. Fortunately, elementary row operations

Elementary row operations

In mathematics, an elementary matrix is a simple matrix which differs from the identity matrix by one single elementary row operation. The elementary matrices generate the general linear group of invertible matrices...

 do not affect the dependence relations between the column vectors. This makes it possible to use row reduction to find a basis

Basis (linear algebra)

In linear algebra, a basis is a set of linearly independent vectors that, in a linear combination, can represent every vector in a given vector space or free module, or, more simply put, which define a "coordinate system"...

 for the column space.

For example, consider the matrix
The columns of this matrix span the column space, but they may not be linearly independent, in which case some subset of them will form a basis. To find this basis, we reduce A to reduced row echelon form:
At this point, it is clear that the first, second, and fourth columns are linearly independent, while the third column is a linear combination of the first two. (Specifically, v₃ = –2v₁ + v₂.) Therefore, the first, second, and fourth columns of the original matrix are a basis for the column space:
Note that the independent columns of the reduced row echelon form are precisely the columns with pivots

Pivot element

The pivot or pivot element is the element of a matrix, an array, or some other kind of finite set, which is selected first by an algorithm , to do certain calculations...

. This makes it possible to determine which columns are linearly independent by reducing only to echelon form

Row echelon form

In linear algebra a matrix is in row echelon form if* All nonzero rows are above any rows of all zeroes, and...

.

The above algorithm can be used in general to find the dependence relations between any set of vectors, and to pick out a basis from any spanning set. A different algorithm for finding a basis from a spanning set is given in the row space

Row space

In linear algebra, the row space of a matrix is the set of all possible linear combinations of its row vectors. The row space of an m × n matrix is a subspace of n-dimensional Euclidean space...

 article; finding a basis for the column space of A is equivalent to finding a basis for the row space of the transpose

Transpose

In linear algebra, the transpose of a matrix A is another matrix AT created by any one of the following equivalent actions:...

 matrix A^T.

Dimension

The dimension of the column space is called the rank

Rank (linear algebra)

The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...

 of the matrix. The rank is equal to the number of pivots in the reduced row echelon form, and is the maximum number of linearly independent columns that can be chosen from the matrix. For example, the 4 × 4 matrix in the example above has rank three.

Because the column space is the image

Image (mathematics)

In mathematics, an image is the subset of a function's codomain which is the output of the function on a subset of its domain. Precisely, evaluating the function at each element of a subset X of the domain produces a set called the image of X under or through the function...

 of the corresponding matrix transformation, the rank of a matrix is the same as the dimension of the image. For example, the transformation R⁴ → R⁴ described by the matrix above maps all of R⁴ to some three-dimensional subspace

Euclidean subspace

In linear algebra, a Euclidean subspace is a set of vectors that is closed under addition and scalar multiplication. Geometrically, a subspace is a flat in n-dimensional Euclidean space that passes through the origin...

.

The nullity of a matrix is the dimension of the null space, and is equal to the number of columns in the reduced row echelon form that do not have pivots. The rank and nullity of a matrix A with n columns are related by the equation:
This is known as the rank-nullity theorem

Rank-nullity theorem

In mathematics, the rank–nullity theorem of linear algebra, in its simplest form, states that the rank and the nullity of a matrix add up to the number of columns of the matrix. Specifically, if A is an m-by-n matrix over some field, thenThis applies to linear maps as well...

.

Relation to the left null space

The left null space of A is the set of all vectors x such that x^TA = 0^T. It is the same as the null space of the transpose

Transpose

In linear algebra, the transpose of a matrix A is another matrix AT created by any one of the following equivalent actions:...

 of A. The left null space is the orthogonal complement to the column space of A.

This can be seen by writing the product of the matrix and the vector x in terms of the dot product

Dot product

In mathematics, the dot product or scalar product is an algebraic operation that takes two equal-length sequences of numbers and returns a single number obtained by multiplying corresponding entries and then summing those products...

 of vectors:
where c₁, ..., c_n are the column vectors of A. Thus x = 0 if and only if x is orthogonal (perpendicular) to each of the column vectors of A.

It follows that the null space of is the orthogonal complement to the column space of A.

For a matrix A, the column space, row space, null space, and left null space are sometimes referred to as the four fundamental subspaces.

See also

• Euclidean subspace
  Euclidean subspace
  In linear algebra, a Euclidean subspace is a set of vectors that is closed under addition and scalar multiplication. Geometrically, a subspace is a flat in n-dimensional Euclidean space that passes through the origin...
  
  
• Kernel (matrix)
• Row space
  Row space
  In linear algebra, the row space of a matrix is the set of all possible linear combinations of its row vectors. The row space of an m × n matrix is a subspace of n-dimensional Euclidean space...
  
  
• Four fundamental subspaces
• Rank (linear algebra)
  Rank (linear algebra)
  The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...
  
  
• Linear span
  Linear span
  In the mathematical subfield of linear algebra, the linear span of a set of vectors in a vector space is the intersection of all subspaces containing that set...
  
  
• Matrix (mathematics)
  Matrix (mathematics)
  In mathematics, a matrix is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements isMatrices of the same size can be added or subtracted element by element...
  
  

External links

, MIT Linear Algebra Lecture on the Four Fundamental Subspaces at Google Video, from MIT OpenCourseWare

MIT OpenCourseWare

MIT OpenCourseWare is an initiative of the Massachusetts Institute of Technology to put all of the educational materials from its undergraduate- and graduate-level courses online, partly free and openly available to anyone, anywhere. MIT OpenCourseWare is a large-scale, web-based publication of...