Polynomial interpolation

# Polynomial interpolation

Overview
In numerical analysis
Numerical analysis
Numerical analysis is the study of algorithms that use numerical approximation for the problems of mathematical analysis ....

, polynomial interpolation is the interpolation
Interpolation
In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....

of a given data set
Data set
A data set is a collection of data, usually presented in tabular form. Each column represents a particular variable. Each row corresponds to a given member of the data set in question. Its values for each of the variables, such as height and weight of an object or values of random numbers. Each...

by a polynomial
Polynomial
In mathematics, a polynomial is an expression of finite length constructed from variables and constants, using only the operations of addition, subtraction, multiplication, and non-negative integer exponents...

: given some points, find a polynomial which goes exactly through these points.
Discussion

Encyclopedia
In numerical analysis
Numerical analysis
Numerical analysis is the study of algorithms that use numerical approximation for the problems of mathematical analysis ....

, polynomial interpolation is the interpolation
Interpolation
In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....

of a given data set
Data set
A data set is a collection of data, usually presented in tabular form. Each column represents a particular variable. Each row corresponds to a given member of the data set in question. Its values for each of the variables, such as height and weight of an object or values of random numbers. Each...

by a polynomial
Polynomial
In mathematics, a polynomial is an expression of finite length constructed from variables and constants, using only the operations of addition, subtraction, multiplication, and non-negative integer exponents...

: given some points, find a polynomial which goes exactly through these points.

## Applications

Polynomials can be used to approximate more complicated curves, for example, the shapes of letters in typography
Typography
Typography is the art and technique of arranging type in order to make language visible. The arrangement of type involves the selection of typefaces, point size, line length, leading , adjusting the spaces between groups of letters and adjusting the space between pairs of letters...

, given a few points. A relevant application is the evaluation of the natural logarithm
Natural logarithm
The natural logarithm is the logarithm to the base e, where e is an irrational and transcendental constant approximately equal to 2.718281828...

and trigonometric function
Trigonometric function
In mathematics, the trigonometric functions are functions of an angle. They are used to relate the angles of a triangle to the lengths of the sides of a triangle...

s: pick a few known data points, create a lookup table
Lookup table
In computer science, a lookup table is a data structure, usually an array or associative array, often used to replace a runtime computation with a simpler array indexing operation. The savings in terms of processing time can be significant, since retrieving a value from memory is often faster than...

, and interpolate between those data points. This results in significantly faster computations. Polynomial interpolation also forms the basis for algorithms in numerical quadrature and numerical ordinary differential equations
Numerical ordinary differential equations
Numerical ordinary differential equations is the part of numerical analysis which studies the numerical solution of ordinary differential equations...

.

Polynomial interpolation is also essential to perform sub-quadratic multiplication and squaring such as Karatsuba multiplication and Toom–Cook multiplication, where an interpolation through points on a polynomial which defines the product yields the product itself. For example, given a = f(x) = a0x0 + a1x1 + ... and b = g(x) = b0x0 + b1x1 + ... then the product ab is equivalent to W(x) = f(x)g(x). Finding points along W(x) by substituting x for small values in f(x) and g(x) yields points on the curve. Interpolation based on those points will yield the terms of W(x) and subsequently the product ab. In the case of Karatsuba multiplication this technique is substantially faster than quadratic multiplication, even for modest-sized inputs. This is especially true when implemented in parallel hardware.

## Definition

Given a set of n + 1 data points (xi,yi) where no two xi are the same, one is looking for a polynomial p of degree at most n with the property

The unisolvence
Unisolvent functions
In mathematics, a collection of n functions ƒ1, ƒ2, ..., ƒn is unisolvent on domain Ω if the vectorsare linearly independent for any choice of n distinct points x1, x2 ... xn in Ω...

theorem states that such a polynomial p exists and is unique, and can be proved by the Vandermonde matrix, as described below.

The theorem states that for n+1 interpolation nodes (xi), polynomial interpolation defines a linear bijection
Bijection
A bijection is a function giving an exact pairing of the elements of two sets. A bijection from the set X to the set Y has an inverse function from Y to X. If X and Y are finite sets, then the existence of a bijection means they have the same number of elements...

where is the vector space
Vector space
A vector space is a mathematical structure formed by a collection of vectors: objects that may be added together and multiplied by numbers, called scalars in this context. Scalars are often taken to be real numbers, but one may also consider vector spaces with scalar multiplication by complex...

of polynomials (defined on any interval containing the nodes) of degree at most n.

## Constructing the interpolation polynomial

Suppose that the interpolation polynomial is in the form
The statement that p interpolates the data points means that
If we substitute equation (1) in here, we get a system of linear equations in the coefficients . The system in matrix-vector form reads
We have to solve this system for to construct the interpolant The matrix on the left is commonly referred to as a Vandermonde matrix.

The condition number
Condition number
In the field of numerical analysis, the condition number of a function with respect to an argument measures the asymptotically worst case of how much the function can change in proportion to small changes in the argument...

of the Vandermonde matrix may be large, causing large errors when computing the coefficients if the system of equations is solved using Gaussian elimination
Gaussian elimination
In linear algebra, Gaussian elimination is an algorithm for solving systems of linear equations. It can also be used to find the rank of a matrix, to calculate the determinant of a matrix, and to calculate the inverse of an invertible square matrix...

.

Several authors have therefore proposed algorithms which exploit the structure of the Vandermonde matrix to compute numerically stable solutions in operations instead of the required by Gaussian elimination. These methods rely on constructing first a Newton interpolation
Newton polynomial
In the mathematical field of numerical analysis, a Newton polynomial, named after its inventor Isaac Newton, is the interpolation polynomial for a given set of data points in the Newton form...

of the polynomial and then converting it to the monomial form above.

### Proof 1

Suppose we interpolate through n + 1 data points with an at-most n degree polynomial p(x) (we need at least n + 1 datapoints or else the polynomial cannot be fully solved for). Suppose also another polynomial exists also of degree at most n that also interpolates the n + 1 points; call it q(x).

Consider . We know,
1. r(x) is a polynomial
2. r(x) has degree at most n, since and are no higher than this and we are just subtracting them.
3. At the n + 1 data points, . Therefore r(x) has n + 1 roots.

But r(x) is an n degree polynomial (or less)! It has one root too many.
Formally, if is any non-zero polynomial, it must be writable as .
By distributivity the n + 1 xs multiply together to make , i.e. one degree higher than the maximum we set.
So the only way r(x) can exist is if r(x) = 0.

So (which could be any polynomial, so long as it interpolates the points) is identical with , and is unique.

### Proof 2

Given the Vandermonde matrix used above to construct the interpolant, we can set up the system

We want to prove that V is nonsingular. Given

since the n + 1 points are distinct, the determinant
Determinant
In linear algebra, the determinant is a value associated with a square matrix. It can be computed from the entries of the matrix by a specific arithmetic expression, while other ways to determine its value exist as well...

can't be zero as is never zero, therefore V is nonsingular and the system has a unique solution.

Either way this means that no matter what method we use to do our interpolation: direct, spline
Spline (mathematics)
In mathematics, a spline is a sufficiently smooth piecewise-polynomial function. In interpolating problems, spline interpolation is often preferred to polynomial interpolation because it yields similar results, even when using low-degree polynomials, while avoiding Runge's phenomenon for higher...

, lagrange
Lagrange polynomial
In numerical analysis, Lagrange polynomials are used for polynomial interpolation. For a given set of distinct points x_j and numbers y_j, the Lagrange polynomial is the polynomial of the least degree that at each point x_j assumes the corresponding value y_j...

etc., (assuming we can do all our calculations perfectly) we will always get the same polynomial.

## Non-Vandermonde solutions

We are trying to construct our unique interpolation polynomial in the vector space of polynomials of degree n. When using a monomial basis
Monomial basis
In mathematics a monomial basis is a way to describe uniquely a polynomial using a linear combination of monomials. This description, the monomial form of a polynomial, is often used because of the simple structure of the monomial basis....

for we have to solve the Vandermonde matrix to construct the coefficients for the interpolation polynomial. This can be a very costly operation (as counted in clock cycles of a computer trying to do the job). By choosing another basis for we can simplify the calculation of the coefficients but then we have to do additional calculations when we want to express the interpolation polynomial in terms of a monomial basis
Monomial basis
In mathematics a monomial basis is a way to describe uniquely a polynomial using a linear combination of monomials. This description, the monomial form of a polynomial, is often used because of the simple structure of the monomial basis....

.

One method is to write the interpolation polynomial in the Newton form and use the method of divided differences
Divided differences
In mathematics divided differences is a recursive division process.The method can be used to calculate the coefficients in the interpolation polynomial in the Newton form.-Definition:Given n data points,\ldots,...

to construct the coefficients, e.g. Neville's algorithm
Neville's algorithm
In mathematics, Neville's algorithm is an algorithm used for polynomial interpolation that was derived by the mathematician Eric Harold Neville. Given n + 1 points, there is a unique polynomial of degree ≤ n which goes through the given points...

. The cost is O
Big O notation
In mathematics, big O notation is used to describe the limiting behavior of a function when the argument tends towards a particular value or infinity, usually in terms of simpler functions. It is a member of a larger family of notations that is called Landau notation, Bachmann-Landau notation, or...

operations, while Gaussian elimination costs O operations. Furthermore, you only need to do O extra work if an extra point is added to the data set, while for the other methods, you have to redo the whole computation.

Another method is to use the Lagrange form of the interpolation polynomial. The resulting formula immediately shows that the interpolation polynomial exists under the conditions stated in the above theorem. Lagrange formula is to be preferred to Vandermorde formula when we are not interested in computing the coefficients of the polynomial, but in computing the value of in a given x not in the original data set. In this case, we can reduce complexity to O.

The Bernstein form was used in a constructive proof of the Weierstrass approximation theorem by Bernstein
Sergei Natanovich Bernstein
Sergei Natanovich Bernstein was a Russian and Soviet mathematician known for contributions to partial differential equations, differential geometry, probability theory, and approximation theory.-Partial differential equations:...

and has nowadays gained great importance in computer graphics in the form of Bézier curve
Bézier curve
A Bézier curve is a parametric curve frequently used in computer graphics and related fields. Generalizations of Bézier curves to higher dimensions are called Bézier surfaces, of which the Bézier triangle is a special case....

s.

## Interpolation error

When interpolating a given function f by a polynomial of degree n at the nodes x0,...,xn we get the error

where

is the notation for divided differences
Divided differences
In mathematics divided differences is a recursive division process.The method can be used to calculate the coefficients in the interpolation polynomial in the Newton form.-Definition:Given n data points,\ldots,...

. When f is n + 1 times continuously differentiable on the smallest interval I which contains the nodes xi and x then we can write the error in the Lagrange form as

for some in I. Thus the remainder term in the Lagrange form of the Taylor theorem
Taylor's theorem
In calculus, Taylor's theorem gives an approximation of a k times differentiable function around a given point by a k-th order Taylor-polynomial. For analytic functions the Taylor polynomials at a given point are finite order truncations of its Taylor's series, which completely determines the...

is a special case of interpolation error when all interpolation nodes xi are identical.

In the case of equally spaced interpolation nodes , it follows that the interpolation error is O. However, this does not yield any information on what happens when . That question is treated in the section Convergence properties.

The above error bound suggests choosing the interpolation points xi such that the product | Π (x − xi) | is as small as possible. The Chebyshev nodes
Chebyshev nodes
In numerical analysis, Chebyshev nodes are the roots of the Chebyshev polynomial of the first kind. They are often used as nodes in polynomial interpolation because the resulting interpolation polynomial minimizes the Runge's phenomenon.-Definition:...

achieve this.

## Lebesgue constants

See the main article: Lebesgue constant
Lebesgue constant (interpolation)
In mathematics, the Lebesgue constants give an idea of how good the interpolant of a function is in comparison with the best polynomial approximation of the function...

.

We fix the interpolation nodes x0, ..., xn and an interval [a, b] containing all the interpolation nodes. The process of interpolation maps the function f to a polynomial p. This defines a mapping X from the space C([a, b]) of all continuous functions on [a, b] to itself. The map X is linear and it is a projection
Projection (linear algebra)
In linear algebra and functional analysis, a projection is a linear transformation P from a vector space to itself such that P2 = P. It leaves its image unchanged....

on the subspace Πn of polynomials of degree n or less.

The Lebesgue constant L is defined as the operator norm
Operator norm
In mathematics, the operator norm is a means to measure the "size" of certain linear operators. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces.- Introduction and definition :...

of X. One has (a special case of Lebesgue's lemma
Lebesgue's lemma
For Lebesgue's lemma for open covers of compact spaces in topology see Lebesgue's number lemmaIn mathematics, Lebesgue's lemma is an important statement in approximation theory. It provides a bound for the projection error.-Statement:...

):

In other words, the interpolation polynomial is at most a factor (L + 1) worse than the best possible approximation. This suggests that we look for a set of interpolation nodes that L small. In particular, we have for Chebyshev nodes
Chebyshev nodes
In numerical analysis, Chebyshev nodes are the roots of the Chebyshev polynomial of the first kind. They are often used as nodes in polynomial interpolation because the resulting interpolation polynomial minimizes the Runge's phenomenon.-Definition:...

:

We conclude again that Chebyshev nodes are a very good choice for polynomial interpolation, as the growth in n is exponential for equidistant nodes. However, those nodes are not optimal.

## Convergence properties

It is natural to ask, for which classes of functions and for which interpolation nodes the sequence of interpolating polynomials converges to the interpolated function as the degree n goes to infinity? Convergence may be understood in different ways, e.g. pointwise, uniform or in some integral norm.

The situation is rather bad for equidistant nodes, in that uniform convergence is not even guaranteed for infinitely differentiable functions. One classical example, due to Carl Runge
Runge's phenomenon
In the mathematical field of numerical analysis, Runge's phenomenon is a problem of oscillation at the edges of an interval that occurs when using polynomial interpolation with polynomials of high degree...

, is the function f(x) = 1 / (1 + x2) on the interval [−5, 5]. The interpolation error ||f − pn|| grows without bound as . Another example is the function f(x) = |x| on the interval [−1, 1], for which the interpolating polynomials do not even converge pointwise except at the three points x = −1, 0, and 1.

One might think that better convergence properties may be obtained by choosing different interpolation nodes. The following theorem seems to be a rather encouraging answer:
For any function f(x) continuous on an interval [a,b] there exists a table of nodes for which the sequence of interpolating polynomials converges to f(x) uniformly on [a,b].

Proof. It's clear that the sequence of polynomials of best approximation converges to f(x) uniformly (due to Weierstrass approximation theorem). Now we have only to show that each may be obtained by means of interpolation on certain nodes. But this is true due to a special property of polynomials of best approximation known from the Chebyshev alternation theorem. Specifically, we know that such polynomials should intersect f(x) at least n+1 times. Choosing the points of intersection as interpolation nodes we obtain the interpolating polynomial coinciding with the best approximation polynomial.

The defect of this method, however, is that interpolation nodes should be calculated anew for each new function f(x), but the algorithm is hard to be implemented numerically. Does there exist a single table of nodes for which the sequence of interpolating polynomials converge to any continuous function f(x)? The answer is unfortunately negative as it is stated by the following theorem:
For any table of nodes there is a continuous function f(x) on an interval [a,b] for which the sequence of interpolating polynomials diverges on [a,b].

The proof essentially uses the lower bound estimation of the Lebesgue constant, which we defined above to be the operator norm of Xn (where Xn is the projection operator on Πn). Now we seek a table of nodes for which

Due to the Banach–Steinhaus theorem, this is only possible when norms of Xn are uniformly bounded, which cannot be true since we know that

For example, if equidistant points are chosen as interpolation nodes, the function from Runge's phenomenon
Runge's phenomenon
In the mathematical field of numerical analysis, Runge's phenomenon is a problem of oscillation at the edges of an interval that occurs when using polynomial interpolation with polynomials of high degree...

demonstrates divergence of such interpolation. Note that this function is not only continuous but even infinitely times differentiable on [−1, 1]. For better Chebyshev nodes
Chebyshev nodes
In numerical analysis, Chebyshev nodes are the roots of the Chebyshev polynomial of the first kind. They are often used as nodes in polynomial interpolation because the resulting interpolation polynomial minimizes the Runge's phenomenon.-Definition:...

, however, such an example is much harder to find because of the theorem:
For every absolutely continuous
Absolute continuity
In mathematics, the relationship between the two central operations of calculus, differentiation and integration, stated by fundamental theorem of calculus in the framework of Riemann integration, is generalized in several directions, using Lebesgue integration and absolute continuity...

function on [−1, 1] the sequence of interpolating polynomials constructed on Chebyshev nodes converges to f(x) uniformly.

## Related concepts

Runge's phenomenon
Runge's phenomenon
In the mathematical field of numerical analysis, Runge's phenomenon is a problem of oscillation at the edges of an interval that occurs when using polynomial interpolation with polynomials of high degree...

shows that for high values of n, the interpolation polynomial may oscillate wildly between the data points. This problem is commonly resolved by the use of spline interpolation
Spline interpolation
In the mathematical field of numerical analysis, spline interpolation is a form of interpolation where the interpolant is a special type of piecewise polynomial called a spline. Spline interpolation is preferred over polynomial interpolation because the interpolation error can be made small even...

. Here, the interpolant is not a polynomial but a spline
Spline (mathematics)
In mathematics, a spline is a sufficiently smooth piecewise-polynomial function. In interpolating problems, spline interpolation is often preferred to polynomial interpolation because it yields similar results, even when using low-degree polynomials, while avoiding Runge's phenomenon for higher...

: a chain of several polynomials of a lower degree.

Interpolation of periodic function
Periodic function
In mathematics, a periodic function is a function that repeats its values in regular intervals or periods. The most important examples are the trigonometric functions, which repeat over intervals of length 2π radians. Periodic functions are used throughout science to describe oscillations,...

s by harmonic
Harmonic analysis
Harmonic analysis is the branch of mathematics that studies the representation of functions or signals as the superposition of basic waves. It investigates and generalizes the notions of Fourier series and Fourier transforms...

functions is accomplished by Fourier transform
Fourier transform
In mathematics, Fourier analysis is a subject area which grew from the study of Fourier series. The subject began with the study of the way general functions may be represented by sums of simpler trigonometric functions...

. This can be seen as a form of polynomial interpolation with harmonic base functions, see trigonometric interpolation
Trigonometric interpolation
In mathematics, trigonometric interpolation is interpolation with trigonometric polynomials. Interpolation is the process of finding a function which goes through some given data points. For trigonometric interpolation, this function has to be a trigonometric polynomial, that is, a sum of sines and...

and trigonometric polynomial
Trigonometric polynomial
In the mathematical subfields of numerical analysis and mathematical analysis, a trigonometric polynomial is a finite linear combination of functions sin and cos with n a natural number. The coefficients may be taken as real numbers, for real-valued functions...

.

Hermite interpolation
Hermite interpolation
In numerical analysis, Hermite interpolation, named after Charles Hermite, is a method of interpolating data points as a polynomial function. The generated Hermite polynomial is closely related to the Newton polynomial, in that both are derived from the calculation of divided differences.Unlike...

problems are those where not only the values of the polynomial p at the nodes are given, but also all derivatives up to a given order. This turns out to be equivalent to a system of simultaneous polynomial congruences, and may be solved by means of the Chinese remainder theorem
Chinese remainder theorem
The Chinese remainder theorem is a result about congruences in number theory and its generalizations in abstract algebra.In its most basic form it concerned with determining n, given the remainders generated by division of n by several numbers...

for polynomials. Birkhoff interpolation is a further generalization where only derivatives of some orders are prescribed, not necessarily all orders from 0 to a k.

Collocation method
Collocation method
In mathematics, a collocation method is a method for the numerical solution of ordinary differential equations, partial differential equations and integral equations...

s for the solution of differential and integral equations are based on polynomial interpolation.

The technique of rational function modeling is a generalization that considers ratios of polynomial functions.

At last, multivariate interpolation
Multivariate interpolation
In numerical analysis, multivariate interpolation or spatial interpolation is interpolation on functions of more than one variable.The function to be interpolated is known at given points and the interpolation problem consist of yielding values at arbitrary points .-Regular grid:For function...

for higher dimensions.