|
|
|
|
Convergence of random variables
|
| |
|
| |
In probability theory, there exist several different notions of convergence of random variables. The convergence (in one of the senses presented below) of sequences of random variables to some limiting random variable is an important concept in probability theory, and its applications to statistics and stochastic processes. For example, if the average of n uncorrelated random variables Yi, i = 1, ..., n, is given by
then as n goes to infinity, Xn converges in probability (see below) to the common mean, μ, of the random variables Yi.
Discussion
Ask a question about 'Convergence of random variables' Start a new discussion about 'Convergence of random variables' Answer questions from other users |
Encyclopedia
In probability theory, there exist several different notions of convergence of random variables. The convergence (in one of the senses presented below) of sequences of random variables to some limiting random variable is an important concept in probability theory, and its applications to statistics and stochastic processes. For example, if the average of n uncorrelated random variables Yi, i = 1, ..., n, is given by
then as n goes to infinity, Xn converges in probability (see below) to the common mean, μ, of the random variables Yi. This result is known as the weak law of large numbers. Other forms of convergence are important in other useful theorems, including the central limit theorem.
Throughout the following, we assume that (Xn) is a sequence of random variables, and X is a random variable, and all of them are defined on the same probability space (Ω, F, P).
Convergence in distribution
Suppose that F1, F2, ... is a sequence of cumulative distribution functions corresponding to random variables X1, X2, ..., and that F is a distribution function corresponding to a random variable X. We say that the sequence Xn converges towards X in distribution, if
-
for every real number a at which F is continuous. Since F(a) = Pr(X ≤ a), this means that the probability that the value of X is in a given range is very similar to the probability that the value of Xn is in that range, provided n is sufficiently large. Convergence in distribution is often denoted by adding the letter over an arrow indicating convergence:
-
Lower-case d is also used, although less frequently.
Convergence in distribution is the weakest form of convergence, and is sometimes called weak convergence (main article: weak convergence of measures). It does not, in general, imply any other mode of convergence. However, convergence in distribution is implied by all other modes of convergence mentioned in this article, and hence, it is the most common and often the most useful form of convergence of random variables. It is the notion of convergence used in the central limit theorem.
A useful result, which may be employed in conjunction with law of large numbers and the central limit theorem, is that if a function g: R → R is continuous, then if Xn converges in distribution to X, then so too does g(Xn) converge in distribution to g(X). (This may be proved using Skorokhod's representation theorem.) This fact could be taken as
a definition for the convergence in distribution.
The Lévy continuity theorem states that converges in distribution to some random variable with the characteristic function if is the pointwise convergent limit of and is continuous at .
Convergence in distribution is also called convergence in law, since the word "law" is sometimes used as a synonym of "probability distribution."
Convergence in probability
To say that the sequence Xn of random variables converges towards X in probability means
for every ε > 0. Formally, pick any ε > 0 and any δ > 0. Let Pn be the probability that Xn is outside a tolerance ε of X. Then, if Xn converges in probability to X then there exists a value N such that, for all n ≥ N, Pn is itself less than δ.
Convergence in probability is often denoted by adding the letter over an arrow indicating convergence:
-
It is also denoted
-
or simply
-
Convergence in probability is the notion of convergence used in the weak law of large numbers. Convergence in probability implies convergence in distribution.
Lemma
Let X, Y be random variables, c a real number and ε > 0; then
Proof of lemma
since
Proof
Recall that in order to prove convergence in distribution, one must show that the sequence of cumulative distribution functions converges to the FX at every point where FX is continuous. Let a be such a point.
For every , due to the preceding lemma, we have:
So, we have
Taking the limit as , we obtain:
But is the cumulative distribution function FX, evaluated at the point a, at which the function is continous by assumption. This means that
and so, taking the limit as , we obtain
In the special case when X = c is a constant (that is, a random variable that attains the deterministic value c with probability one), convergence of Xn to c in probability is equivalent to convergence to c in distribution.
Convergence in probability defines a topology on the space of random variables over a fixed probability space. In fact, this topology is metrizable and a metric d that generates the topology is given by
Almost sure convergence
To say that the sequence Xn converges almost surely or almost everywhere or with probability 1 or strongly towards X means
This means that the values of Xn approach the value of X, in the sense (see almost surely) that events for which Xn does not converge to X have probability 0. Using the probability space and the concept of the random variable as a function from Ω to R, this is equivalent to the statement
Another, equivalent, way of defining almost sure convergence is as follows: Let . Then the almost sure convergence becomes
-
Almost sure convergence is often denoted by adding the letters a.s. over an arrow indicating convergence:
-
Almost sure convergence implies convergence in probability, and hence implies convergence in distribution. It is the notion of convergence used in the strong law of large numbers.
Sure convergence
To say that the sequence or random variables (Xn) defined over the same probability space (i.e., a random process) converges surely or everywhere or pointwise towards X means
where is the sample space of the underlying probability space over which the random variables are defined.
This is the notion of pointwise convergence of sequence functions extended to sequence of random variables. (Note that random variables themselves are functions).
Sure convergence of a random variable implies all the other kinds of convergence stated above, but there is no payoff in probability theory by using sure convergence compared to using almost sure convergence. The difference between the two only exists on sets with probability zero. This is why the concept of sure convergence of random variables is very rarely used.
Convergence in mean
We say that the sequence Xn converges in the r-th mean or in the Lr norm towards X, if r ≥ 1, E|Xn|r < ∞ for all n, and
-
where the operator E denotes the expected value. Convergence in rth mean tells us that the expectation of the r-th power of the difference between Xn and X converges to zero.
This type of convergence is often denoted by adding the letter Lr over an arrow indicating convergence:
-
The most important cases of convergence in r-th mean are:
- When Xn converges in r-th mean to X for r = 1, we say that Xn converges in mean to X.
- When Xn converges in r-th mean to X for r = 2, we say that Xn converges in mean square to X. This is also sometimes referred to as convergence in mean, and is sometimes denoted
Convergence in the r-th mean, for r > 0, implies convergence in probability (by Markov's inequality), while if r > s ≥ 1, convergence in r-th mean implies convergence in s-th mean. Hence, convergence in mean square implies convergence in mean.
Implications
The chain of implications between the various notions of convergence are noted in their
respective sections. They are, using the arrow notation
No other implications other than these hold in general, but a number of special cases do permit the converse implications:
- If Xn converges in distribution to a constant c, then Xn converges in probability to c.
- If Xn converges in probability to X, and if for all n and some b, then Xn converges in rth mean to X for all r ≥ 1. In other words, if Xn converges in probability to X and all random variables Xn are almost surely bounded above and below, then Xn converges to X also in any rth mean.
-
- then we say that Xn converges almost completely, or fast in probability towards X. When Xn converges almost completely towards X then it also converges almost surely to X. In other words, if Xn converges in probability to X sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ε > 0), then Xn also converges almost surely to X. This is a direct implication from the Borel-Cantelli lemma.
- If Sn is a sum of n real independent random variables:
-
- then Sn converges almost surely if and only if Sn converges in probability.
-
See also
External links
|
| |
|
|
