Convergence Concepts

Implications of convergence

1. Almost Sure Convergence

Examples of statements that hold almost surely (a.s.)

• Let $X,X'$ be two random variables.  Then $X=X'$ a.s. means $$P[X=X']=1;$$ that is, there exists an event $N\in \mathcal{B}$, such that $P(N)=0$ and if $\omega\in N^c$, then $X(\omega)=X'(\omega)$.
• If $\{X_n\}$ is a sequence of random variables, then $\lim_{n\rightarrow \infty}X_n$ exists a.s. means there exists an event $N\in \mathcal{B}$, such that $P(N)=0$ and if $\omega\in N^c$ then $$\lim_{n\rightarrow \infty}X_n(w)$$ exists. It also means that for a.a. $\omega$, $$\underset{n\rightarrow \infty}{\text{lim sup}}X_n(\omega)=\underset{n\rightarrow \infty}{\text{lim inf}}X_n(\omega).$$ We will write $\lim_{n\rightarrow \infty}X_n = X$ or $X_n \overset{a.s.}{\rightarrow}X$.
• If $\{X_n\}$ is a sequence of random variables, then $\sum_n X_n$ converges a.s. means there exists an event $N\in \mathcal{B}$, such that $P(N)=0$, and $\omega \in N^c$ implies $\sum_n X_n(w)$ converges.

2. Convergence in Probability

Suppose $X_n, n\ge 1$ and $X$ are random variables.  Then ${X_n}$ converges in probability (i.p.) to $X$, written $X_n \overset{P}{\rightarrow}X$, if for any $\epsilon > 0$ $$\lim_{n\rightarrow \infty} P[|X_n-X|>\epsilon]=0.$$
Almost sure convergence of $\{X_n\}$ demands that for a.e. $\omega$, $X_n(w)-X(w)$ gets small and stay small.  Convergence i.p. is weaker and merely requires that the probability of the difference $X_n(w)-X(w)$ being non-trivial become small.

It is possible for a sequence to converge in probability but not almost surely.

Theorem 1. Convergence a.s. implies convergence i.p. Suppose that $X_n, n\ge 1$ and $X$ are random variables on a probability space $(\Omega,\mathcal{B},P)$.  If $$X_n \rightarrow X, \; a.s.$$ then $$X_n \overset{P}{\rightarrow}X.$$
Proof.  If $X_n \rightarrow X$ a.s. then for any $\epsilon$,
$$\begin{aligned} 0\;&=P([|X_n-X|>\epsilon]i.o.) \\   &=P(\underset{n\rightarrow \infty}{\text{lim sup}}[|X_n-X|>\epsilon]) \\   &=\lim_{N\rightarrow \infty}P(\bigcup_{n\ge N}[|X_n-X|>\epsilon] ) \\   &\ge \lim_{n\rightarrow \infty}P[|X_n-X|>\epsilon] \end{aligned}$$

3. $L_p$ Convergence

Recall the notation $X\in L_p$ which means $E(|X|^p)<\infty$. For random variables $X,Y\in L_p$, we define the $L_p$ metric for $p\ge 1$ by
$$d(X,Y)=(E|X-Y|^p)^{1/p}.$$  This metric is norm induced because
$$\|X\|_p := (E|X|^p)^{1/p}$$ is a norm on the space $L_p$.

A sequence $\{X_n\}$ of random variables converges in $L_p$ to $X$, written
$$X_n \overset{L_p}{\rightarrow}X ,$$ if
$$E(|X_n-X|^p) \rightarrow 0$$ as $n\rightarrow \infty$.

Facts about $L_p$ convergence.

1. $L_p$ convergence implies convergence in probability: For $p>0$, if $X_n\overset{L_p}{\rightarrow} X$ then $X_n \overset{P}{\rightarrow}X$.  This follows readily from Chebychev's inequality, $$P[|X_n-X|\ge \epsilon] \leq  \frac{E(|X_n-X|^p|)}{\epsilon^p} \rightarrow 0.$$
2. Convergence in probability does not imply $L_p$ convergence.  What can go wrong is that the $n$th function in the sequence can be huge on a very small set.
   Example.  Let the probability space be $([0,1],\mathcal{B}([0,1]),\lambda)$, where $\lambda$ is Lebesgue measure and define
   $$X_n = 2^n 1_{(0,\frac{1}{n}) }$$ then
   $$P[|X_n| > \epsilon ] = P \left( (0,\frac{1}{n}) \right)  = \frac{1}{n} \rightarrow 0$$ but
   $$E(|X_n|^p) = 2^{np} \frac{1}{n} \rightarrow \infty$$
3. $L_p$ convergence does not imply almost sure convergence.
   Example.  Consider the functions $\{X_n\}$ defined on $([0,1],\mathcal{B}([0,1]),\lambda)$, where $\lambda$ is Lebesgue measure.
   $$\begin{align*}     X_1 &= 1_{[0,\frac{1}{2}]},  \quad  X_2 = 1_{[\frac{1}{2},1]} \\     X_3 &= 1_{[0,\frac{1}{3}]},  \quad  X_4 = 1_{[\frac{1}{3},\frac{2}{3}]} \\     X_5 &= 1_{[\frac{1}{3},1]},  \quad  X_6 = 1_{[0,\frac{1}{4}]}, \cdots \\ \end{align*}$$  and so on, Note that for any $p>0$,
   $$E(|X_1|^p)=E(|X_2|^p)=\frac{1}{2},\\     E(|X_3|^p)=E(|X_4|^p)=E(|X_5|^p)=\frac{1}{3}, \\     E(|X_6|^p)=\frac{1}{4}, \cdots$$ so  $E(|X_n|^p) \rightarrow 0$ and
   $$X_n \xrightarrow[]{L_p} 0.$$
   Observe that $\{X_n\}$ does not converge almost surely to 0.