3.9. Infinitely divisible distributions

$\newcommand{\argmin}{\mathop{\mathrm{argmin}}\limits}$ $\newcommand{\argmax}{\mathop{\mathrm{argmax}}\limits}$

A certain kind of well behaving distributions has characteristic functions that can be represented in canonical form. In this section we cover conditions that such distributions have and its canonical representation.


Infinitely divisible distributions

Let $F$ be a distribution with characteristic function $\varphi.$ $F$ is inifinitely divisible (ID for short) if one of the followings hold.
(i) There exists a squence of distributions $(F_n)$ such that $F=F_n*\cdots* F_n$ for all $n\in\mathbb{N}.$
(ii) There exists random variables $X,X_{nk}$ in a probability space $(\Omega, \mathcal{F}, P)$ such that $X\overset{d}{=}X_{n1}+\cdots+X_{nn}$ for all $n,$ where $X\sim F$, $X_{nk}\sim F_n$ for all $k$ and $X_{nk}$'s are rowwise independent.
(iii) There exists a sequence of characteristic functions $(\varphi_n)$ such that $\varphi = (\varphi_n)^n.$

Here $*$ denotes convolution. In fact all three conditions are equivalent. As an example, we can easily check that a normal distribution $X\sim\mathcal{N}(\mu,\sigma^2)$ is infinitely divisible since $X\overset{d}{=}X_{n1}+\cdots+X_{nn}$ for rowwise independent $X_{nk}\sim\mathcal{N}(\frac{\mu}{n},\frac{\sigma^2}{n}).$

First important property is that characteristic functions of ID distributions always have non-zero values. For this, we need a lemma that applies to all characteristic functions.

For a ch.f. $\varphi,$ $$1-|\varphi(2t)|^2 \le 4(1-|\varphi(t)|^2).$$

Proof is simple using elementary trigonometrics. Notice that $|\varphi|^2$ is a real-valued ch.f. so it suffices to show that $\text{Re}(1-\varphi(2t)) \le 4\text{Re}(1-\varphi(t))$ for a given $\varphi.$ $$\begin{aligned} \text{Re}(1-\varphi(t)) &= \int(1-\cos tx)dF(x) \\ &= \int2\sin^2 \frac{t}{2}dF(x) \\ &= \int\frac{\sin^2 tx}{2\cos^2\frac{t}{2}x}dF(x) \\ &= \int\frac{1}{2}\sin^2 tx dF(x) \\ &= \int\frac{1}{4}(1-\cos 2tx) dF(x) \\ &= \frac{1}{4}\text{Re}(1-\varphi(2t)). \end{aligned}$$
$$\text{For an infinitly divisible $\varphi$,}~ \varphi(t)\ne0,~ \forall t.$$

Proof is by induction. Since $\varphi$ is ID, there exists $\varphi_n$ such that $\varphi=(\varphi_n)^n.$ We know that $\varphi\to1$ as $t\to0,$ so there exists $a>0$ such that $|\varphi(t)|>0$ for all $|t|\le a.$
Given $t$ that $|t|\le a,$ $$|\varphi_n(t)| = |\varphi(t)^\frac{1}{n}| \ge \left(\inf_{|t|\le a}|\varphi(t)|\right)^\frac{1}{n} \to_n 1.$$ so for all $0\le\epsilon\le1,$ there exists $N>0$ such that $|\varphi_n(t)| > 1-\epsilon$ for all $n\ge N.$ By the lemma, for $n\ge N$ and $|t|\le a,$ $$1-|\varphi_n(2t)|^2 \le 4(1-(1-\epsilon)^2) \le 8\epsilon.$$ Thus for large n, $|\varphi_n(2t)|^2 \ge 1-8\epsilon > 0,$ for all $0 < \epsilon < 1/8.$ This gives $\varphi(2t) \ne 0$ for all $|t|\le a.$ Repeatedly apply the process to get $\varphi \ne 0$ for all $t.$


Canonical representation

Characteristic functions of Infinitely divisible distributions can be uniquely represented in a certain form. Furthermore, if a characteristic function can be written in such form, then it is infinitly divisible. We call it a canonical form. While there are several equivalent canonical representations, I would like to cover the one by Kolmogorov. The first theorem is about sufficiency of ID distribution.

Sufficiency

Let $\varphi$ be a characteristic function. If $$\varphi(t) = \exp\left\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu(x) \right\},~ \forall t$$ for some finite measure $\mu$ on $(\mathbb{R}, \mathcal{B}(\mathbb{R})),$ then $\varphi$ is infinitely divisible with mean 0 and variance $\mu(\mathbb{R}).$
expand proof

(case 1. $\mu$ has a mass only at 0.)
Let $\sigma^2 = \mu(\mathbb{R}) = \mu\{0\}>0$ then $\varphi(t) = e^\frac{t^2\sigma^2}{2}$ which is the ch.f. of $\mathcal{N}(0,\sigma^2)$ so it is ID.
(case 2. $\mu$ has a mass only at $x\ne0.$)
Let $\mu\{x\}=\lambda x^2$ for some $\lambda>0.$ Then $\varphi(t)=e^{\lambda(e^{itx}-1-itx)}$ which is a ch.f. of $x(Z_\lambda-\lambda)$ where $Z\sim\mathcal{P}(\lambda).$ Let $X_{nk}\overset{\text{iid}}{\sim} \mathcal{P}(\frac{\lambda}{n})$ for $1\le k\le n.$ $x(Z_\lambda-\lambda) \overset{d}{=} x\sum_{k=1}^n(X_{nk}-\frac{\lambda}{n})$ so it is ID.
(case 3. $\mu$ has masses at $x_1,\cdots,x_k.$)
Let $\mu\{x_i\}=\delta_i>0$ and $\varphi_i(t) = \exp\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu_i(x) \}$ where $\mu_i(\mathbb{R})=\delta_i.$ By case 2, $\varphi_i$ is ID with mean 0 and variance $\delta_i.$ Thus for all $n,$ there exists ch.f.s $\varphi_{jn}$ such that $\varphi_n = (\varphi_{jn})^n.$ It follows that $\varphi = \prod_{j=1}^k \varphi_j = \left(\prod_{j=1}^k \varphi_{jn}\right)^n$ thus $\varphi$ is ID. Let $X\sim\varphi$ and $X_i\overset{\text{indep}}{\sim} \varphi_i$ then $X\overset{d}{=}X_1+\cdots+X_k$ so $EX=0,\text{Var}(X)=\mu\{x_1,\cdots,x_k\}.$
(case 4. general finite $\mu$.)
Let $\mu_k\{j\cdot2^{-k}\} = \mu(j\cdot2^{-k}, (j+1)2^{-k}],$ $j\in J_k = \{0,\pm1,\pm2,\cdots,\pm2^{2k}\}.$ Then $\mu_k$ has masses on $\{j\cdot2^{-k}:~ j \in J_k\}.$ Since $\mu_k(\mathbb{R}) \to \mu(\mathbb{R}) > 0$ as $k\to\infty,$ $\mu_k(\mathbb{R})>0$ for all large $k.$
Now assume that $f:\mathbb{R}\to\mathbb{R}$ is continuous and vanishes at infinity (i.e. $\lim_{|x|\to\infty}f(x)=0$). Let $$f_k = \begin{cases} f(j\cdot2^{-k}) &,~ x\in (j\cdot2^{-k}, (j+1)2^{-k}] \\ 0 &,~ \text{otherwise} \end{cases}$$ be a step function, then $\int{f}d\mu_k = \int{f_k}d\mu.$ As $k\to\infty,$ $f_k\to f.$ Since $|f_k| \le |f| \le \sup_x |f(x)| < \infty,$ apply BCT and we get $\int{f_k}d\mu \to \int{f}d\mu.$
By the case 3, $\varphi_k(t) = \exp\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu_k(x) \}$ is ID. since the integrand is continuous and vanishes at infinity, $\varphi_k \to \varphi$ as $k\to\infty.$ Since $\varphi(0)=1$ and $\varphi$ is continuous at 0, by continuity theorem $\varphi$ is a ch.f. for some random variable.
In addition, $EX^2 \le \liminf_k EX_k^2 < \infty$ for $X\sim\varphi$ and $X_k\sim\varphi_k.$ By moment generating property of ch.f., $iEX = \varphi'(0) = 0$ and $-\text{Var}(X) = -\mu(\mathbb{R}).$ Let another $\psi_n(t) = \exp\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\frac{\mu}{n}(x) \}$ then it is a ch.f. Observe that $\varphi = (\psi_n)^n$ so $\varphi$ is ID.

In other words, \(\log \varphi(t) = \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu(x).\) We call the right hand side the canonical representation of $\varphi$ and $\mu$ the canonical measure. Note that $\frac{|e^{itx}-1-itx|}{x^2} \le t^2$ so the integral is well-defined. For $x=0,$ we define $\frac{e^{itx}-1-itx}{x^2} \big\vert_{x=0} = -\frac{t^2}{2}$ by continuity. Also note that \(\frac{|e^{itx}-1-itx|}{x^2} \le t^2 \wedge \frac{2|t|}{|x|} \to 0 \text{ as } |x|\to\infty.\) This follows from error estimation of the second-order Taylor series.

Necessity

To show the necessity part for more general class of characteristic functions, we define the condition R.

A rowwise independent triangular array $(X_{nk})_{k=1}^{r_n}$ satisfies R if the followings hold.
(i) $EX_{nk}=0,$ $\sigma_{nk}^2 = EX_{nk}^2 < \infty,$ $s_n^2 = \sum_{k=1}^{r_n} \sigma_{nk}^2 > 0.$
(ii) $\sup_n s_n^2 < \infty.$
(iii) $\max_{1\le k\le r_n} \sigma_{nk}^2 \to 0$ as $n\to\infty.$

For the proof of the next theorem, we need the following lemma.

Let $(\mu_n)$ be a sequence of finite measures with $\sup_n \mu_n(\mathbb{R}) < \infty.$ There exists a subsequence $(\mu_{nk})$ and a finite measure $\mu$ such that $\mu_{nk} \overset{w}{\to} \mu$ and $\int{h}d\mu_{nk} \to \int{h}d\mu$ for all $h$ that is continuous and vanishes at infinity.
Let $F$ be the limiting distribution of $S_n=X_{n1}+\cdots+X_{nr_n}$ for some $(X_{nk})$ that satisfies R. Then $\varphi,$ the ch.f. of $F,$ has a unique canonical representation: $$\varphi(t) = \text{exp}\left\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu(x) \right\}.$$
expand proof

$$\begin{aligned} \bigg| \underbrace{\prod_{k=1}^{r_n} \varphi_{nk}(t)}_\text{(i)} - \underbrace{\prod_{k=1}^{r_n} e^{\varphi_{nk}(t)-1}}_\text{(ii)} \bigg| &\le \sum_{k=1}^{r_n} |\varphi_{nk}(t) - e^{\varphi_{nk}(t)-1}| \\ &\le \sum_{k=1}^{r_n} |\varphi_{nk}(t) - 1|^2 \\ &\le \sum_{k=1}^{r_n} (t^2\sigma_{nk}^2)^2 \\ &\le t^4 \max_{1\le k\le r_n} \sigma_{nk}^2 \cdot s_n^2 \to 0. \end{aligned}$$ The first inequality is from 3.4.3, the second is from 3.4.4, the third is from 3.3.19, and $\to0$ is by condition R. In addition, $$\text{(i)} \to \varphi \text{ as } n\to\infty$$ also by condition R. $$\begin{aligned} \text{(ii)} &= \sum_{k=1}^{r_n} \int(e^{itx}-1) dF_{nk}(x) \\ &= \sum_{k=1}^{r_n} \int\frac{e^{itx}-1-itx}{x^2} x^2 dF_{nk}(x) \\ &= \int\frac{e^{itx}-1-itx}{x^2} d\left(\sum_{k=1}^{r_n}x^2 F_{nk}(x)\right) \end{aligned}$$ Let $\mu_n(-\infty,x] = \sum_{k=1}^{r_n} \int_{-\infty}^xy^2 dF_{nk}(y),$ then $$\text{(ii)} = \int\frac{e^{itx}-1-itx}{x^2} d\mu_n(x)$$ and $\mu_n(\mathbb{R}) = s_n^2.$ So $\sup_n \mu_n(\mathbb{R}) < \infty$ and there exists $(\mu_{nj}),\mu$ such that $\mu_{nj} \overset{w}{\to} \mu$ and $\int{h}d\mu_{nj} \to \int{h}d\mu$ for all $h$ that is continuous and vanishes at infinity. By the above mentioned fact, $$\int\frac{e^{itx}-1-itx}{x^2} d\mu_{nj}(x) \to \int\frac{e^{itx}-1-itx}{x^2} d\mu(x).$$ By convergence of (i) and (ii), the existence part of the proof is done.
For the uniqueness part, we only need to show that such $\mu$ is unique. Suppose $$\int\frac{e^{itx}-1-itx}{x^2} d\mu(x) = \int\frac{e^{itx}-1-itx}{x^2} d\nu(x),~ \forall t.$$ This implies $\int e^{itx}d\mu(x) = \int e^{itx}d\nu(x).$ Put $t=0$ to both sides and we get $c:=\mu(\mathbb{R})=\nu(\mathbb{R}).$ Dividing both sides with $c,$ $\mu/c$ and $\nu/c$ becomes probabilty measures with identical ch.f.s and the proof is done.


Acknowledgement

This post series is based on the textbook Probability: Theory and Examples, 5th edition (Durrett, 2019) and the lecture at Seoul National University, Republic of Korea (instructor: Prof. Sangyeol Lee).