3.9. Infinitely divisible distributions
A certain kind of well behaving distributions has characteristic functions that can be represented in canonical form. In this section we cover conditions that such distributions have and its canonical representation.
Infinitely divisible distributions
(i) There exists a squence of distributions $(F_n)$ such that $F=F_n*\cdots* F_n$ for all $n\in\mathbb{N}.$
(ii) There exists random variables $X,X_{nk}$ in a probability space $(\Omega, \mathcal{F}, P)$ such that $X\overset{d}{=}X_{n1}+\cdots+X_{nn}$ for all $n,$ where $X\sim F$, $X_{nk}\sim F_n$ for all $k$ and $X_{nk}$'s are rowwise independent.
(iii) There exists a sequence of characteristic functions $(\varphi_n)$ such that $\varphi = (\varphi_n)^n.$
Here $*$ denotes convolution. In fact all three conditions are equivalent. As an example, we can easily check that a normal distribution $X\sim\mathcal{N}(\mu,\sigma^2)$ is infinitely divisible since $X\overset{d}{=}X_{n1}+\cdots+X_{nn}$ for rowwise independent $X_{nk}\sim\mathcal{N}(\frac{\mu}{n},\frac{\sigma^2}{n}).$
First important property is that characteristic functions of ID distributions always have non-zero values. For this, we need a lemma that applies to all characteristic functions.
Proof is simple using elementary trigonometrics. Notice that $|\varphi|^2$ is a real-valued ch.f. so it suffices to show that $\text{Re}(1-\varphi(2t)) \le 4\text{Re}(1-\varphi(t))$ for a given $\varphi.$ $$\begin{aligned} \text{Re}(1-\varphi(t)) &= \int(1-\cos tx)dF(x) \\ &= \int2\sin^2 \frac{t}{2}dF(x) \\ &= \int\frac{\sin^2 tx}{2\cos^2\frac{t}{2}x}dF(x) \\ &= \int\frac{1}{2}\sin^2 tx dF(x) \\ &= \int\frac{1}{4}(1-\cos 2tx) dF(x) \\ &= \frac{1}{4}\text{Re}(1-\varphi(2t)). \end{aligned}$$
Proof is by induction. Since $\varphi$ is ID, there exists $\varphi_n$ such that $\varphi=(\varphi_n)^n.$ We know that $\varphi\to1$ as $t\to0,$ so there exists $a>0$ such that $|\varphi(t)|>0$ for all $|t|\le a.$
Given $t$ that $|t|\le a,$ $$|\varphi_n(t)| = |\varphi(t)^\frac{1}{n}| \ge \left(\inf_{|t|\le a}|\varphi(t)|\right)^\frac{1}{n} \to_n 1.$$ so for all $0\le\epsilon\le1,$ there exists $N>0$ such that $|\varphi_n(t)| > 1-\epsilon$ for all $n\ge N.$ By the lemma, for $n\ge N$ and $|t|\le a,$ $$1-|\varphi_n(2t)|^2 \le 4(1-(1-\epsilon)^2) \le 8\epsilon.$$ Thus for large n, $|\varphi_n(2t)|^2 \ge 1-8\epsilon > 0,$ for all $0 < \epsilon < 1/8.$ This gives $\varphi(2t) \ne 0$ for all $|t|\le a.$ Repeatedly apply the process to get $\varphi \ne 0$ for all $t.$
Canonical representation
Characteristic functions of Infinitely divisible distributions can be uniquely represented in a certain form. Furthermore, if a characteristic function can be written in such form, then it is infinitly divisible. We call it a canonical form. While there are several equivalent canonical representations, I would like to cover the one by Kolmogorov. The first theorem is about sufficiency of ID distribution.
Sufficiency
expand proof
(case 1. $\mu$ has a mass only at 0.)
Let $\sigma^2 = \mu(\mathbb{R}) = \mu\{0\}>0$ then $\varphi(t) = e^\frac{t^2\sigma^2}{2}$ which is the ch.f. of $\mathcal{N}(0,\sigma^2)$ so it is ID.
(case 2. $\mu$ has a mass only at $x\ne0.$)
Let $\mu\{x\}=\lambda x^2$ for some $\lambda>0.$ Then $\varphi(t)=e^{\lambda(e^{itx}-1-itx)}$ which is a ch.f. of $x(Z_\lambda-\lambda)$ where $Z\sim\mathcal{P}(\lambda).$ Let $X_{nk}\overset{\text{iid}}{\sim} \mathcal{P}(\frac{\lambda}{n})$ for $1\le k\le n.$ $x(Z_\lambda-\lambda) \overset{d}{=} x\sum_{k=1}^n(X_{nk}-\frac{\lambda}{n})$ so it is ID.
(case 3. $\mu$ has masses at $x_1,\cdots,x_k.$)
Let $\mu\{x_i\}=\delta_i>0$ and $\varphi_i(t) = \exp\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu_i(x) \}$ where $\mu_i(\mathbb{R})=\delta_i.$ By case 2, $\varphi_i$ is ID with mean 0 and variance $\delta_i.$ Thus for all $n,$ there exists ch.f.s $\varphi_{jn}$ such that $\varphi_n = (\varphi_{jn})^n.$ It follows that $\varphi = \prod_{j=1}^k \varphi_j = \left(\prod_{j=1}^k \varphi_{jn}\right)^n$ thus $\varphi$ is ID. Let $X\sim\varphi$ and $X_i\overset{\text{indep}}{\sim} \varphi_i$ then $X\overset{d}{=}X_1+\cdots+X_k$ so $EX=0,\text{Var}(X)=\mu\{x_1,\cdots,x_k\}.$
(case 4. general finite $\mu$.)
Let $\mu_k\{j\cdot2^{-k}\} = \mu(j\cdot2^{-k}, (j+1)2^{-k}],$ $j\in J_k = \{0,\pm1,\pm2,\cdots,\pm2^{2k}\}.$ Then $\mu_k$ has masses on $\{j\cdot2^{-k}:~ j \in J_k\}.$ Since $\mu_k(\mathbb{R}) \to \mu(\mathbb{R}) > 0$ as $k\to\infty,$ $\mu_k(\mathbb{R})>0$ for all large $k.$
Now assume that $f:\mathbb{R}\to\mathbb{R}$ is continuous and vanishes at infinity (i.e. $\lim_{|x|\to\infty}f(x)=0$). Let $$f_k = \begin{cases} f(j\cdot2^{-k}) &,~ x\in (j\cdot2^{-k}, (j+1)2^{-k}] \\ 0 &,~ \text{otherwise} \end{cases}$$ be a step function, then $\int{f}d\mu_k = \int{f_k}d\mu.$ As $k\to\infty,$ $f_k\to f.$ Since $|f_k| \le |f| \le \sup_x |f(x)| < \infty,$ apply BCT and we get $\int{f_k}d\mu \to \int{f}d\mu.$
By the case 3, $\varphi_k(t) = \exp\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu_k(x) \}$ is ID. since the integrand is continuous and vanishes at infinity, $\varphi_k \to \varphi$ as $k\to\infty.$ Since $\varphi(0)=1$ and $\varphi$ is continuous at 0, by continuity theorem $\varphi$ is a ch.f. for some random variable.
In addition, $EX^2 \le \liminf_k EX_k^2 < \infty$ for $X\sim\varphi$ and $X_k\sim\varphi_k.$ By moment generating property of ch.f., $iEX = \varphi'(0) = 0$ and $-\text{Var}(X) = -\mu(\mathbb{R}).$ Let another $\psi_n(t) = \exp\{ \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\frac{\mu}{n}(x) \}$ then it is a ch.f. Observe that $\varphi = (\psi_n)^n$ so $\varphi$ is ID.
In other words,
\(\log \varphi(t) = \int_{-\infty}^\infty \frac{e^{itx}-1-itx}{x^2} d\mu(x).\)
We call the right hand side the canonical representation of $\varphi$ and $\mu$ the canonical measure. Note that $\frac{|e^{itx}-1-itx|}{x^2} \le t^2$ so the integral is well-defined. For $x=0,$ we define $\frac{e^{itx}-1-itx}{x^2} \big\vert_{x=0} = -\frac{t^2}{2}$ by continuity. Also note that
\(\frac{|e^{itx}-1-itx|}{x^2} \le t^2 \wedge \frac{2|t|}{|x|} \to 0 \text{ as } |x|\to\infty.\)
This follows from error estimation of the second-order Taylor series.
Necessity
To show the necessity part for more general class of characteristic functions, we define the condition R.
(i) $EX_{nk}=0,$ $\sigma_{nk}^2 = EX_{nk}^2 < \infty,$ $s_n^2 = \sum_{k=1}^{r_n} \sigma_{nk}^2 > 0.$
(ii) $\sup_n s_n^2 < \infty.$
(iii) $\max_{1\le k\le r_n} \sigma_{nk}^2 \to 0$ as $n\to\infty.$
For the proof of the next theorem, we need the following lemma.
expand proof
$$\begin{aligned} \bigg| \underbrace{\prod_{k=1}^{r_n} \varphi_{nk}(t)}_\text{(i)} - \underbrace{\prod_{k=1}^{r_n} e^{\varphi_{nk}(t)-1}}_\text{(ii)} \bigg| &\le \sum_{k=1}^{r_n} |\varphi_{nk}(t) - e^{\varphi_{nk}(t)-1}| \\ &\le \sum_{k=1}^{r_n} |\varphi_{nk}(t) - 1|^2 \\ &\le \sum_{k=1}^{r_n} (t^2\sigma_{nk}^2)^2 \\ &\le t^4 \max_{1\le k\le r_n} \sigma_{nk}^2 \cdot s_n^2 \to 0. \end{aligned}$$ The first inequality is from 3.4.3, the second is from 3.4.4, the third is from 3.3.19, and $\to0$ is by condition R. In addition, $$\text{(i)} \to \varphi \text{ as } n\to\infty$$ also by condition R. $$\begin{aligned} \text{(ii)} &= \sum_{k=1}^{r_n} \int(e^{itx}-1) dF_{nk}(x) \\ &= \sum_{k=1}^{r_n} \int\frac{e^{itx}-1-itx}{x^2} x^2 dF_{nk}(x) \\ &= \int\frac{e^{itx}-1-itx}{x^2} d\left(\sum_{k=1}^{r_n}x^2 F_{nk}(x)\right) \end{aligned}$$ Let $\mu_n(-\infty,x] = \sum_{k=1}^{r_n} \int_{-\infty}^xy^2 dF_{nk}(y),$ then $$\text{(ii)} = \int\frac{e^{itx}-1-itx}{x^2} d\mu_n(x)$$ and $\mu_n(\mathbb{R}) = s_n^2.$ So $\sup_n \mu_n(\mathbb{R}) < \infty$ and there exists $(\mu_{nj}),\mu$ such that $\mu_{nj} \overset{w}{\to} \mu$ and $\int{h}d\mu_{nj} \to \int{h}d\mu$ for all $h$ that is continuous and vanishes at infinity. By the above mentioned fact, $$\int\frac{e^{itx}-1-itx}{x^2} d\mu_{nj}(x) \to \int\frac{e^{itx}-1-itx}{x^2} d\mu(x).$$ By convergence of (i) and (ii), the existence part of the proof is done.
For the uniqueness part, we only need to show that such $\mu$ is unique. Suppose $$\int\frac{e^{itx}-1-itx}{x^2} d\mu(x) = \int\frac{e^{itx}-1-itx}{x^2} d\nu(x),~ \forall t.$$ This implies $\int e^{itx}d\mu(x) = \int e^{itx}d\nu(x).$ Put $t=0$ to both sides and we get $c:=\mu(\mathbb{R})=\nu(\mathbb{R}).$ Dividing both sides with $c,$ $\mu/c$ and $\nu/c$ becomes probabilty measures with identical ch.f.s and the proof is done.
Acknowledgement
This post series is based on the textbook Probability: Theory and Examples, 5th edition (Durrett, 2019) and the lecture at Seoul National University, Republic of Korea (instructor: Prof. Sangyeol Lee).