1.2. Distributions

Dec 18, 2020 • Sihyung Park
probability   Durrett  

$\newcommand{\argmin}{\mathop{\mathrm{argmin}}\limits}$ $\newcommand{\argmax}{\mathop{\mathrm{argmax}}\limits}$

In this subsection, we define random variables and distribution functions.

• Random variables
• Distribution functions

Random variables

In measure theory, a function on a measurable space $A$ onto another measurable space $B$ is measurable if its inverse image of measurable sets are measurable sets. If $A$ is a probability space and $B$ is a Borel measurable space of $\mathbb{R}$, then we call this function a random variable.

Let $(\Omega, \mathcal{F}, P)$ be a probability space. $X: \Omega \to \mathbb{R}$ is a random variable, if for all $B \in \mathcal{ B}$, $X^{-1}(B) \in \mathcal{F}$.

We also say $X$ is $\mathcal{ F}$- measurable or write $X \in \mathcal{ F}$. That is, random variables are Borel measurable real-valued functions defined on a probability space.

(1) If $(\Omega, \mathcal{ F}, P)$ is a discrete probability space, every function is a random variable.
(2) An indicator function $\mathbf{1}_A$, $A \in \mathcal{ F}$ is a random variable.

In undergraduate statistics, we used random variables as if they were values. Now we can understand true meaning behind these notations: \(P(-1\le X \le 1) = P(X^{-1}([-1, 1])) = P(\{\omega\in\Omega:~ X(\omega) \in [-1,1]\})\) and such. Random variables are defined as a measurable function so that their inverse images can be measured by the probability measure $P$.

An important fact is that every random variable induces a probability measure on $(\mathbb{ R}, \mathcal{ B}(\mathbb{ R}))$. (note that it is not on $(\Omega, \mathcal{ F})$.)

$P_X(A) := P(X \in A),~ A \in \mathcal{\mathbb{R}}$ is a probability measure induced by a random variable $X$. $P_X$ is called the distribution of $X$.

It is not difficult to show that such $P_X$ is a probability measure.

Distribution functions

A distribution function of $X$ is defined in terms of probability.

$F:\mathbb{ R}\to\mathbb{ R}$ is a distribution function of $X$.
$\Leftrightarrow$ $F(x) := P(X\le x) = P_X(-\infty, x],~ \forall x \in \mathbb{ R}$.

We already saw in section 1.1 that a distribution function uniquely determines a distribution (or a random variable). The following theorem implies that every function that satisfies some conditions can be regarded as a distribution function of some random variable.

$F$ is a distribution of some random variable $X$ if and only if it is (i) non-decreasing, (ii) right-continuous, (iii) $\lim_{x\to-\infty}F(x) = 0$ and $\lim_{x\to\infty}F(x) = 1$.

expand proof

($\Rightarrow$) is trivial.
($\Leftarrow$) we proof this by construction. Let $\Omega = [0,1]$, $\mathcal{ F} = \mathcal{B}([0,1])$, $P=\lambda$ (Lebesgue measure). Let $X(\omega) = \sup\{y:~ F(y)<\omega\}$. Then $X$ is a random variable. To show $P(X\le x) = P(\{\omega:~ 0\le\omega\le F(x)\}),~ \forall x$, it is equivalent to show that $\{\omega:~ X(\omega) \le x\} = \{\omega: \omega \le F(x)\},~ \forall x$.
  First given $x$, $\omega_0 \in \{\omega:~ \omega \le F(x)\}$, we get $x \notin \{y:~ F(y) < \omega_0\}$. Thus $X(\omega_0) \le x$ and we get $\{\omega:~ X(\omega) \le x\} \supset \{\omega: \omega \le F(x)\}$. Next, suppose $\omega_0 \notin \{\omega: \omega \le F(x)\}$, then $\omega_0 > F(x)$. Since $F$ is right-continuous, there exists $\epsilon > 0$ such that $F(x) \le F(x+\epsilon) < \omega_0$. Hence $x<x+\epsilon\le X(\omega_0)$ and $\{\omega:~ X(\omega) \le x\} \subset \{\omega: \omega \le F(x)\}$. Finally we get $F(x) = \lambda(0, F(x)] = P(\{\omega:~ X(\omega) \le x\}) = P(X \le x)$.

Acknowledgement

This post is based on the textbook Probability: Theory and Examples, 5th edition (Durrett, 2019) and the lecture at Seoul National University, Republic of Korea (instructor: Prof. Johan Lim).