### 4. Hilbert Space Theory

Objective of this chapter is to completely characterize $L^2(\mu),$ the famous Hilbert space. To achieve our goal, we will use the fact that a Hilbert space can be seen as an infinite-dimensional vector space where there exists a “orthogonal basis”. i.e. any element in the space can be decomposed into an infinite linear combination of orthogonal components. In fact, the basis decomposition yields to the main result that $L^2$ is actually isomorphic to $\ell^2.$

» continue reading

### 3. $L^p$ Space

As we defined the Lebesgue integral and proved the basic properties, it is time to study the space of functions with finite integrals.

» continue reading

### 2. Construction of Lebesgue Measure

In this chapter, we construct the Lebesgue measure on $\mathbb{R}^d.$ For this, we prove Riesz representation theorem and use the result to construct a complete measure space $(\mathbb{R}, \mathfrak{M}, m)$ such that integration with respect to $m$ is equal to Riemann integration for all Riemann-integrable functions. We then use $\sigma$-compactness of $\mathbb{R}$ to show that such $m$ is the Lebesgure measure.

» continue reading

### 4. Multivariate Kernel Density Estimation

Until now, I only covered univariate cases. From now on, our focus is on multivariate cases where samples $X_1,\cdots,X_n\in\mathbb{R}^d$ are independently drawn from the density $p:\mathbb{R}^d\to[0,\infty)$ with respect to the Lebesgue measure.

» continue reading

### 3. Bandwidth Selection

Recall that the optimal bandwidth $h_\text{opt}$ derived from MISE was given as the following:

» continue reading