In Chapter 3, we will focus on the uniform law of large numbers and its sufficiencies. The first part of the chapter consists of the simplest case: ULLN under finite bracketing entropy condition. After that, techniques to prove sufficiencies of ULLN will be introduced. Finally, ULLN under limiting $L^1$-entropy, popular function classes, or under constraints in VC dimension will be covered.
» continue reading

This section covers the important results regarding the upper bound of entropies of popular classes of functions. We will accept most of the results without proving them, since it might obscure our objective: LLN and CLT on function spaces.
» continue reading

While computing complexity of classes of functions by its cardinality is natural, it reports complexity of every infinite dimensional classes to be the same. We need tools that can differentiate complexity more meticulously. Such tools are entropies of various kinds.
» continue reading

Chapter 2 concerns essential notions of the field. First, we will define the empirical distribution and the empirical process of a random sample. Next, various kinds of entropies and their basic relationships will be covered. Finally, upper bounds of entropy of some of the function classes will be mentioned.
» continue reading

The last example de Geer (2000) presents is the consistency of maximum likelihood estimates as the last example.¹ Specifically, I would like to briefly prove Hellinger consistency of MLE provided that some form of law of large numbers holds.
» continue reading