Minimax estimation and model identification for high-dimensional regression.

Abstract

This dissertation consists of two parts. In Part I, adaptive minimax estimation over sparse `q-hulls is studied. Given a dictionary of Mn initial estimates of the unknown regression function, we aim to construct linearly aggregated estimators that target the best performance among all linear combinations under a sparse q-norm (0 <_ q <_ 1) constraint. Besides identifying the optimal rates of aggregation for these `q-aggregation problems, our multi-directional (or adaptive) strategies by model mixing or model selection achieve the optimal rates simultaneously over the full range of 0 <_ q <_ 1 for general Mn and upper bound tn of the q-norm. Both random and fixed designs, with known or unknown error variance, are handled, and the `q-aggregations examined in this work cover major types of aggregation problems previously studied in the literature. Consequences on minimax-rate adaptive regression under `q- constrained coefficients are also provided. In Part II, the relationship between consistency and minimax-rate optimality in possibly high-dimensional regression estimation is investigated. In model selection where the true model is fixed, it is now well-known that if a model selection method is consistent, it cannot be minimax-rate optimal at the same time. We investigate this con ict in a high-dimensional regression setting where the true model is a changing target, and show that consistency and minimaxrate optimality may co-exist in a single model selection method. Our results provide a comprehensive guideline for characteristics of a model selection method which can be consistent and minimax-rate optimality at the same time.