Topics in High Dimensional Statistics

Abstract

In the era of big data, statistical application fields have been frequently encounteringdata sets where features measured for each sample are more than sample size. In these data sets, which we call high-dimensional data, traditional statistical tools are not feasible, imposing challenges from both theoretical and computational perspectives. This thesis is devoted to discuss several novel high dimensional methodologies with both solid theoretical justification and computational efficiency to cope with the new challenges in big data era. Chapter 2 of the thesis systematically studies the estimation of a high dimensional heteroscedastic regression model. In particular, the emphasis is on how to detect and estimate the heteroscedasticity effects reliably and efficiently. To this end, we propose a cross-fitted residual regression approach and prove the resulting estimator is selectionconsistent for heteroscedasticity effects and establish its rates of convergence. Efficient algorithm is developed such that our method can be solved extremely fast. Chapter 3 introduces a novel methodology called sparse convoluted rank regression. The method is shown to maintain the good theoretical property of rank regression, a very popular alternative to the least squares. Moreover, it avoids the computational burden of rank regression caused by non-smooth loss, by adopting a smooth objective function, which is derived from a statistical point of view. Chapter 4 proposes a method called density convoluted support vector machine (DCSVM) for high dimensional classification. Theoretical error bound is established, and numerical examples demonstrate that our method outperforms SVM and other competitors in terms of both prediction accuracy and computational speed.