Selected topics of high-dimensional sparse modeling

Abstract

In this thesis we study three problems over high-dimensional sparse modeling. We first discuss the problem of high-dimensional covariance matrix estimation. Nowadays, massive high-dimensional data are more and more common in scientific investigations. Here we focus on one type of covariance matrices - bandable covariance matrices in which the dependence structure of variables follows a nature order. Many off-diagonal elements are very small, especially when they are far away from diagonal, which technically makes the covariance matrix very sparse. It has been shown that the tapering covariance estimator attains the optimal minimax rates of convergence for estimating large bandable covariance matrices. The estimation risk critically depends on the choice of tapering parameter. We develop a Stein's Unbiased Risk Estimation (SURE) theory for estimating the Frobenius risk of the tapering estimator. SURE tuning selects the minimizer of SURE curve as the chosen tapering parameter. Covariance matrix is finally estimated according to the selected tapering parameter in the tapering covariance estimator. The second part of the thesis is about high-dimensional varying-coefficient model. Varying-coefficient model is used when the effects of some variables depend on the values of other variables. One interesting and useful varying-coefficient model is that the coefficients of all variables are changing over time. Non-parametric method based on B-splines is used to estimate marginal coefficient of each variable, and varing-coefficient Independence Screening (VIS) is proposed to screen important variables. To improve the performance of the algorithm, Iterative VIS (IVIS) procedure is proposed. In the third part of the thesis, we study a high-dimensional extension of traditional factor analysis by relaxing the independence assumption of the error term. In the new model, we assume that the inverse covariance is sparse but not necessarily diagonal. We propose a generalized E-M algorithm to fit the extended factor analysis model. Our new model not only makes factor analysis more flexible, but also could be used to discover the hidden conditional structure of variables after common factors are discovered and removed.