Variable Selection Deviation Measures

Abstract

Many exciting results have been obtained on model selection methods for high- dimensional data in both efficient algorithms and theoretical developments. The powerful penalization methods for the variable selection in both regression and classification can give sparse representations of the data even when the number of predictors is much larger than the sample size. One important question then is: How do we know when a sparse pattern identified by a model method is reliable? In this dissertation, we propose variable selection deviation measures that give one a proper sense on how many predictors in the selected set are likely trustworthy in certain aspects. In the first part of this thesis, we investigate the instability of the penalization methods (Lasso, SCAD, MCP and Stability Selection) in term of the variable selection. Three instability measures are studied: sequential instability, parametric bootstrap instability and perturbation instability. Then, we propose a variable selection deviation measure (VSD) to quantify the uncertainty of the selected sparse set. Simulation and a real data example demonstrate the utility of the VSD measures for application. In the second part, we propose the VSD measures for the generalized linear model (GLM), in particular, logistic regression. The VSD measures rely on good weights on the models and they help quantifying the deviation of the selected model from the true model. For the generalized linear model, we adopt the ACM (Yang (2000)) to define the weight function for GLM VSD measures. We also propose the weight function and algorithm of the VSD for Poisson regression. We implement the VSD measures on simulated dataset and four real data examples. We build an R package called glmvsd to calculate the VSD measures. After providing the target model that user wants to evaluate, this package will calculate the VSD measures according to different weight functions defined in this thesis. The package can also calculate the three instability measures for several model selection methods. In Chapter 4, the manual of this package is presented.