A Unified Algorithm for Fitting Penalized Models with High Dimensional Data

Abstract

In the light of high dimensional problems, research on the penalized model has received much interest. Correspondingly, several algorithms have been developed for solving penalized high dimensional models. In this thesis, we propose fast and efficient unified algorithms for computing the solution path for a collection of penalized models. In particular, we study the algorithm for solving l1 penalized learning problems and the algorithm for solving group-lasso learning problems. These algorithm take advantage of a majorization-minimization trick to make each update simple and efficient. The algorithms also enjoy a proven convergence property. To demonstrate the generality of our algorithms, we further extend these algorithms on a class of elastic net penalized large margin classification methods and the elastic net penalized Cox's proportional hazards model. These algorithms have been implemented in three R packages gglasso, gcdnet and fastcox, which are publicly available from the Comprehensive R Archive Network (CRAN) at http://cran.r-project.org/web/packages. On simulated and real data, our algorithms consistently outperform the existing software in speed for computing penalized models and often delivers better quality solutions.