Several Modern Machine Learning Methods: Theory and Algorithm

Thumbnail Image

Persistent link to this item

View Statistics

Journal Title

Journal ISSN

Volume Title


Several Modern Machine Learning Methods: Theory and Algorithm

Published Date




Thesis or Dissertation


This dissertation includes three projects on three different machine-learning topics. Each introduces a modern machine learning method for classification, multi-response regression, and feature selection problems, respectively. The first project is titled "Leaky Hockey Stick Loss: The First Negatively Divergent Margin-based Loss Function for Classification". Many modern classification algorithms are formulated through the regularized empirical risk minimization (ERM) framework, where the risk is defined based on a loss function. The loss function in decision theory is non-negative by definition. However, this project highlights that the non-negativity of the loss function in ERM is not necessary in order to be classification-calibrated and to produce a Bayes consistent classifier, which is demonstrated by introducing the leaky hockey stick loss. The second project is titled "Enhanced Response Envelope via Envelope Regularization". The envelope model provides substantial efficiency gains over the standard multivariate linear regression by identifying the material part of the model and by excluding the immaterial part. This project proposes the enhanced response envelope model by incorporating the envelope regularization term in its formulation. It is shown that the enhanced response envelope can yield better out-of-sample prediction risk than the original envelope model. Finally, the third project is titled "Exactly Uncorrelated Sparse Principal Component Analysis". Sparse principal component analysis (PCA) aims to find principal components as linear combinations of a subset of the original input variables without sacrificing the fidelity of the classical PCA. Many applications of PCA prefer uncorrelated principal components. However, it is nontrivial to handle sparsity and uncorrelatedness properties in a sparse PCA method. This project proposes an exactly uncorrelated sparse PCA method named EUSPCA, whose formulation is motivated by the original views and motivations of PCA as advocated by Pearson and Hotelling.



University of Minnesota Ph.D. dissertation. July 2023. Major: Statistics. Advisor: Hui Zou. 1 computer file (PDF); ix, 126 pages.

Related to




Series/Report Number

Funding information

Isbn identifier

Doi identifier

Previously Published Citation

Suggested citation

Kwon, Oh-Ran. (2023). Several Modern Machine Learning Methods: Theory and Algorithm. Retrieved from the University Digital Conservancy,

Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.