Kwon, Oh-Ran2023-11-282023-11-282023-07https://hdl.handle.net/11299/258782University of Minnesota Ph.D. dissertation. July 2023. Major: Statistics. Advisor: Hui Zou. 1 computer file (PDF); ix, 126 pages.This dissertation includes three projects on three different machine-learning topics. Each introduces a modern machine learning method for classification, multi-response regression, and feature selection problems, respectively. The first project is titled "Leaky Hockey Stick Loss: The First Negatively Divergent Margin-based Loss Function for Classification". Many modern classification algorithms are formulated through the regularized empirical risk minimization (ERM) framework, where the risk is defined based on a loss function. The loss function in decision theory is non-negative by definition. However, this project highlights that the non-negativity of the loss function in ERM is not necessary in order to be classification-calibrated and to produce a Bayes consistent classifier, which is demonstrated by introducing the leaky hockey stick loss. The second project is titled "Enhanced Response Envelope via Envelope Regularization". The envelope model provides substantial efficiency gains over the standard multivariate linear regression by identifying the material part of the model and by excluding the immaterial part. This project proposes the enhanced response envelope model by incorporating the envelope regularization term in its formulation. It is shown that the enhanced response envelope can yield better out-of-sample prediction risk than the original envelope model. Finally, the third project is titled "Exactly Uncorrelated Sparse Principal Component Analysis". Sparse principal component analysis (PCA) aims to find principal components as linear combinations of a subset of the original input variables without sacrificing the fidelity of the classical PCA. Many applications of PCA prefer uncorrelated principal components. However, it is nontrivial to handle sparsity and uncorrelatedness properties in a sparse PCA method. This project proposes an exactly uncorrelated sparse PCA method named EUSPCA, whose formulation is motivated by the original views and motivations of PCA as advocated by Pearson and Hotelling.enThesis or Dissertation