Nonlinear Regression Based on Neural Networks: Prediction, Sufficient Dimension Reduction, and Model Compression
2022-06
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Nonlinear Regression Based on Neural Networks: Prediction, Sufficient Dimension Reduction, and Model Compression
Authors
Published Date
2022-06
Publisher
Type
Thesis or Dissertation
Abstract
In the modern era, neural networks have achieved great success in various fields of applications. Since neural networks allow the modeling of nonlinear relationships, they have become popular and useful tools for tackling different problems such as prediction, classification, clustering, dimension reduction, visualization, computer vision, and many others. This thesis focuses on three interrelated topics in the study of neural networks under the regression setting with two primary goals. One is to incorporate traditional statistical learning methods and recent machine learning techniques, the other is to utilize statistical theory to provide theoretical justification and understanding of some neural network methods. With the rapid growth and development in modern information technology, high-dimensional data are ubiquitous in various scientific fields. Specifically, recent machine learning techniques on prediction performances under high-dimensional data frameworks have supported many applications for gene expression inferencing. We propose a deep neural network learning approach that incorporates deep learning techniques and statistical methods to effectively enhance the predictive accuracy to address a multi-task learning problem in gene expression profiling. Unlike the conventional complicated multi-task learning methods, our proposed method can effectively learn the interrelations from larger-scale tasks. As the data structure of high dimensions may raise challenges in terms of both computation and inferencing, it is often of interest to find a reduced dimensionality of the data from linear combinations of the predictors that can retain as much original information as possible in the relationship between the response and the original predictors. We propose a sufficient dimension reduction method that is able to not only obtain the structural dimension effectively without prior knowledge, but also estimate the central space. In addition, the framework is extended to the approximated case, which may be more applicable in practice. With the development of powerful extensions of classical works in deep learning, many state-of-the-art architectures today have achieved spectacular results and increasing attention in various applications. However, these works generally rely on complex architectures with tens and even hundreds of layers, which result in the need for tremendous storage space, time allocation, computation costs, and a copious amount of data. These concerns motivate research in model compression. Many empirical results have shown that when a certain model is more complex than what might be needed, different techniques can be applied to reduce model complexity and still retain the generalization power in the original model. The theoretical work, however, is limited. We examine neural network pruning from the perspective of sparse linear approximation and provide a theoretical understanding of how to characterize a network that is compressible and by how much it can be properly pruned with little prediction accuracy degradation.
Description
University of Minnesota Ph.D. dissertation.June 2022. Major: Statistics. Advisors: Yuhong Yang, Jie Ding. 1 computer file (PDF); x, 117 pages.
Related to
Replaces
License
Collections
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Yang, Wenjing. (2022). Nonlinear Regression Based on Neural Networks: Prediction, Sufficient Dimension Reduction, and Model Compression. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/241630.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.