Between Dec 19, 2024 and Jan 2, 2025, datasets can be submitted to DRUM but will not be processed until after the break. Staff will not be available to answer email during this period, and will not be able to provide DOIs until after Jan 2. If you are in need of a DOI during this period, consider Dryad or OpenICPSR. Submission responses to the UDC may also be delayed during this time.
 

Nonlinear Regression Based on Neural Networks: Prediction, Sufficient Dimension Reduction, and Model Compression

Loading...
Thumbnail Image

Persistent link to this item

Statistics
View Statistics

Journal Title

Journal ISSN

Volume Title

Title

Nonlinear Regression Based on Neural Networks: Prediction, Sufficient Dimension Reduction, and Model Compression

Published Date

2022-06

Publisher

Type

Thesis or Dissertation

Abstract

In the modern era, neural networks have achieved great success in various fields of applications. Since neural networks allow the modeling of nonlinear relationships, they have become popular and useful tools for tackling different problems such as prediction, classification, clustering, dimension reduction, visualization, computer vision, and many others. This thesis focuses on three interrelated topics in the study of neural networks under the regression setting with two primary goals. One is to incorporate traditional statistical learning methods and recent machine learning techniques, the other is to utilize statistical theory to provide theoretical justification and understanding of some neural network methods. With the rapid growth and development in modern information technology, high-dimensional data are ubiquitous in various scientific fields. Specifically, recent machine learning techniques on prediction performances under high-dimensional data frameworks have supported many applications for gene expression inferencing. We propose a deep neural network learning approach that incorporates deep learning techniques and statistical methods to effectively enhance the predictive accuracy to address a multi-task learning problem in gene expression profiling. Unlike the conventional complicated multi-task learning methods, our proposed method can effectively learn the interrelations from larger-scale tasks. As the data structure of high dimensions may raise challenges in terms of both computation and inferencing, it is often of interest to find a reduced dimensionality of the data from linear combinations of the predictors that can retain as much original information as possible in the relationship between the response and the original predictors. We propose a sufficient dimension reduction method that is able to not only obtain the structural dimension effectively without prior knowledge, but also estimate the central space. In addition, the framework is extended to the approximated case, which may be more applicable in practice. With the development of powerful extensions of classical works in deep learning, many state-of-the-art architectures today have achieved spectacular results and increasing attention in various applications. However, these works generally rely on complex architectures with tens and even hundreds of layers, which result in the need for tremendous storage space, time allocation, computation costs, and a copious amount of data. These concerns motivate research in model compression. Many empirical results have shown that when a certain model is more complex than what might be needed, different techniques can be applied to reduce model complexity and still retain the generalization power in the original model. The theoretical work, however, is limited. We examine neural network pruning from the perspective of sparse linear approximation and provide a theoretical understanding of how to characterize a network that is compressible and by how much it can be properly pruned with little prediction accuracy degradation.

Description

University of Minnesota Ph.D. dissertation.June 2022. Major: Statistics. Advisors: Yuhong Yang, Jie Ding. 1 computer file (PDF); x, 117 pages.

Related to

Replaces

License

Collections

Series/Report Number

Funding information

Isbn identifier

Doi identifier

Previously Published Citation

Other identifiers

Suggested citation

Yang, Wenjing. (2022). Nonlinear Regression Based on Neural Networks: Prediction, Sufficient Dimension Reduction, and Model Compression. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/241630.

Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.