Classification with Mixture of Experts Models
2022-12
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Classification with Mixture of Experts Models
Authors
Published Date
2022-12
Publisher
Type
Thesis or Dissertation
Abstract
Mixture of experts (MoE) layers allow for an increase in model parameters without acorresponding increase in computational cost by utilizing sparse dynamic computation
across “expert” modules during both inference and training. In this work we study
whether these sparse activations of expert modules are semantically meaningful in
classification tasks; in particular, we investigate whether experts develop specializations
that reveal semantic relationships among classes. This work replaces the classification
head of selected deep networks on classification tasks with an MoE layer. MoE layers allow
for the experts to specialize in ways that are qualitatively intuitive, and quantitatively
match structural descriptions of their relationships better than the classification heads
in the original networks.
Keywords
Description
University of Minnesota M.S. thesis. December 2022. Major: Computer Science. Advisor: Dongyeop Kang. 1 computer file (PDF); vi, 35 pages.
Related to
Replaces
License
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Mooney, James. (2022). Classification with Mixture of Experts Models. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/252470.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.