Between Dec 19, 2024 and Jan 2, 2025, datasets can be submitted to DRUM but will not be processed until after the break. Staff will not be available to answer email during this period, and will not be able to provide DOIs until after Jan 2. If you are in need of a DOI during this period, consider Dryad or OpenICPSR. Submission responses to the UDC may also be delayed during this time.
 

Sheaf Representation Learning

Loading...
Thumbnail Image

Persistent link to this item

Statistics
View Statistics

Journal Title

Journal ISSN

Volume Title

Title

Sheaf Representation Learning

Published Date

2023-12

Publisher

Type

Thesis or Dissertation

Abstract

The observed behaviors of a wide variety of processes are implicitly supported by a collection of spatial or relational constraints placed upon regions of the domain which generate the data under study. Often modeled discretely as a network-structured or combinatorial object, the topology dictating the pattern with which these domain elements interact influences the global character of these systems by restricting the space of possible data observations and biasing behavior to abide by the local topological constraints inherited by the global configuration. In short, the topology of a domain constrains the structure of its associated data. To properly and efficiently infer from data the parameters of functions which describe these processes, machine learning models must be imbued with structural inductive biases which align the function’s representations with the constraints imposed by the domain topology. Despite this crucial relationship between domain topology and observed data, a general theoretical framework for specifying and reasoning about these constraints within machine learning is largely lacking. This thesis provides a sheaf-theoretic framework for specifying and reasoning about model representational constraints imposed by the topology of the domain. Central to this framework is the observation that associating data to a topological space, a common modeling motif in modern machine learning, is an inherently functorial operation, inviting the reinterpretation of a fundamental machine learning paradigm in a category-theoretic language. Through this more general language, we gain the capacity to reason about model classes which satisfy the compositional properties imposed by the functorial association of data to topology. By specifying this functor to be a sheaf, the representational constraints implied by a cover on the domain may be operationalized by requiring functionally-consistent data assignments along intersecting regions of the domain. When the representation target of this functor is vector-valued, as is typically the case in machine learning, this sheaf-theoretic approach admits a linear-algebraic implementation. While this sheaf-theoretic approach to topologically-constrained representation learning inherits significant modeling power from its abstract formulation, we show that this approach integrates with and properly generalizes a number of existing graph embedding and signal processing algorithms by specifying simplistic assumptions regarding the complexity of the representational constraints imposed by the domain. In the graph signal processing setting, we introduce sheaf neural networks, a generalization of graph neural networks which provides better modeling expressivity, especially when node interactions imply functional interactions among topologically-connected signals. This sheaf-theoretic generalization of graph neural networks also clarifies, in a categorical language, the value of more complex relational inductive biases in terms of expressivity and alignment with domain-implied constraints. We also introduce sheaf embedding: an approach to encoding relational data in a new representation which permits the specification of complex typing schematics within the domain interactions while retaining global consistency objectives. We find this abstraction particularly apt within the knowledge graph reasoning domain, where these knowledge sheaves generalize a number of widely-applied knowledge graph embedding methods and, through their spectral properties, admit linear-algebraic extensions for both logical andinductive query reasoning.

Description

University of Minnesota Ph.D. dissertation. December 2023. Major: Computer Science. Advisor: Paul Schrater. 1 computer file (PDF); x, 151 pages.

Related to

Replaces

License

Collections

Series/Report Number

Funding information

Isbn identifier

Doi identifier

Previously Published Citation

Other identifiers

Suggested citation

Gebhart, Thomas. (2023). Sheaf Representation Learning. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/260630.

Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.