Provably Learning From Data: New Algorithms And Models For Matrix And Tensor Decompositions

Abstract

Learning and leveraging patterns in data has fueled the recent advances in data driven services. As these solutions become more ubiquitous, and get incorporated into critical applications in healthcare and transportation, there is an increasing need to understand the limits of these learning algorithms and to develop algorithms with guarantees. Moreover, with data being generated at unprecedented rates, these algorithms need to be fast, learn on-the-fly (online), handle large volumes of data (scalable), and be computationally efficient, while possessing guarantees on their behavior. Furthermore, to make the learning-based products widely applicable there is also a need to make their reasoning and decision making process transparent (interpretable). These challenges inspire and motivate this dissertation. Specifically, we focus on analyzing various matrix/tensor demixing and factorization tasks, where we leverage the inherent interpretability endowed by the structure of problem (such as sparsity and low-rankness) to characterize the (theoretical) conditions for successful recovery, and analyze their performance in real-world settings. To this end, we make contributions on three fronts. First, we develop algorithm-aware theoretical guarantees for sparse matrix and tensor factorization tasks. Second, we establish algorithm-agnostic theoretical results for matrix demixing models and demonstrate their applications on real-world datasets. Lastly, we develop application-specific techniques for navigation and source separation. Bringing together Algorithms, Theory, and Applications, the techniques and theoretical results developed as part of this dissertation facilitate and motivate future explorations into the inner workings of learning algorithms for their safe use in critical applications.