Kargas, Nikolaos2021-02-222021-02-222020-12https://hdl.handle.net/11299/218706University of Minnesota Ph.D. dissertation. December 2020. Major: Electrical/Computer Engineering. Advisor: Nicholas Sidiropoulos. 1 computer file (PDF); x, 106 pages.Distribution estimation lies at the heart of statistical learning. Given the probability distribution of a set of random variables it is possible to compute marginal or conditional probabilities, moments and design optimal estimators. In this dissertation, we reveal an interesting link between multivariate statistics and tensors and propose a novel framework for joint Probability Mass Function (PMF) estimation given limited and possibly very incomplete data samples. We show that any joint PMF admits a naive Bayes model representation with a finite-alphabet latent variable. If the latent alphabet size is under a certain threshold, then the joint PMF of an arbitrary number of random variables can be identified from three-dimensional marginal distributions. We develop a practical and efficient algorithm that is shown to work well on both simulated and real data. We also extend our approach to mixture models of continuous variables. We consider the special case of mixture models whose component distributions factor into the product of the associated marginals and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. The second part of the dissertation focuses on the problem of nonlinear function approximation. In practice, when labeled data are available we are often interested in methods that directly model the relationship or the conditional distribution function between the features and the target variable. It is desirable to develop methods that are expressive enough to capture a wide class of functions and at the same time are scalable and efficient. We show that the canonical polyadic decomposition model offers an appealing solution for modeling and learning a general nonlinear function. We formulate the problem as a smooth tensor decomposition problem with missing data and prove that under certain conditions correct nonlinear system identification is possible. We extend our method to multivariate functions of continuous inputs by proposing a generalization of the canonical polyadic decomposition from tensors to multivariate functions. The merits of our approach are illustrated using several synthetic and real multivariate regression tasks.endensity estimationfunction approximationmachine learningsupervised learningtensor decompositionunsupervised learningThesis or Dissertation