Browsing by Subject "unsupervised learning"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    Tensor Modeling of High-dimensional Distributions and Nonlinear Functions
    (2020-12) Kargas, Nikolaos
    Distribution estimation lies at the heart of statistical learning. Given the probability distribution of a set of random variables it is possible to compute marginal or conditional probabilities, moments and design optimal estimators. In this dissertation, we reveal an interesting link between multivariate statistics and tensors and propose a novel framework for joint Probability Mass Function (PMF) estimation given limited and possibly very incomplete data samples. We show that any joint PMF admits a naive Bayes model representation with a finite-alphabet latent variable. If the latent alphabet size is under a certain threshold, then the joint PMF of an arbitrary number of random variables can be identified from three-dimensional marginal distributions. We develop a practical and efficient algorithm that is shown to work well on both simulated and real data. We also extend our approach to mixture models of continuous variables. We consider the special case of mixture models whose component distributions factor into the product of the associated marginals and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. The second part of the dissertation focuses on the problem of nonlinear function approximation. In practice, when labeled data are available we are often interested in methods that directly model the relationship or the conditional distribution function between the features and the target variable. It is desirable to develop methods that are expressive enough to capture a wide class of functions and at the same time are scalable and efficient. We show that the canonical polyadic decomposition model offers an appealing solution for modeling and learning a general nonlinear function. We formulate the problem as a smooth tensor decomposition problem with missing data and prove that under certain conditions correct nonlinear system identification is possible. We extend our method to multivariate functions of continuous inputs by proposing a generalization of the canonical polyadic decomposition from tensors to multivariate functions. The merits of our approach are illustrated using several synthetic and real multivariate regression tasks.
  • Thumbnail Image
    Item
    Unsupervised Learning of Latent Structure from Linear and Nonlinear Measurements
    (2019-06) Yang, Bo
    The past few decades have seen a rapid expansion of our digital world. While early dwellers of the Internet exchanged simple text messages via email, modern citizens of the digital world conduct a much richer set of activities online: entertainment, banking, booking for restaurants and hotels, just to name a few. In our digitally enriched lives, we not only enjoy great convenience and efficiency, but also leave behind massive amounts of data that offer ample opportunities for improving these digital services, and creating new ones. Meanwhile, technical advancements have facilitated the emergence of new sensors and networks, that can measure, exchange and log data about real world events. These technologies have been applied to many different scenarios, including environmental monitoring, advanced manufacturing, healthcare, and scientific research in physics, chemistry, bio-technology and social science, to name a few. Leveraging the abundant data, learning-based and data-driven methods have become a dominating paradigm across different areas, with data analytics driving many of the recent developments. However, the massive amount of data also bring considerable challenges for analytics. Among them, the collected data are often high-dimensional, with the true knowledge and signal of interest hidden underneath. It is of great importance to reduce data dimension, and transform the data into the right space. In some cases, the data are generated from certain generative models that are identifiable, making it possible to reduce the data back to the original space. In addition, we are often interested in performing some analysis on the data after dimensionality reduction (DR), and it would be helpful to be mindful about these subsequent analysis steps when performing DR, as latent structures can serve as a valuable prior. Based on this reasoning, we develop two methods, one for the linear generative model case, and the other one for the nonlinear case. In a related setting, we study parameter estimation under unknown nonlinear distortion. In this case, the unknown nonlinearity in measurements poses a severe challenge. In practice, various mechanisms can introduce nonlinearity in the measured data. To combat this challenge, we put forth a nonlinear mixture model, which is well-grounded in real world applications. We show that this model is in fact identifiable up to some trivial indeterminancy. We develop an efficient algorithm to recover latent parameters of this model, and confirm the effectiveness of our theory and algorithm via numerical experiments.
  • Thumbnail Image
    Item
    Unsupervised methods to discover events from spatio-temporal data
    (2016-05) Chen, Xi
    Unsupervised event detection in spatio-temporal data aims to autonomously identify when and/or where events occurred with little or no human supervision. It is an active field of research with notable applications in social, Earth, and medical sciences. While event detection has enjoyed tremendous success in many domains, it is still a challenging problem due to the vastness of data points, presence of noise and missing values, the heterogeneous nature of spatio-temporal signals, and the large variety of event types. Unsupervised event detection is a broad and yet open research area. Instead of exploring every aspect in this area, this dissertation focuses on four novel algorithms that covers two types of important events in spatio-temporal data: change-points and moving regions. The first algorithm in this dissertation is the Persistence-Consistency (PC) framework. It is a general framework that can increase the robustness of change-point detection algorithms to noise and outliers. The major advantage of the PC framework is that it can work with most modeling-based change-point detection algorithms and improve their performance without modifying the selected change-point detection algorithm. We use two real-world applications, forest fire detection using a satellite dataset and activity segmentation from a mobile health dataset, to test the effectiveness of this framework. The second and third algorithms in this dissertation are proposed to detect a novel type of change point, which is named as contextual change points. While most existing change points more or less indicate that the time series is different from what it was before, a contextual change point typically suggests an event that causes the relationship of several time series changes. Each of these two algorithms introduces one type of contextual change point and also presents an algorithm to detect the corresponding type of change point. We demonstrate the unique capabilities of these approaches with two applications: event detection in stock market data and forest fire detection using remote sensing data. The final algorithm in this dissertation is a clustering method that discovers a particular type of moving regions (or dynamic spatio-temporal patterns) in noisy, incomplete, and heterogeneous data. This task faces two major challenges: First, the regions (or clusters) are dynamic and may change in size, shape, and statistical properties over time. Second, numerous spatio-temporal data are incomplete, noisy, heterogeneous, and highly variable (over space and time). Our proposed approach fully utilizes the spatial contiguity and temporal similarity in the spatio-temporal data and, hence, can address the above two challenges. We demonstrate the performance of the proposed method on a real-world application of monitoring in-land water bodies on a global scale.