Browsing by Subject "tensor decomposition"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Scalable Learning Adaptive to Unknown Dynamics and Graphs(2019-06) Shen, YanningWith the scale of information growing every day, the key challenges in machine learning include the high-dimensionality and sheer volume of feature vectors that may consist of real and categorical data, as well as the speed and the typically streaming format of data acquisition that may also entail outliers and misses. The latter may be present, either unintentionally or intentionally, in order to cope with scalability, privacy, and adversarial behavior. These challenges provide ample opportunities for algorithmic and analytical innovations in online and nonlinear subspace learning approaches. Among the available nonlinear learning tools, those based on kernels have merits that are well documented. However, most rely on a preselected kernel, whose prudent choice presumes task-specific prior information that is generally not available. It is also known that kernel-based methods do not scale well with the size or dimensionality of the data at hand. Besides data science, the urgent need for scalable tools is a core issue also in network science that has recently emerged as a means of collectively understanding the behavior of complex interconnected entities. The rich spectrum of application domains comprises communication, social, financial, gene-regulatory, brain, and power networks, to name a few. Prominent tasks in all network science applications are those of topology identification and inference of nodal processes evolving over graphs. Most contemporary graph-driven inference approaches rely on linear and static models that are simple and tractable, but also presume that the nodal processes are directly observable. To cope with these challenges, the present thesis first introduces a novel online categorical subspace learning approach to track the latent structure of categorical data `on the fly.' Leveraging the random feature approximation, it then develops an adaptive online multi-kernel learning approach (termed AdaRaker), which accounts not only for data-driven learning of the kernel combination, but also for the unknown dynamics. Performance analysis is provided in terms of both static and dynamic regrets to quantify the novel learning function approximation. In addition, the thesis introduces a kernel-based topology identification approach that can even account for nonlinear dependencies among nodes and across time. To cope with nodal processes that may not be directly observable in certain applications, tensor-based algorithms that leverage piecewise stationary statistics of nodal processes are developed, and pertinent identifiability conditions are established. To facilitate real-time operation and inference of time-varying networks, an adaptive tensor decomposition based scheme is put forth to track the topologies of time-varying networks. Last but not least, the present thesis offers a unifying framework to deal with various learning tasks over possibly dynamic networks. These tasks include dimensionality reduction, classification, and clustering. Tests on both synthetic and real datasets from the aforementioned application domains are carried out to showcase the effectiveness of the novel algorithms throughout.Item Tensor Modeling of High-dimensional Distributions and Nonlinear Functions(2020-12) Kargas, NikolaosDistribution estimation lies at the heart of statistical learning. Given the probability distribution of a set of random variables it is possible to compute marginal or conditional probabilities, moments and design optimal estimators. In this dissertation, we reveal an interesting link between multivariate statistics and tensors and propose a novel framework for joint Probability Mass Function (PMF) estimation given limited and possibly very incomplete data samples. We show that any joint PMF admits a naive Bayes model representation with a finite-alphabet latent variable. If the latent alphabet size is under a certain threshold, then the joint PMF of an arbitrary number of random variables can be identified from three-dimensional marginal distributions. We develop a practical and efficient algorithm that is shown to work well on both simulated and real data. We also extend our approach to mixture models of continuous variables. We consider the special case of mixture models whose component distributions factor into the product of the associated marginals and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. The second part of the dissertation focuses on the problem of nonlinear function approximation. In practice, when labeled data are available we are often interested in methods that directly model the relationship or the conditional distribution function between the features and the target variable. It is desirable to develop methods that are expressive enough to capture a wide class of functions and at the same time are scalable and efficient. We show that the canonical polyadic decomposition model offers an appealing solution for modeling and learning a general nonlinear function. We formulate the problem as a smooth tensor decomposition problem with missing data and prove that under certain conditions correct nonlinear system identification is possible. We extend our method to multivariate functions of continuous inputs by proposing a generalization of the canonical polyadic decomposition from tensors to multivariate functions. The merits of our approach are illustrated using several synthetic and real multivariate regression tasks.