Scalable Kernel Learning, Tensors in Community Identification, and Robust Adversary Detection in Deep Neural Networks

Abstract

The presence of ubiquitous sensors continuously recording massive amounts of information has lead to an unprecedented data collection, whose exploitation is expected to bring about scientific and social advancements in everyday lives. Along with the ever-increasing amount of data, incredible progress in the fields of Machine Learning, Pattern Recognition, and Optimization has also contributed to the growing expectations. Such progress however, has also brought to light certain limitations in state-of-the-art learning machines, manifesting the roadblocks in the research path ahead. For instance, in addition to practical considerations pertaining to non-stationary, noisy and unsupervised settings, various applications often run on limited memory and stringent computational resources, thus requiring efficient and light-weight algorithms to cope with extreme volumes. Furthermore, certain characteristics such as presence of outliers or adversaries as well as the complex nature of real-world interactions call for robust algorithms, whose performance will be resilient in the face of deviations from nominal settings. The present thesis contributes to learning over unsupervised, complex, and adversarial data. Emphasis is laid on concocting online, scalable and robust algorithms, enabling streaming analytics of sequential measurements based on vector, matrix, and tensor-based views of supervised and unsupervised learning tasks. For online and scalable learning, a novel kernel-based feature extraction framework is put forth, in which limited memory and computational resources are accounted for via maintaining an affordable \emph{budget}. Furthermore, complex interactions of real-world networks are analyzed from a community identification point-of-view, in which a novel tensor-based representation along with provable optimization techniques robustify state-of-the-art alternatives. Finally, the performance of deep convolutional neural network based image classifiers is investigated when adversaries disturbing input images are modeled as imperceptible yet carefully-crafted perturbations. To this end, a general class of high-performance Bayesian detectors of adversaries is developed. Extensive experimentation on synthetic as well as numerous real datasets demonstrates the effectiveness, interpretability and scalability of the proposed learning, identification, and detection algorithms. More importantly, the process of design and experimentation sheds light on the behavior of different methods and the peculiarities of real-world data, while at the same time it generates new ideas and directions to be explored.