Domain-knowledge-guided machine learning for networked systems.

Abstract

Advances in networked systems, such as 5G networks and smart grids, are set to seamlessly connect the cyber and physical worlds, and fundamentally transform our lives. Machine learning (ML), with its powerful predictive capabilities, is promising in addressing the challenges posed by the complexity and dynamics of networked systems. While ML has demonstrated success in domains related to human perception, its application to networked systems requires meticulous design, because networked systems exhibit inherent correlation, interdependence, and adherence to principles and constraints. Applying black-box ML techniques to networked systems without considering these aspects may limit success, and ML methods not conforming to system constraints may lead to faulty designs. This dissertation focuses on fully harnessing ML to intelligently manage, model, and enhance networked systems by integrating domain knowledge with ML and aligning ML with system constraints. To this end, we focus on three major lines of research. First, to model partially observed power grids, we incorporate physical laws of power systems, such as Kirchhoff's laws and system topology, into neural networks to solve the power flow problem, given that system models can be inaccurate or unavailable. To the best of our knowledge, this is the first work of integrating physical knowledge into data-driven methods within the domain of power grids and our physics-guided neural networks consistently achieve superior accuracy and generalizability compared to existing unconstrained data-driven approaches. Second, to adapt caching policies to dynamic and uncertain user requests, we integrate the theoretical insights of Belady's MIN algorithm with the predictive capabilities of generative neural networks to develop a deep learning-based caching algorithm that approximates the theoretical optimum. Our approach significantly reduces latency, enhancing system and network responsiveness, while also optimizing bandwidth utilization by minimizing redundant data transmissions. Third, to improve the quality of experience in video streaming over 5G networks, we develop a neural network-based Multiple Description Video Codec and a multi-path, multi-stream streaming system. This jointly co-designing video codec and multipath video streaming using MDC effectively utilizes the abundant 5G channel resources while mitigating the challenges posed by wildly fluctuating 5G throughput and unstable, noisy 5G channel conditions.