One of the foremost intellectual challenges of this century is to understand the collective behavior of complex systems. Such systems are ubiquitous, and range from ``engineered systems'' including the Internet and online social grids, to complex natural phenomena such as neural connections in the brain, and interactions between genes. Networks lie at the heart of complex systems, encoding pairwise interactions between their constituent components. In this regard, complexity captures the fact that it is difficult to derive holistic system behavior from knowledge of individual components. The key premise of network science is that despite the diversity of complex systems, the behavior of their underlying networks is driven by a common set of laws. Contemporary studies focus on models and tools to understand, predict, and control the behavior of networks. However, most of these approaches are tailored to analysis of static networks, whose node and link structure does not change with time. Cognizant of the dynamic nature of most real-world networks, analysts mostly focus on static snapshots or aggregate views of studied systems, and meaningful insights cannot be guaranteed. Indeed, the recently growing trend in analysis of dynamic networks is testament to the critical need to live up to this challenge. Moreover, issues arising from temporal network evolution are exacerbated by inherent Big Data challenges. Many large-scale networks comprise billions of nodes, which are typically associated with high-dimensional, and streaming features. Furthermore, it is often impractical to observe the entire network, and analyses must be conducted on manageable or easily accessible samples of the network. Acknowledging these limitations, this dissertation leverages recent advances in statistical signal processing, optimization, and machine learning to address the aforementioned challenges. Emphasis is placed on statistical learning approaches capable of exploiting sparsity, or low rank, attributes that have been shown useful for complexity reduction. Focusing on canonical network inference tasks such as topology identification, detection of communities, and unveiling anomalous nodes, this dissertation puts forth novel statistical models, and develops efficient algorithms for dynamic network analytics. Motivated by the need for real-time processing, online renditions of the developed algorithms are advocated for handling streaming network data. For each of the research themes considered, extensive tests are conducted on simulated and real data, while pertinent comparisons with competing approaches are drawn wherever possible.