In recent years, there has been a significant increase in the applications dealing with dynamic, high-dimensional, heterogeneous data streams. For example, in the domains such as healthcare, activity recognition, aviation systems, etc. multiple sensors provide a record of many continuous and discrete parameters over long periods of time, and the objective is to monitor behavior of the objects, discover meaningful patterns or detect anomalous events. In spite of a vast literature on data mining and machine learning techniques, these problems have continued to remain difficult. Primarily this is due to a challenge of proper characterization of the interdependencies between multiple data sources, being a mixture of continuous and discrete type. Moreover, for applications that deal with data monitoring or unusual behavior detection, the additional challenge is a design of discovery algorithms aimed at extracting patterns, trends, anomalies in unsupervised settings where data is commonly noisy and even partially unobservable. In this work, we propose a suite of models and methods for the analysis of such data by using a Dynamic Bayesian Network (DBN) representation. DBN is a general tool for establishing dependencies between variables evolving in time, and is used to represent complex stochastic processes to study their properties or make predictions on the future behavior. The main challenge in using DBN is to identify a model structure, learn its parameters with estimation guarantees and perform efficient inference. Our work has made advances in addressing the above problems, especially in the context of anomaly detection, by proposing several frameworks for anomaly detection in multivariate time series data and building efficient algorithms for learning and inference.