Data-Driven Exploratory Interfaces for Contextualizing Parameter Spaces: Adding Intuition to Big Data

Abstract

This dissertation investigates how to understand the complexities of large parameter spaces through user interaction. Both researchers and practitioners have embraced the hope of using Big Data (extremely large, heterogeneous, and unstructured datasets) to solve complex problems using statistical methods like artificial intelligence or machine learning. Unfortunately, for many disciplines (e.g. the scientific fields), these computational algorithms are black boxes that provide answers without the important explanations required for developing hypotheses. Interactive visualization offers the promise of adding intuition to Big Data, however, current approaches do not scale to the large data sizes, high-dimensionality, and ill-defined complexity. In order to address these challenges, we introduce Data-Driven Exploratory Interfaces (DDEIs), interfaces that are scalable, enable contextual navigation, and use meaningful feature interaction for intuitive exploration. Using DDEIs, we analyze Big Data in the scientific context using three separate applications, each focusing on a different ``Big'' aspect of Big Data. These include medical device design (memory intensive, spatially complex), shock physics (high-dimensional, many instances), and cell migration (memory intensive, spatially complex, high-dimensional, and many instances). In each case, we first look at traditional methods for visualizing and understanding these large datasets, then we overcome the limitations with key concepts introduced by DDEIs (specifically enabling users to define the contexts with respect to their questions, and then explore the relevant trends in the parameter space). In addition to studying the ``Big'' aspects of Big Data (Challenge 1), in parallel, we study two other challenges (Challenge 2 and Challenge 3) relevant to understanding Big Data. The second challenge involves investigating approaches to solving the high-dimensional sparsity problems (Challenge 2). Here we use prediction and sampling strategies to analyze the gaps in knowledge. In the third challenge, we consider how to use interaction to navigate and understand ill-defined features in the data (Challenge 3). Using DDEIs, we combine natural exploration techniques with data-driven algorithms to understand application specific features across parameter space sampling properties (input, output, local, global). In this dissertation we show that DDEIs can overcome the limitations of traditional visualization approaches while creating new intuitive views into otherwise ill-defined and complex phenomena.