The Analysis of Multimedia through the Lens of Music Theory: A Model for Analyzing the Interaction of Auditory, Visual, and Narrative Stimuli in Multimediatic Artforms

Abstract

Scholars often separate the study of multimedia into distinct fields based on the platform of distribution: films and television, video games, staged performances (such as opera, ballet, and theater), and so on. While such fragmentation can provide a clear focus for research, it also prevents communication across the scholarly divide. In this dissertation, I suggest a model for analyzing narrative multimedia applicable across genres and platforms. Synchronization schemas account for the combination of sonic, visual, and narrative elements through the shared metric of intensity. I explore the interaction of these various modalities by focusing on a particular synchronization schema I call the plunge—that moment when a narrative agent makes a risky and definitive choice, which can range from minor dramatic moments to piece-defining climaxes. Comparing instances of the plunge in television, film, opera, ballet, and video games demonstrates remarkable consistency in its presentation despite shifting genre conventions and styles. I situate my work within the existing scholarship on musical topics and schemas to suggest expanding their purviews to include multimedia works. After a review of this literature, I present a method of analysis and define a variety of plunge sub-types with examples from filmic, staged, and ludic media. A focused study of the synchronization schema in the animated film Spider-Man: Into the Spider-Verse (2018) highlights the applicability of the proposed method. By generalizing narrative moments such as the plunge, underlying norms and deviations can be identified and analyzed, allowing for more nuanced engagement among scholars of multimedia.