Quality of Service for Continuous Media Metrics, Validation, Implementation and Performance Evaluation

Abstract

Multimedia, delivered for human consumption consist of a collection of media streams delivered in a cohesive and comprehensible manner. A methodology for designing multimedia systems modeling some aspect of the real world, designing Quality of Service (QoS) metrics to measure its quality, performing user studies to validate them, evaluating existing systems on validated metrics, and if found deficient, improving them or redesigning new systems. Choosing to model the aspect of the real world as lossy continuous media, this dissertation consists developing metrics to model intra-stream continuity and interstream synchronization of lossy continuous media streams, validating them through user experiments, and evaluating the Berkeley Continuous Media Toolkit (GMT) on proposed metrics, and designing improvements to CMT to make it adhere to programmer specified QoS metrics. Proposed metrics specify continuity and synchronization, with tolerable limits on average and bursty defaults from perfect continuity, timing and synchronization constraints. Continuity specification of a CM stream consists of its sequencing, display rate and drift profiles. The sequencing profile of a CM stream consists of tolerable aggregate and consecutive frame miss ratios. Rate profiles specify the average rendition rate and its variation. Given a rate profile, the ideal time unit for frame display is determined as an offset from the beginning of the stream. Drift profile specifies the average and bursty deviation of schedules for frames from such fixed points in time. Synchronization requirements of a collection of CM streams are specified by mixing, rate and synchronization drift profiles. Mixing profiles specify vectors of frames that can be displayed simultaneously. They consist of average and bursty losses of synchronization. Rate profiles consist of average rates and permissible deviations thereof. Synchronization drift profiles specify permissible aggregate and bursty time drifts between schedules of simultaneously displayable frames. It is shown that rate profiles of a collection of synchronized streams is definable in terms of rate profiles of its component streams. It is also shown that mixing and drift profiles of a collection of streams are non-definable in terms of sequencing and drift profiles of its constituents. An important consequence of the mutual independence of synchronization and continuity specification is that, in a general purpose platform with limited resources, synchronized display of CM streams may require QoS tradeoffs. An algori thm that makes such tradeoffs is presented as a proof of applicability of our metrics in a realistic environment. The proposed metrics were validated by means of a user survey. It consisted of presenting user with a series of professionally edited CM clips with controlled defects and obtain their opinion by means of Likert and imperceptible/tolerable/annoying scale. Viewer discontent for aggregate video losses gradually increases with the amount of loss. We concluded that 17 /100 to 23/100 average video losses are tolerated, and above 23/100 is unacceptable. Furthermore, as observed, a consecutive video loss of about two video frames in 100 does not cause user dissatisfaction. Although losing two consecutive video frames is noticed by most users, once this threshold is reached there is not much room for quality degradation due to consecutive losses. This figure for audio is 3 frames. Our results indicate that even a 20% rate variation in a newscast type video does not result is significant user dissatisfaction. The situation of audio rate variations are much more different. Even about 5% rate variation in audio is noticed by most observers. Our results also indicate that at aggregate audio-video synchronization loss of about 20/lO0n human tolerance plateaus out. This figure is about 3 frames for consecutive audio-video synchronization loss. For the performance evaluation part, losses and timing drift in stream continuity and synchronization were measured in the presence of processor and network loads. For stream continuity it was observed that increasing loads at lower frame rates significantly increases the aggregate frame drops, and similarly at higher rates, increasing loads at higher frame rates significantly increases consecutive frame drops. Because at a higher rates a large number of consecutive frames are dropped, the ones that are played appear in a more timely manner. For synchronization losses, it was shown that according to Steinmetz' metric CMT provides imperceptible audio-video mis-synchronization for about 10 seconds, and tolerable synchronization for about 13 seconds from the start of the clips for local clients under low processor loads. It is also shown that under high loads, synchronization is achieved at the cost of losing media frames. In order to control continuity and synchronization losses we propose four solutions. Firstly, to control losses, objects processing CM streams in CMT to be made QoS aware, in the sense that delaying vs. dropping frames be based on user specified QoS parameters. Secondly, in order to achieve QoS based synchronization a new paradigm Stream Groups, where grouped objects simultaneously fetch, transport and render corresponding streams from respective constituent streams. Thirdly, to introduce buffers at client sites, where servers can fill them up and clients run independent of servers. Fourthly, to have a feed-back mechanism from clients to control their respective servers.