Managing Data Quality in Observational Citizen Science

Abstract

Observational citizen science is an effective way to supplement the environmental datasets compiled by professional scientists. Involving volunteers in data collection has the added educational benefits of increased scientific awareness and local ownership of environmental concerns. This thesis provides an in-depth exploration of observational citizen science and the associated challenges and opportunities for HCI research. We focus on data quality as a key lens for understanding observational citizen science, and how it differs from the related domains of crowdsourcing, open collaboration, and volunteered geographic information. In order to understand data quality, we performed a qualitative analysis of data quality assurance practices in River Watch, a regional water quality monitoring program. We found that data quality in River Watch is primarily maintained through universal adherence to standard operating procedures, rather than through a computable notion of “accuracy”. We also found that rigorous data quality assurance practices appear to enhance rather than hinder the educational goals of the program participants. In order to measure data quality, we conducted a quantitative analysis of CoCoRaHS, a multinational citizen science project for observing precipitation. Given the importance of long-term participation to data consumers, we focused on volunteer retention as our primary metric for data quality. Through survival analysis, we found that participant age is a significant predictor of retention. Compared to all other age groups, participants aged 60-70 are much more likely to sign up for CoCoRaHS, and to remain active for several years. We propose that the nature of the task can profoundly influence the types of participants attracted to a project. In order to improve data quality, we derived a general workflow model for observational citizen science, drawing on our findings in River Watch, CoCoRaHS, and similar programs. We propose a data model for preserving provenance metadata that allows for ongoing data exchange between disparate technical systems and participant skill levels. We conclude with general principles that should be taken into consideration when designing systems and protocols for managing citizen science data.