Maintaining the efficiency of open production systems at scale: a case study of Wikipedia

Abstract

This dissertation represents an exploration of the function and failures of critical subsystems in open production communities with Wikipedia as a case study. Specifically, I explore the nature of rejection via Wikipedia's informal, post-hoc quality control system and identify a consistent ownership bias that undermines Wikipedia's ethos of openness. I also quantify an inherent trade-off between the speed and efficiency of quality control in Wikipedia and the motivation of rejected contributors -- especially new editors. I then proceed to show how Wikipedia's shifting focus on quality control and formal process has led to a dramatic decline in the rate of retention of desirable new editors that threatens the long-term viability of the project.In light of these results, I present studies of two experimental software systems intended to explore potential solutions to this steady decrease in participation. First I draw on social learning theory to evaluate the effectiveness of a new mode of peripheral participation through reader-submitted feedback. I experimentally demonstrate effective strategies for increasing the rate of contributions without decreasing quality and argue for efficient moderation support in order to make quality control worth volunteer time spent away from editing the encyclopedia. Next, I describe the design and three month field study of a new intelligent software system intended to both efficiently support socialization practices in Wikipedia and bring visibility to the systemic problems that lead to declining newcomer retention. I show evidence that the system works in both regards: critical newcomer socialization activities are made dramatically more efficient and users of the system reflect openly on the breakdowns in Wikipedia's quality control processes.This work has already had impact within the Wikipedia community and in directing the strategy employed by the Wikimedia Foundation in designing and evaluating new software for Wikipedia editors.