Towards an Effective Organization-Wide Bulk Email System

Abstract

Bulk email (emails sent to a large list of recipients) is widely used in organizations to communicate messages to employees. It is an important tool in making employees aware of policies, events, leadership updates, etc. However, in large organizations, the problem of overwhelming communication is widespread. Ineffective organizational bulk emails waste employees’ time and organizations’ money, and cause a lack of awareness or compliance with organizations’ missions and priorities. Prior research mainly studied commercial bulk emails from a single stakeholder’s perspective, such as helping senders improve open rates or helping recipients filter unsolicited bulk emails. However, within organizations, bulk email communication involves multiple stakeholders (employees, communicators, managers, leaders, the organization itself, etc.) with different priorities. The goal of organizational bulk email system is to both reach organization-wide communication effectiveness and provide positive experiences for all the stakeholders. This thesis focuses on understanding and improving organizational bulk email systems by 1) conducting qualitative research to understand different stakeholders’ perceptions of the system and its current effectiveness; 2) proposing economic models to describe stakeholders’ actions/cost/value; 3) conducting field studies to evaluate personalization methods’ effects on getting employees to read bulk messages; 4) designing tools to support communicators in evaluating, designing, and targeting bulk emails. We performed these studies at the University of Minnesota, interviewing 25 employees (both senders and recipients), and including 317 participants in our studies in total. We found that the university's current organizational bulk email system is ineffective as only 22% of the information communicated was retained by employees. The failure of this system was systemic — it had many stakeholders, but none of them necessarily had a global view of the system or the impacts of their own actions. Then to encourage employees to read high-level information, we implemented a multi-stakeholder personalization framework that mixed important-to-organization messages with employee-preferred messages and improved the studied bulk email's recognition rate by 20%. On the sender side, we iteratively designed and deployed a prototype of an organizational bulk email evaluation platform (CommTool). In field evaluation, we found several features (such as bulk emails' message-level performance) of CommTool helped communicators in designing bulk emails. At the same time, to enable these message-level metrics, we collected ground-truth eye-tracking data and developed a novel neural network technique to estimate how much time each message is being read using recipients' interactions with browsers only, which improved the estimation accuracy from 54% (heuristics) to 73%. In summary, this work sheds light on how to design organizational bulk email systems that communicate effectively and respect different stakeholders' value.