Exploring the Ecology of Evaluation Contract Work in the United States: Implications for Industry

Abstract

While evaluators have provided goods and services to U.S. states and the federal government for decades, little is known about how the market’s demand-driven nature impacts evaluation practice. This study explored the likelihood evaluation firms and universities acquired newly funded evaluation-specific contracts from the U.S. Department of Health and Human Services (HHS) each fiscal year (FY) between FY 2008-2022, some of the factors that influenced the likelihood a firm or university acquired new HHS evaluation contracts, external research and evaluation providers’ perceptions of the federal evaluation contracts landscape, and the ways external research and evaluation providers positioned themselves within the federal evaluation market to compete for resources. Contracts data from USAspending.gov and semi-structured interviews with 11 practicing evaluators and researchers were used to explicate the demand-driven nature of external evaluation contract work in the U.S. Contracts data focused on the composition of evaluators (i.e., organizational size, type, niche) and length of time in the HHS arena between FY 2008-2022. Interview data focused on practicing evaluators’ perceptions of external environmental factors in the market—including changes in presidential administrations, the emergence of the COVID-19 pandemic, and George Floyd’s murder—in relation to their practice. Study results suggest that not small firms are more likely than small firms or universities to acquire new HHS contracts. Interviewee perceptions did not necessarily align with the literature; a few evaluators, all specializing in disability and special education-related work, described how changes in presidential administrations did not have a major impact on the number or types of federal contracts made available over the years. Discussions on the increase in work related to economic, health, and racial disparities; experiences with demand-side barriers to embedding diversity, equity, and inclusion in research and evaluation; their organization’s niche, size, and type; and the need to build internal organizational capacity were all framed in the context of needing to be responsive to funder (i.e., state and federal government) demands. Overall, this research provides a glimpse into the marketplace conditions and structures for contractual evaluative work in the United States and the implications such structure poses for the evaluation industry.