Exploring Evaluation Capacity Building in Community-Based Health Organizations in India: What Works and Why

Abstract

This research explored evaluation capacity building (ECB) needs and strategies, as well as facilitators and barriers experienced by program staff and independent evaluators working with community-based health organizations (CBHOs) in India. It suggests a significant shift from donor-led to CBHO-led evaluation practice in the community health sector of India. CBHOs, however, have limited access to evaluation training, resources, and macro-level evaluation infrastructure. The overall purpose of the study was to understand how CBHOs in India build their capacity to conduct evaluations and use evaluation results in guiding program planning and improvement. This was explored with the following questions: 1) what are the internal and external contexts driving the need to engage in ECB? 2) what strategies and approaches are viewed by the organizations as most important in bringing about evaluation capacity in Indian CBHOs? and 3) what, according to the leadership of these organizations, enables or obstructs the process of ECB? To answer these questions, the study conducted semi-structured, in-depth interviews with twenty-three (23) CBHO employees and seven (7) independent evaluators. The interview data were analyzed using constant comparative analysis, which aided in identifying themes across the data. The results of the analysis demonstrate that CBHOs build their evaluation capacity by embedding ECB in program structures, facilitating on-the-job training, and building a support network of evaluation experts to guide their ECB efforts. The study also observed that organizations committed to developing evaluation capacity are intentional about organizational learning and seek to establish peer learning structures and create learning opportunities for their employees. In addition, senior leadership in such organizations demonstrates support of evaluation by facilitating on-the-job capacity building, securing adequate funds for evaluation, and creating a democratic work culture that encourages critical thinking. On the other hand, short project timelines, shifting donor priorities, limited government interest in following evidence-informed policy, and resource constraints hinder the use of evaluation and negatively affect organizations’ interest in ECB. The senior-level program staff indicated a need to build critical thinking capacities at CBHOs to enhance ECB processes. Entry-level staff expressed their interest in learning evaluation skills and methods to implement internal evaluations. The experiences of grassroots health workers suggest limited access to evaluation training, resources, and support structures. The availability of evaluation training seminars, resource persons, and other support material is limited at regional locations and in regional languages, highlighting an urgent need for decentralized and culturally responsive ECB interventions. This research adds to the literature by identifying needs, strategies, and both facilitating and impeding factors for developing evaluation capacity at CBHOs. CBHOs can use this information to design and implement ECB interventions. The study can be helpful for government, institutes of higher education, and international donor and development organizations as they create ECB resources and provide effective support to grassroots ECB initiatives.