Browsing by Subject "Evaluation Capacity Building"

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Building a Fractal Evaluation: A Study on Improving Skills and Attitudes towards Evaluation Through a Design Similar to Program Pedagogy
    (2015-05) Taylor, Anika
    This thesis is a study on using an evaluation model that mimics the educational pedagogy of an organization as an approach to evaluation capacity building. Referred to as a fractal evaluation, this model of using a "self-similar"� approach to learning in an organization sought to improve in staff both their skills in and attitudes around evaluation, or their Evaluation Assets. After the implementation of this model in the education department of a small science museum, interviews and surveys were conducted with staff to determine the effectiveness of using this approach to improve their attitudes and mindsets around the practice of evaluation. The results of the study suggest that staff attitudes and interest in the topic both improved notably along with an expressed increase in staff investment in the process and desire for ongoing learning about program evaluation.
  • Thumbnail Image
    Item
    Building Evaluation Capacity in Educational Organizations: A Longitudinal Case Study of One Metropolitan School District
    (2016-07) Rohmer-Hirt, Johnna
    Although evaluation capacity building (ECB) is relatively prevalent in the literature, there is not an extensive empirical body of work related to the topic. The purpose of this study was to add to the existing literature through a 30-year historical longitudinal case study of a large metropolitan school district from the perspective of its leadership. This research examined the longitudinal evidence of the context, processes, and activities that impact ECB within a K-12 educational setting. In addition, it traced the development of an internal evaluation department and the steps to build supports for data use and evaluation in the midst of accountability, using process use as the primary strategy toward ECB. The study consisted of interviews with 14 district leaders, along with document reviews over the years 1985 to 2015, encompassing significant legislative enactments such as No Child Left Behind (NCLB) and the public accountability era in K-12 education. The district leadership was found to value data use across time, along with community involvement and input, and demonstrated a strong legislative awareness and presence. The leaders who were interviewed also viewed evaluation as mission and vision critical across time. Substantial changes were made over the 30-year time frame in the number and types of staff members in the department, the internal process and activities of the department, how the department collaborated with others across the organization, and the organizational reporting structure. In addition, organizational factors such as the shift from evaluation for mandatory reasons (such as grants and legislative requirements) to evaluation for internal decision making and inquiry occurred over the 30-year span, along with important changes in technology and data integrity, accessibility, and understandability. Eight key recommendations to propel ECB emerged from the research: 1) attend to data integrity, accessibility, and understandability – including technology; 2) capitalize on one-time and special funding; 3) recognize that leadership matters and change is disruptive; 4) hire the right people; 5) remember that disposition/personality matters; 6) make ECB an intentional, ongoing focus; 7) work toward consistency, not the new next best thing; and 8) build on previous work instead of completely starting over.
  • Thumbnail Image
    Item
    Evaluation Capacity Building Towards Developing a Special Education System in a Large Midwestern School District
    (2016-06) Sterner Sampers, Camille
    Abstract This study researched the continued practice of evaluation capacity building (ECB) in an educational context, specifically the special education department of a large Midwestern school district. This descriptive and exploratory study studied the activities, processes, and outcomes of ECB that followed King’s case study (2002) fifteen years later. The Two-County School District was very large (e.g., geography, number of schools, student enrollment) and, as a result, had extensive resources for ECB, which included access to evaluation experts and data tools for sophisticated data use. The breadth and depth of the resources in this large school district put them in a positive place, where the Director of Special Education and other special education evaluation champions did the work of ECB, using the expanded and modified program evaluation model and process that Campbell initiated in 1999 to 2001. Galles, the district’s evaluation leader since 2007, contributed to ECB as the district’s highly skilled evaluation leader. Her evaluation leadership and expertise as well as the technical assistance from the Department of Research, Evaluation, and Assessment positively supported the special education department’s ECB projects. Taylor, the Director of Special Education (2009 – 2013), and several special education evaluation champions demonstrated a strong ECB vision and commitment to achieve change and improve the quality of special education programs. An evaluation leader, committed evaluation champions at multiple levels, dedicated resources, and clear communication are critical to lead and sustain ECB efforts.
  • Thumbnail Image
    Item
    Exploring Evaluation Capacity Building in Community-Based Health Organizations in India: What Works and Why
    (2021-06) Dighe, Satlaj
    This research explored evaluation capacity building (ECB) needs and strategies, as well as facilitators and barriers experienced by program staff and independent evaluators working with community-based health organizations (CBHOs) in India. It suggests a significant shift from donor-led to CBHO-led evaluation practice in the community health sector of India. CBHOs, however, have limited access to evaluation training, resources, and macro-level evaluation infrastructure. The overall purpose of the study was to understand how CBHOs in India build their capacity to conduct evaluations and use evaluation results in guiding program planning and improvement. This was explored with the following questions: 1) what are the internal and external contexts driving the need to engage in ECB? 2) what strategies and approaches are viewed by the organizations as most important in bringing about evaluation capacity in Indian CBHOs? and 3) what, according to the leadership of these organizations, enables or obstructs the process of ECB? To answer these questions, the study conducted semi-structured, in-depth interviews with twenty-three (23) CBHO employees and seven (7) independent evaluators. The interview data were analyzed using constant comparative analysis, which aided in identifying themes across the data. The results of the analysis demonstrate that CBHOs build their evaluation capacity by embedding ECB in program structures, facilitating on-the-job training, and building a support network of evaluation experts to guide their ECB efforts. The study also observed that organizations committed to developing evaluation capacity are intentional about organizational learning and seek to establish peer learning structures and create learning opportunities for their employees. In addition, senior leadership in such organizations demonstrates support of evaluation by facilitating on-the-job capacity building, securing adequate funds for evaluation, and creating a democratic work culture that encourages critical thinking. On the other hand, short project timelines, shifting donor priorities, limited government interest in following evidence-informed policy, and resource constraints hinder the use of evaluation and negatively affect organizations’ interest in ECB. The senior-level program staff indicated a need to build critical thinking capacities at CBHOs to enhance ECB processes. Entry-level staff expressed their interest in learning evaluation skills and methods to implement internal evaluations. The experiences of grassroots health workers suggest limited access to evaluation training, resources, and support structures. The availability of evaluation training seminars, resource persons, and other support material is limited at regional locations and in regional languages, highlighting an urgent need for decentralized and culturally responsive ECB interventions. This research adds to the literature by identifying needs, strategies, and both facilitating and impeding factors for developing evaluation capacity at CBHOs. CBHOs can use this information to design and implement ECB interventions. The study can be helpful for government, institutes of higher education, and international donor and development organizations as they create ECB resources and provide effective support to grassroots ECB initiatives.