Browsing by Subject "Evaluation"

Now showing 1 - 20 of 42
  • Thumbnail Image
    Item
    Across the Patient Care Practitioner Threshold: Identifying Threshold Concepts and Evaluating the Teaching of the Pharmacists’ Patient Care Process
    (2017-05) Kolar, Claire
    Abstract Background. The practice of the pharmacy now has a specific Pharmacists’ Patient Care Process (PPCP) to be utilized by pharmacists and pharmacy educators. There may be threshold concepts associated with students’ learning the PPCP. A threshold concept is idea or concept that transforms the learner’s way of thinking about a certain topic or discipline. They are often troublesome for the learner, but once identified, can influence the teaching of or be used in the evaluation of the teaching of the PPCP. The first aim of this study was to identify threshold concepts associated with pharmacy students learning the PPCP. The second aim was to create, validate and test an instrument based on the previously identified threshold concepts to evaluate to what extent the PPCP is taught across pharmacy curriculum. Methods. The first phase of this study convened five focus groups to identify possible threshold concepts. The data was analyzed by deductive content analysis and confirmed by an expert consensus panel using the Nominal Group Technique. In phase two, the Patient Care Threshold Concepts Evaluation Instrument (PCTC-EI) was created using the identified threshold concepts. A Content Validity Index was calculated for the items on the PCTC-EI. The PCTC-EI was then administered to a purposive sample of pharmacy faculty and students at one institution. Results. Five threshold concepts, including Threshold Concept #4 Discern a patient’s medication experience and incorporate his or her individual knowledge and beliefs in to the care provided (Medication Experience), were identified from the focus groups and confirmed by the expert consensus panel. Thirty-eight (38) students and faculty, of the 59 invited (64.4%) provided responses to the PCTC-EI. Only 42.1% of respondents stated the average graduate’s ability regarding Medication Experience was Very Good or Excellent. Discussion. The five patient care threshold concepts identified are related to key concepts in pharmaceutical care and complement the PPCP. They can be used in a variety of ways within a pharmacy curriculum. In addition, the results of PCTC-EI illustrate the extent of teaching the PPCP and demonstrate the value of curricular-level evaluation.
  • Thumbnail Image
    Item
    Between abandonment and adoption: the value of E-valu-ation for Turkey's Educational Decision Making
    (2014-06) Cakici, Hanife
    This dissertation examines the value of evaluation for Turkey's educational decision making. Relying on individual interviews and document review, the study analyzes how key stakeholders - government officials, academics, and civil society representatives - and governmental acts envision and portray the role and utility of evaluation for public decision making specifically in the education sector. Drawing on social science literature from the fields of evaluation, comparative education, public policy, and international development, this study addresses the need to decolonize the concept and practice of evaluation, as this trans-disciplinary field is rapidly cutting across geographic, historic, social, and cultural borders. This study revisits the origins of evaluation practice in the global Northern context, traces its expansion into the global South across a number of sites, and argues that context matters in transferring, borrowing, negotiating, establishing, practicing, and using the concept and practice of evaluation. Evidence for this study's conclusions comes from Turkey's relatively immature history with evaluation in the education decision domain. Motivated by the desire to become one of the top ten largest economies in the world by 2023, Turkey's rapid development underlined educational achievement and growth as the roadmap. This quest necessitated a specific form of educational governance and decision making driven by the principles of effectiveness and efficiency. At the heart of these principles, monitoring and evaluation (M&E) has long lain as a tool of accountability, learning, and improvement, in which Turkey's entire public administrative culture has historically lagged behind. In response to this immaturity, supranational authorities and international donors have provided financial and technical impetus for locating M&E systems, practice, and information in the Turkish education decision domain. Coupled with the country's official drive for modernization, international actors, to a great extent, paved the way to legal arrangements for streamlining evaluation. Specifically, the Green Paper published after the European Union's "Strengthening the Capacity of the Ministry of National Education Project" later became the conceptual foundation for Decree No. 652 that helped establish M&E units at the Ministry for the first time in Turkey's educational history. Despite all these efforts, the study reveals that evaluation remains as a new concept that is closely associated with quantification, performance-based budgeting, and compliance. Evaluation's value mostly resides in its symbolic representation of modern norms of governance to which Turkey eagerly wants to commit. Yet, reported confusion about what evaluation really entails, as a concept and as a field of practice, is paired with highly centralized and politically polarized educational governance, all together situating evaluation in foster care in Turkey: it is neither fully adopted, nor is it completely abandoned. Adopting a constructivist-critical outlook on the role of evaluation in the global South, this study endeavors to locate this field of practice in the broader context of international development with its negotiated margins, borders, and struggles. By suggesting that evaluation is a marker of a country's quest to modernize and Westernize, this study sheds light on the direction of cross-cultural expansion of the field of evaluation.
  • Thumbnail Image
    Item
    Bridge Health Monitoring and Inspections – A Survey of Methods
    (Minnesota Department of Transportation, 2009-09) Gastineau, Andrew; Johnson, Tyler; Schultz, Arturo
    Since the collapse of the I-35W bridge in August 2007, bridge health monitoring has become an area of intense interest. This report defines terminology related to bridge health monitoring and provides a general glossary of available monitoring systems. The glossary is meant to help readers make an informed decision by understanding how different systems function and their strengths and weakness. The authors developed a questionnaire to send to commercial companies offering monitoring systems. Of the 72 questionnaires that were sent to commercial companies, 38 companies responded and are included in this report. From information provided with these questionnaires, available commercial systems are briefly summarized. Criteria for system evaluation were developed to help the bridge owner narrow down company choices for bridge application. After the owner answers a set of questions pertaining to a particular bridge, a program developed in Microsoft EXCEL helps the bridge owner decide the best system for a particular situation. An example is provided for program clarity. Once company choice is narrowed down, additional criteria were developed to aid in final product choice.
  • Thumbnail Image
    Item
    Building a Fractal Evaluation: A Study on Improving Skills and Attitudes towards Evaluation Through a Design Similar to Program Pedagogy
    (2015-05) Taylor, Anika
    This thesis is a study on using an evaluation model that mimics the educational pedagogy of an organization as an approach to evaluation capacity building. Referred to as a fractal evaluation, this model of using a "self-similar"� approach to learning in an organization sought to improve in staff both their skills in and attitudes around evaluation, or their Evaluation Assets. After the implementation of this model in the education department of a small science museum, interviews and surveys were conducted with staff to determine the effectiveness of using this approach to improve their attitudes and mindsets around the practice of evaluation. The results of the study suggest that staff attitudes and interest in the topic both improved notably along with an expressed increase in staff investment in the process and desire for ongoing learning about program evaluation.
  • Thumbnail Image
    Item
    Building Evaluation Capacity in Educational Organizations: A Longitudinal Case Study of One Metropolitan School District
    (2016-07) Rohmer-Hirt, Johnna
    Although evaluation capacity building (ECB) is relatively prevalent in the literature, there is not an extensive empirical body of work related to the topic. The purpose of this study was to add to the existing literature through a 30-year historical longitudinal case study of a large metropolitan school district from the perspective of its leadership. This research examined the longitudinal evidence of the context, processes, and activities that impact ECB within a K-12 educational setting. In addition, it traced the development of an internal evaluation department and the steps to build supports for data use and evaluation in the midst of accountability, using process use as the primary strategy toward ECB. The study consisted of interviews with 14 district leaders, along with document reviews over the years 1985 to 2015, encompassing significant legislative enactments such as No Child Left Behind (NCLB) and the public accountability era in K-12 education. The district leadership was found to value data use across time, along with community involvement and input, and demonstrated a strong legislative awareness and presence. The leaders who were interviewed also viewed evaluation as mission and vision critical across time. Substantial changes were made over the 30-year time frame in the number and types of staff members in the department, the internal process and activities of the department, how the department collaborated with others across the organization, and the organizational reporting structure. In addition, organizational factors such as the shift from evaluation for mandatory reasons (such as grants and legislative requirements) to evaluation for internal decision making and inquiry occurred over the 30-year span, along with important changes in technology and data integrity, accessibility, and understandability. Eight key recommendations to propel ECB emerged from the research: 1) attend to data integrity, accessibility, and understandability – including technology; 2) capitalize on one-time and special funding; 3) recognize that leadership matters and change is disruptive; 4) hire the right people; 5) remember that disposition/personality matters; 6) make ECB an intentional, ongoing focus; 7) work toward consistency, not the new next best thing; and 8) build on previous work instead of completely starting over.
  • Thumbnail Image
    Item
  • Thumbnail Image
    Item
    Curriculum policy, controversy, and change: Minnesota's profile of learning, 1993-2003
    (2008-10) DeLapp, Peggy Reed
    This historical case study of Minnesota's Profile of Learning examines how and why the policy was created, implemented, and finally repealed. The Profile of Learning, Minnesota's first attempt at a statewide standards-based curriculum policy, was developed beginning in 1993, officially adopted in 1998, and repealed in 2003. Intended to stimulate change through statewide high expectations for students and associated improvements in curriculum and instruction, it instead became controversial and contested. The study uses a conceptual framework that addresses the social and political context for the policy, the policy actors and their beliefs about the purpose of schooling and the knowledge that should be contained in the curriculum, and the policy levers of curriculum, instruction, and teacher learning. Through the use of primary and secondary source documents and interviews, a chronology of significant events in the life of the Profile was constructed. The story was then examined to determine probable causes for the policy's failure. These included shifts in the state and national political contexts, varied policy stances on the part of policy actors and stakeholders, interest group activity from a group strongly opposed to the policy, conflicting beliefs about purpose and knowledge, and the use of ineffective policy levers. The case study suggests the need for improved attention to the policymaking process, particularly at the transition points between the policy ideas and formulation and subsequently between policy formulation and implementation. The lack of clear policy goals, appropriate policy instruments, and attention to feasibility contributed to later difficulties with the policy, as did the failure to use evaluative information to make adjustments during the implementation process. In addition, stronger collaboration between policymakers and practitioners is needed for the development of effective curriculum policy.
  • Thumbnail Image
    Item
    Curriculum Policy, Controversy, and Change: Minnesota's Profile of Learning, 1993-2003
    (2008-10) DeLapp, Peggy Reed
    This historical case study of Minnesota’s Profile of Learning examines how and why the policy was created, implemented, and finally repealed. The Profile of Learning, Minnesota’s first attempt at a statewide standards-based curriculum policy, was developed beginning in 1993, officially adopted in 1998, and repealed in 2003. Intended to stimulate change through statewide high expectations for students and associated improvements in curriculum and instruction, it instead became controversial and contested. The study uses a conceptual framework that addresses the social and political context for the policy, the policy actors and their beliefs about the purpose of schooling and the knowledge that should be contained in the curriculum, and the policy levers of curriculum, instruction, and teacher learning. Through the use of primary and secondary source documents and interviews, a chronology of significant events in the life of the Profile was constructed. The story was then examined to determine probable causes for the policy’s failure. These included shifts in the state and national political contexts, varied policy stances on the part of policy actors and stakeholders, interest group activity from a group strongly opposed to the policy, conflicting beliefs about purpose and knowledge, and the use of ineffective policy levers. The case study suggests the need for improved attention to the policymaking process, particularly at the transition points between the policy ideas and formulation and subsequently between policy formulation and implementation. The lack of clear policy goals, appropriate policy instruments, and attention to feasibility contributed to later difficulties with the policy, as did the failure to use evaluative information to make adjustments during the implementation process. In addition, stronger collaboration between policymakers and practitioners is needed for the development of effective curriculum policy.
  • Thumbnail Image
    Item
    Definitional Tension: The Construction of Race In and Through Evaluation
    (2019-12) Shanker, Vidhya
    Despite the centrality of racialized difference to evaluation, the field has yet to develop a body of literature or guidelines for practice that advance understanding of difference and inequality, including its own role therein. The purpose of this study was to broaden understanding of observed differences and inequality in evaluation beyond individuals and individual lifetimes. Drawing from critical theories of systemic oppression and system dynamics, it used a discourse-historical approach to answer three questions: How has the U.S. scholarly evaluation literature constructed racialized difference? How has that construction changed since the field began formalizing? How is that trajectory related to surrounding systems? Results showed four discursive patterns: (1) minoritization and ambivalence toward whiteness; (2) the invocation of diversity and inclusion; (3) the replacement of race with culture; and (4) the rise of and decoupled relationship between indigeneity and colonization. All four patterns were tied to meso-level dynamics. In the second two, existing recruitment and training efforts initiated and led by and for evaluators representing racially otherized groups at lower levels of the American Evaluation Association were elevated to the association’s board-level, where leadership and language were broadened to represent dimensions of difference beyond race. Analysis of archival documents and interviews tied this meso-level pivot away from race to macro-level discourse and policies associated with racialized neoliberalization, which attributes inequality to individual as opposed to structural deficits. Unlike “Equal Opportunity” or “Affirmative Action,” “diversity,” “inclusion,” and “culture” depoliticize difference and privatize the responsibility for—and benefits of—desegregation. In fourth pattern, literature that authors who identified as indigenous published, which explicitly complicated the relationship between indigeneity and colonization, increased sharply and remained higher following the organizing efforts led by evaluation scholars and practitioners who identify as indigenous. Their efforts remained in their hands rather than being elevated or broadened. Variation among the patterns suggests that the American Evaluation Association’s relations with its racially otherized members and with educational institutions, large firms, philanthropy, and government are linked to the field’s construction of racialized difference through existing institutional mechanisms. Whether the mechanisms counteract or amplify racialized neoliberalization depends on whether they circulate capital in ways that enable otherized groups to exercise collective agency and produce knowledge for structural change.
  • Thumbnail Image
    Item
    Developing evidence-based effective principles for working with homeless youth: a developmental Evaluation of the Otto Bremer Foundation's support for collaboration among agencies serving homeless youth
    (2014-02) Murphy, Nora F.
    The purpose of this research was trifold. First, it was an attempt to gain an understanding of the experiences of fourteen unaccompanied, homeless youth between the ages of 18 and 24, living in the Twin Cities metro area, who have utilized services at two or more of the six grantee organizations. The second purpose was to understand how the shared principles of these organizations have been implemented in practice. The third purpose was to explore the extent to which implementation of these principles helps lead to healthy youth development from the perspective of the youth. This study was conducted as part of The Otto Bremer Foundation (OBF) Support for Homeless Youth and is a component of a utilization-focused developmental evaluation. The researcher employed a multiple case study approach. Qualitative data were generated from interviews with the youths themselves, street workers, agency staff, and Foundation staff. Fourteen individual case studies were written, and a cross-case analysis was conducted. The analysis provides insight into how the principles are enacted, as well as how they support a young person's healthy trajectory. This study found that all nine principles were evident in case stories, albeit some more than others. All principles interacted and overlapped, but each added something unique to the organizations' approach to working with youth. Implications for practice, policy, and funding are discussed.
  • Thumbnail Image
    Item
    Discrepancy and evaluation in romantic relationships: testing the emotion in relationships model.
    (2009-06) Beckes, Lane Alexander
    This paper is a test of Ellen Berscheid's Emotion in Relationships Model (ERM; Berscheid, 1983; Berscheid & Ammazzalorso, 2001). This model is based primarily on the Discrepancy/Evaluation Theory of emotion propsed by George Mandler (1975; 1990a). The ERM predicts that emotion in interpersonal relationships occurs when our relationship partner violates our expectancies and interrupts our behavioral sequences. This expectancy violation leads to arousal. Cognitive evaluation of the situation then either simultaneously or subsequently determines whether the violation is positive or negative based on whether it provides an opportunity to promote the individual's welfare or poses a threat to the individual's welfare. The ERM also expands upon Mandler's ideas by formulating hypotheses related to the infrastructure of the relationship, specifically how interdependent relationship partners are. This paper provides strong evidence for the expectancy - arousal relationship in an experimental paradigm that tests people in intact relationships, using a real time interaction between the participant's and their partners. The ERM is well supported by the data and evidence for a variety of expectancy sources such as the partner's past behavior, social norms, individual differences in attachment history, and relationship interdpendence or behavioral closeness is gleaned and discussed.
  • Thumbnail Image
    Item
    Educational Expectations in an Urban American Indian Community: A Phenomenological Investigation
    (2017-05) Vertigan Swerdfiger, Jacqueline
    Educational Expectations in an Urban American Indian Community: A Phenomenological Investigation. This investigation uses narrative to explore the educational experiences and expectations of 10 urban, Midwestern United States American Indians. Results include insights into community-based evaluation, suggest an emerging field of Indigenous Educational Evaluation, and offers a model and suggestions that may help guide future evaluations of educational programs serving American Indian students.
  • Thumbnail Image
    Item
  • Thumbnail Image
    Item
    Evaluation of the Mujeres en Liderazgo Program at Waite House, 2008-2011 Empowerment Survey—Pre and Post Test Comparison
    (Hubert H. Humphrey School of Public Affairs, 2012-06-22) Fink, Carrie
    I will be conducting an evaluation of the Mujeres en Liderazgo program at the Waite House Neighborhood Center. The Mujeres program supports the development of strong community leaders through a combination of popular education, storytelling, and community engagement. The overall goal of the program is to increase civic participation and support the leadership of Latina women within their personal life, family, workplace, and community. Specifically, the program supports a greater engagement of Latina immigrant women in the greater Twin Cities community.
  • Thumbnail Image
    Item
    An Exploratory Study of Stakeholder Perspectives on Quality Priorities for Long-Term Services and Supports for Older Adults
    (2018-11) Davila, Heather
    Major state and federal initiatives are underway to assess and improve the quality of long-term services and supports (LTSS) received by older adults in nursing homes, assisted living facilities, and other community-based settings. Although people who use LTSS (consumers) and their families are key stakeholders in LTSS quality, they are rarely included in the policy and organizational discussions in which priorities for LTSS quality are set. Prior studies have typically used qualitative methods and examined consumer and family views on LTSS quality in relation to only one setting. Within the U.S., most of these studies were completed 15-20 years ago and results may not reflect the values of consumers and families today. Further, prior studies have rarely included multiple stakeholder perspectives, limiting the ability to compare priorities across different groups. The present study explored which aspects of LTSS quality consumers, families, and LTSS professionals value most highly and whether stakeholder views on quality priorities differ. The study addressed nine domains of quality from a person-centered perspective. Several methods were used, including: 1) development of an integrated valuing framework for LTSS quality; 2) use of Q methodology, which uses a forced ranking distribution, factor analysis, and qualitative interpretation of patterns observed; and 3) a survey containing rating, ranking, and open-ended questions. A convenience sample of 70 individuals in Minnesota representing the targeted stakeholder groups participated the Q methodology component of the study and 417 participated in the survey. Across two data collection methods, participants overall identified safety/security, dignity/respect, and staffing/staff competence as the highest priorities for LTSS quality. Although there was considerable agreement among stakeholders on top priorities, consumers rated most aspects of LTSS quality as less important than professionals, which may indicate diminished or adjusted expectations. However, consumers placed more emphasis on aspects of the physical environment than professionals. Study participants, particularly consumers and families, rated autonomy/choice as less important than most other aspects of LTSS quality, a finding that necessitates further exploration. By-person factor analysis and participant comments provide insights into survey findings. Study results highlight the need for more dialogue to clarify stakeholders’ values related to LTSS and ensure policies and practices align with these goals.
  • Thumbnail Image
    Item
    Flattening the Eviction Curve: A Quasi-Experimental Evaluation of the Brooklyn Center Tenant Protection Ordinance
    (2024-02-29) Gramlich, Jack
    This paper uses two quasi-experimental methods—synthetic control (SC) and difference-in-differences (DiD)—to evaluate the effects of the 2022 Brooklyn Center Tenant Protection Ordinance. The ordinance was adopted at a time when eviction filings were on the rise across the state. Descriptive statistics provide an indication that after the ordinance was adopted, Brooklyn Center’s eviction rate did not increase by as much as the eviction rate in other parts of suburban Hennepin County. For SC models, I compared Brooklyn Center to most other Hennepin County cities. I found statistically significant evidence that the ordinance reduced eviction rates in the period 37-48 weeks after policy adoption. This result survived several placebo tests (though it was sensitive to whether Brooklyn Park was included in the donor pool). Results for filing rates did not survive all placebo tests. For DiD, I drew from a sample of most block groups in suburban Hennepin County. Conditioning on pre-treatment covariates via doubly robust DiD, I found the policy brought reduced eviction rates and filing rates in some of the first eight months after policy adoption. DiD models survived a wide variety of robustness checks. SC and DiD provided consistent evidence of reduced eviction rates in some periods of time. The two methods produced mixed evidence on filing rates, and did not produce strong evidence of policy effects for other outcomes. This paper concludes that when evictions spiked across Minnesota following the expiration of COVID-19 eviction moratorium policies, the City of Brooklyn Center flattened the eviction curve.
  • Thumbnail Image
    Item
    A formative evaluation of a Midwest District's integrated services pilot
    (2014-06) Resch, Laurie J.
    The purpose of the Integrated Services pilot was to develop models for the delivery of special education, supplemental programs, and English as a second language services that were based on a philosophy of inclusion. The intent was to provide all students with learning experiences in which they could be successful both academically and socially in the most inclusive environment. The pilot was conducted at four elementary schools in a large Midwestern public school district. The pilot schools served students with diverse needs and were situated in divergent communities, which provided four unique contexts for the pilot. Teacher collaboration and co-teaching were utilized at each pilot school to achieve pilot goals. The purpose of this formative evaluation was to inform pilot development and to determine if the Integrated Services Pilot should be expanded to other elementary schools in the district. To inform these decisions, qualitative and quantitative data were collected in five areas: (1) professional development and support; (2) collaboration and professional learning communities; (3) impact on teacher knowledge and practice; (4) impact on student engagement and achievement; and (5) overall perceptions of the pilot. Surveys, individual and group interviews, and student achievement data provided the data for analysis and evaluation of pilot results. Teachers, instructional coaches, and principals reported that the pilot positively impacted both student engagement and student achievement. Additionally, the Integrated Services Pilot had a positive impact on teacher practice and on the relationships between support service and classroom co-teaching partners. Participants indicated that they were eager to continue with the Integrated Services model and had no desire to return to past practice.
  • Thumbnail Image
    Item
    Health Literacy Program for Minnesota Seniors: Evaluation Summary
    (2010-12-08) Aspinall, Erinn E; Beschnett, Anne; Ellwood, Alisha; Koppa, Pat
    The Evaluation Summary document presents the research findings from the Health Literacy Program for Minnesota Seniors (HeLP MN Seniors). This includes the results from the needs assessment focus groups, the workshop pre-tests and post-tests, and the outcomes assessment survey that were conducted as part of the pilot workshops. The findings were used to inform the development of an evidence-based health literacy workshop program for older adults.
  • No Thumbnail Available
    Item
    Health Literacy Program for Minnesota Seniors: Needs Assessment Focus Group Guiding Questions [Evaluation Instrument]
    (2010-12-07) Aspinall, Erinn E; Brasure, Michelle
    The Needs Assessment Focus Group Guiding Questions were created for the Health Literacy Program for Minnesota Seniors (HeLP MN Seniors). It provides focus group questions for instructors wanting to learn about the specific needs of their audience in advance of teaching the workshops.
  • No Thumbnail Available
    Item
    Health Literacy Program for Minnesota Seniors: Outcomes Assessment Survey [Evaluation Instrument]
    (2010-12-07) Beschnett, Anne; Ellwood, Alisha
    The Outcomes Assessment Survey was created for the Health Literacy Program for Minnesota Seniors (HeLP MN Seniors). It provides a questionnaire to gather outcomes data from workshop participants.