Browsing by Subject "Accountability"

Now showing 1 - 15 of 15

Accountability mechanisms in public multi-campus systems of higher education.
(2011-03) Rothchild, Mary Todd
Since the 1990s, higher education has been faced with a significantly different context for performance expectations. Public policy makers and governing boards of state systems entrusted by the public to serve them have faced increasing scrutiny for outcomes related to student access and success, tuition affordability and efficiency in operations. During this time, policy reform largely focused on the structural design of higher education but with less attention paid to governance methods and the effectiveness of accountability tools to meet performance expectations. This study is guided by the theory that public multi-campus systems of higher education, as loosely-coupled organizations, could enhance performance when utilizing governance practices and accountability mechanisms that encourage high levels of autonomy and accountability. Using a two-part, quantitative and qualitative research methodology, this study was designed to define and measure accountability mechanisms used in the transactional environment between state higher education leadership (board members and system leaders) and institutional leaders. Using a subset of ten state systems of public higher education, each state was determined to be either high-performing or low-performing. In the quantitative study, institutional leaders were asked in an Internet survey to rate or rank 18 accountability mechanisms on five characteristics: importance, discretion, "high stakes," and whether they reflected professional norms of higher education or were political or market-based. In the qualitative study, interviews of system leaders examined their views on the balance between accountability and autonomy in multi-campus systems. The results from the survey indicated that four mechanisms were considered high in importance and high stakes by institutional leaders in both low- and high-performing states: strategic planning, state funding, institutional accreditation and system policies. While these mechanisms each play different roles in system governance and accountability, system leaders can utilize these accountability mechanisms as tools to develop flexible system policies and funding mechanisms, coordinate planning between system and institutional goals, and use data systems for assessment and accreditation. Governing leaders of public multi-campus systems of higher education could use the study results as they design normative values for accountability and autonomy by considering them as mutually supportive constructs. Evidence from the study illustrated the importance of building shared commitment through connections between campuses and the system/state. In addition, system leaders should be attentive to governance policies and practices that encompass aspects of campus self-determination and provide effective incentives for responding to the system's organizational priorities.
Addressing the inclusion of English language learners the educational accountability system:lessons learned from peer review.
(2010-01) Christensen, Laurene L.
This study investigated the inclusion of English language learners (ELLs) in state standards and assessments, as measured by comments made by peer reviewers in the federal evaluation of states' standards and assessments. As required by the Elementary and Secondary Education Act (ESEA), reauthorized in 2004 as No Child Left Behind (NCLB), states are required to develop challenging content standards in English language arts and mathematics, and to assess students on these standards. Nationwide, English language learners have been performing below proficient on these state tests. The goal of this study was to determine the elements of the standards and assessment system that peer reviewers thought could be addressed in order to improve the accessibility of these assessments for ELLs.This dissertation employed two research methods: a frequency and distribution analysis of the peer comments relevant to ELLs across the seven Critical Elements of the peer review, and a thematic analysis of the comments, using inductive methods to establish themes. The study found that comments relevant to assessing English language learners were present in all seven critical areas and in all states. All states had two or more critical elements with comments relevant to ELLs. Key themes from the qualitative analysis included accommodations, diverse stakeholders, reporting accuracy, inclusion, reporting for parents, migrant students, comparability of test forms, bias review, DIF analysis, and standards. States have begun to address the inclusion of ELLs in statewide standards and assessments; however, state agency personnel have more work to do in these areas before assessments can be seen to be fully accessible to ELLs.
Agendas, Actions, and Accountability in International Development: A Case Study of the USAID Education Strategy for 2011-2015
(2014-12) Willis, Garth
Beginning in 2010, the United States Agency for International Development (USAID) implemented a series of institutional reforms designed to rebuild USAID as the world's premier development agency. This research examined one component of this larger reform effort, the USAID Education Strategy for 2011-2015: Opportunity Through Learning. The purpose of this research is to examine how the current agenda for basic education was determined, what actions occurred based on this agenda, and how USAID will be held accountable for results. The problem is that while new strategies are routinely developed and implemented, without accountability for results lessons learned to do not inform future policy. The education strategy is an exemplary case study to examine USAID's redesigned approach to international development as: 1) it is the first sector strategy developed under Administrator Rajiv Shah and paved the way for future strategies, 2) it resulted in significant changes to priorities and programs in education, and 3) it represents an unprecedented commitment by USAID to be held accountable for results. This case study of the 2011 Education Strategy provides insights into USAID's new operational model and the implications the model poses for international development assistance. Lessons learned can also inform other global agendas including the design of education indicators for the Sustainable Development Goals. The research uses a framework of agendas, actions, and accountability to analyze the political context and alternative agendas that are the drivers of strategy development and implementation at USAID. This research highlights how agendas are formulated and strategies developed, how ensuing actions depend on both bureaucratic and political processes to deliver results, and how accountability for results is an ambiguous process that has profound implications for the development agenda. Through a document review and interviews with policy specialists and technical experts at USAID, this research adds to existing research by examining and critiquing the policy processes of agendas, actions, and accountability in the context of international development, where decisions are shaped by a global set of actors and contexts.
Building Evaluation Capacity in Educational Organizations: A Longitudinal Case Study of One Metropolitan School District
(2016-07) Rohmer-Hirt, Johnna
Although evaluation capacity building (ECB) is relatively prevalent in the literature, there is not an extensive empirical body of work related to the topic. The purpose of this study was to add to the existing literature through a 30-year historical longitudinal case study of a large metropolitan school district from the perspective of its leadership. This research examined the longitudinal evidence of the context, processes, and activities that impact ECB within a K-12 educational setting. In addition, it traced the development of an internal evaluation department and the steps to build supports for data use and evaluation in the midst of accountability, using process use as the primary strategy toward ECB. The study consisted of interviews with 14 district leaders, along with document reviews over the years 1985 to 2015, encompassing significant legislative enactments such as No Child Left Behind (NCLB) and the public accountability era in K-12 education. The district leadership was found to value data use across time, along with community involvement and input, and demonstrated a strong legislative awareness and presence. The leaders who were interviewed also viewed evaluation as mission and vision critical across time. Substantial changes were made over the 30-year time frame in the number and types of staff members in the department, the internal process and activities of the department, how the department collaborated with others across the organization, and the organizational reporting structure. In addition, organizational factors such as the shift from evaluation for mandatory reasons (such as grants and legislative requirements) to evaluation for internal decision making and inquiry occurred over the 30-year span, along with important changes in technology and data integrity, accessibility, and understandability. Eight key recommendations to propel ECB emerged from the research: 1) attend to data integrity, accessibility, and understandability – including technology; 2) capitalize on one-time and special funding; 3) recognize that leadership matters and change is disruptive; 4) hire the right people; 5) remember that disposition/personality matters; 6) make ECB an intentional, ongoing focus; 7) work toward consistency, not the new next best thing; and 8) build on previous work instead of completely starting over.
Career services' contributions to learning outcomes of seniors at a research intensive university.
(2009-06) Boettcher, Brett Michael
Few empirical research studies have delved into what college students learn by participating in the services offered by career services, specifically one-on-one advising/counseling, workshops, resource use, events, on-campus recruiting and resource library use. This study examined the extent to which college seniors achieve learning outcomes based on the use or non-use of a career services office. Subjects from both groups were compared on both institutional and career development learning outcomes through an on-line survey. They were also compared based on categorical variables including gender, race/ethnicity, undergraduate degree, graduation date, and school of enrollment. A total of 204 (26%) subjects responded to the survey. The results indicated that a B.A. undergraduate degree (p<0.05) and being registered in the study school of arts and sciences and the school of education and social policy (p<0.05) were associated with the use of career services. There was a significant association between the institutional learning outcome "perspective on how my degree will contribute to my overall career" (p<0.01) and the career development learning outcomes "decide on a career field that fits you" and "develop an understanding of how your degree fits in the working world" (p<0.05) and the non-use of career services. For both institutional and career development learning outcomes, the use of resources other than career services had more impact. The only gender difference was for the institutional learning outcome "knowledge about the importance of being engaged in civic activities in the community" (p<0.05) for which females had a higher association than males. Results indicated that the services "on-campus interviews" and "on-line resources" had the greatest impact on users of career services, while "one-on-one advising/counseling" and "workshops" did not have a significant impact. Learning levels were also found to be similar for users and non-users of career services.
Consumers and Makers: Exploring Opposing Paradigms of Millennial College Readiness
(2017-04) Jackson, Matthew
The political and technological circumstances of the past two decades have culminated in opposing epistemic paradigms of college readiness, where millennial students’ conceptual understanding of “learning” is both narrowed to meet the demands of school systems bound to accountability and amplified by a rapidly evolving digital world. The researcher theorized that students situated within these paradigms may have developed dispositions toward the purpose of learning as primarily either consumption-oriented (consumers) or creation-oriented (makers). This study hypothesized that correlations existed among these consumer/maker dispositions and millennial college students’ epistemic beliefs and key learning skills. The researcher developed an original survey instrument that was provided to a sample of 625 first year students (primarily 18-19 years of age) at a Midwestern liberal arts university. Quantitative, statistical analyses of responses were completed to develop constructs, understand variables, and determine the nature of relationships between variables. The results of these analyses found that respondents were 3-to-28-times more likely to demonstrate consumer dispositions than maker dispositions. The data supported the hypothesis of this study: statistically significant, positive correlations were present in 13 out of 24 instances, suggesting that as one approached the likelihood of having a maker disposition, one was also more likely to exhibit sophistication of epistemic beliefs and to have initiated or developed key learning skills through both high school experiences and the use of information-communication technologies.
The Currency of Public Trust: Ways the nonprofit sector builds trust among constituents.
(2023) Rodel Sorum, Kristina
In the nonprofit sector, building trust among constituents is of paramount importance to an organization's effectiveness. This paper explores how organizational leadership prioritize constituents and establish a trusting reciprocal relationship with them. Through interviews with leadership figures at nine arts nonprofits in Minneapolis and Saint Paul Minnesota, this paper provides real examples in trust building, repair after a breach in trust, and provides recommendations for best practices.
A Descriptive Mixed-Methods Examination of Corporate Social Responsibility, Accountability, and HRD in the Access Economy
(2021-04) Hart-Mrema, Tasha
App-based companies have emerged as legitimate forms of business, giving way to gigs, odd jobs, and convenient access to goods, products, or services. As with all business, growth brings the need for socially responsible change. This study presents an original exploration of factors that underlie stakeholder perceptions of corporate social responsibility (CSR) in the access economy (i.e. gig/sharing economy). The access economy is framed as a complex adaptive system and benefits of platform companies are discussed along with prevalent criticisms regarding worker classifications and unethical business practices. By utilizing a mixed methods survey, empirical evidence is provided that not only identifies a negative association between observed versus expected CSR in the access economy, but also provides insight on the need for four types of platform stakeholder responsibility. Although findings revealed that perceptions on accountability were spread across stakeholder groups, the overarching theme is that more accountability should be allocated to platform companies. The data support the idea that consumers expect platform companies to go beyond mere onboarding and to establish some form of worker protections in line with what would generally be expected from a more conventional business model. Opinions of platform accountability appear to be influenced by expectations associated with specific industries (e.g. transportation vs. tourism) and whether or not workers would have direct contact with consumers. Individual factors such as ethical ideology, risk aversion, desire for certain types of service attributes, and propensity toward social proof techniques are shown to be associated with perceptions of CSR. This study serves as a starting point for the field of HRD to enter the access economy.
Exploring the Accuracy of School Rankings for Accountability Decisions
(2023-12) Vue, Yi
In the U.S., the school accountability system in education uses testing as a major part of its decision-making process. Part of this decision making comes from the result of student test scores, which are aggregated to estimate school scores (typically in the metric of percent proficient). Regarding precision of such school metrics, measurement error is assumed to have a net value of zero across the population of students within a school, but students are not randomly assigned to schools. The aggregation of student scores to obtain school scores retains the measurement error from the students within their respective schools. This study used available data to emulate how one state uses test scores to rank schools and simulated various conditions to examine how the accuracy of the school rankings might be affected.The conditions examined multiple measurement error models to evaluate changes in accuracy based on the various measurement error models. If measurement error has a net value of zero, accuracy should not be affected across this condition. Another condition examined the different methods of aggregating student scores to obtain school scores. If student measurement error is aggregated along with the student score to estimate school scores, some schools may have more error than other schools, and this information may be helpful when making interpretations about school rankings. The last condition was the reliability of the test. Real data summary statistics were used in the study, so the minimum and maximum test reliability was examined regarding how accuracy was affected across the other conditions. Several major takeaways were found. When there was more error variance, the accuracy of school rankings dropped, but changes in the specification of the error distribution did not change the accuracy of school rankings. The biggest association with accuracy of school rankings was how student test scores were aggregated to obtain school scores. The aggregation of student scores condition had the largest difference in accuracy across the conditions, on average. In this study, I hoped to inspire more research to provide validity evidence to support the decision-making process of the accountability system, especially studies that attempt to minimize error within these processes.
Institutional Accountability and competition for resources in undergraduate education among U.S. public four-year institutions.
(2012-01) Akey, Lynn D.
With a growing concern that society’s needs are not being met, there are heightened expectations for accountability for public purposes. At the same time higher education institutions are experiencing increasing competition, as well as decreasing state support for public higher education. The concern is that competition for resources is overtaking accountability for public purposes. Using an observational correlational research design, this study explored the relationships between institutional competition for resources and accountability for public purposes at the undergraduate level among 428 U. S. public four-year institutions. The study examined institutional competition organized around the four key markets that generate institutional revenues (student enrollment, research funding, public fiscal support, and private giving) and institutional accountability for public purposes defined by accountability measures most frequently included in state-level performance accountability systems (access, affordability, and completion). The relationship between institutional competition for resources and accountability for public purposes was also examined considering institutional fiscal health and market segment. The accountability triangle, resource dependency theory, and postsecondary market taxonomy provided a conceptual framework for the study. The results of the study indicate a statistically significant relationship between institutional competition for resources and institutional accountability for public purposes. In particular, a negative relationship was observed between institutional competition and accountability for access (R2 = .16) and affordability (R2 = .05), and a positive relationship was observed between institutional competition and accountability for completion (R2 = .09). Institutional fiscal health was a statistically significant factor in only the relationship between institutional competition and accountability for access producing an increase in the predictive power of the model (R2 = .16 to R2 = .18). In addition, no differences were found in the nature of the relationship between institutional competition and institutional accountability for public purposes across the market segments of higher education. Most importantly, this study provides empirical evidence of a relationship between the complicated constructs of institutional competition for resources and institutional accountability for public purposes at the undergraduate level within U.S. public four-year institutions. With the force of institutional competition likely to increase and the necessity for higher education to serve public purposes critical, additional research further exploring the relationships between institutional competition and institutional accountability at the undergraduate level is crucial.
Making sense of testing: English language learners and statewide assessment
(2008-07) Stone, Karla Rae
The federal No Child Left Behind (NCLB) Act of 2001 has dramatically changed the educational landscape for all students by increasing the number of standardized tests used for accountability purposes. The impact is profound on students for whom English is not their first language, in part because of the sheer number of tests they are required to take. The challenges increase when tests become high stakes for students in the form of high school exit exams, an increasingly prevalent requirement in states, though not required under NCLB. The policies for tests are complex and can easily be viewed as devoid of any attention to the human side of testing--the ground level, nuanced, and sometimes complicated application of policy at the student level. Yet it is at this level where a policy becomes truly validated. The literature on accountability testing rarely considers student perceptions. This research is an attempt to contribute to that void by combining a student voice paradigm with a sense-making framework. Case study methodology was used and incorporated methods that moved from breadth towards depth as the study evolved. In the case of this research, that evolution was from participant observation (Phase I) to document review, inquiry groups and interviews (Phase II). Participants' pre-existing knowledge and experiences with testing are explored alongside the new knowledge gained as participants in this research study. Findings clearly demonstrate that the stakes connected to testing influence the ways in which students make sense of testing, as do the policy signals. Beyond contributing to the fields of educational policy and student voice initiatives, the intent of this research is to spur advocacy on behalf of and in conjunction with English language learners. Whether for systems accountability (the Test of Emerging Academic English), or for individual accountability as a requirement for a diploma in Minnesota (the Basic Skills Tests), the findings reveal both the complexities of and urgency for exploring testing policies from students points of view. Additionally, findings validate the need for students' voices in policy research, and suggest several implications based on reported experiences with statewide testing.
Minnesota State Teacher Development, Evaluation and Peer Support Model Pilot: Initial Evaluation Report
(2014-02-05) Dretzke, Beverly; Ingram, Debra; Kwon, Melissa; Peterson, Kristin; Sheldon, Timothy; Wahlstrom, Kyla; Crampton, Anne; Dahmes, Shannon; Larabee, Kaitlyn; Lim, Alicia
Minnesota Statute requires that districts begin evaluating teachers in the 2014-2015 school year. In response to the statute, during early winter 2011, the Minnesota Department of Education (MDE) convened a work group to consult with the Commissioner to develop a state model for teacher growth and development. In winter 2013, MDE released the Minnesota State Teacher Development, Evaluation, and Peer Support Model (hereafter “Model”) and began planning for a pilot of the Model during the 2013- 2014 school year (hereafter “Pilot”). The Model includes three components: 1) teacher practice, 2) student engagement, and 3) student learning and achievement. Sixteen school districts and one charter school across Minnesota agreed to participate in the Pilot. Six of the districts are implementing the full Model (all three components) and nine districts are implementing one or two components of the Model. The size of participating districts varies widely, ranging from 287 students to 7,356 students. In August 2013, the Joyce Foundation funded the University of Minnesota’s Center for Applied Research and Educational Improvement (CAREI) to conduct an evaluation of the Pilot. This report summarizes results of surveys and interviews conducted during November and December 2013 with Pilot participants. This status report is the first of three reports. The report for the entire Pilot year will be available in August 2014 and a final report, which will include information on the value-added assessments, an element of the student learning and achievement component, will be submitted to the Joyce Foundation in December 2014. The data in this report is preliminary and encompasses only the first three months of the school year; thus readers should not over-generalize the findings or conclusions presented here. The purpose of this interim report is to provide formative feedback to the Minnesota Department of Education (MDE).
Minnesota State Teacher Development, Evaluation and Peer Support Model Pilot: Initial Evaluation Report Executive Summary
(2014-02-05) Dretzke, Beverly; Ingram, Debra; Kwon, Melissa; Peterson, Kristin; Sheldon, Timothy; Wahlstrom, Kyla; Crampton, Anne; Dahmes, Shannon; Larabee, Kaitlyn; Lim, Alicia
Minnesota Statute requires that districts begin evaluating teachers in the 2014-2015 school year. In response to the statute, during early winter 2011, the Minnesota Department of Education (MDE) convened a work group to consult with the Commissioner to develop a state model for teacher growth and development. In winter 2013, MDE released the Minnesota State Teacher Development, Evaluation, and Peer Support Model (hereafter “Model”) and began planning for a pilot of the Model during the 2013- 2014 school year (hereafter “Pilot”). The Model includes three components: 1) teacher practice, 2) student engagement, and 3) student learning and achievement. Sixteen school districts and one charter school across Minnesota agreed to participate in the Pilot. Six of the districts are implementing the full Model (all three components) and nine districts are implementing one or two components of the Model. The size of participating districts varies widely, ranging from 287 students to 7,356 students. In August 2013, the Joyce Foundation funded the University of Minnesota’s Center for Applied Research and Educational Improvement (CAREI) to conduct an evaluation of the Pilot. This report summarizes results of surveys and interviews conducted during November and December 2013 with Pilot participants. This status report is the first of three reports. The report for the entire Pilot year will be available in August 2014 and a final report, which will include information on the value-added assessments, an element of the student learning and achievement component, will be submitted to the Joyce Foundation in December 2014. The data in this report is preliminary and encompasses only the first three months of the school year; thus readers should not over-generalize the findings or conclusions presented here. The purpose of this interim report is to provide formative feedback to the Minnesota Department of Education (MDE).
Using data to increase student achievement:a case study of success in a sanctioned school.
(2011-05) Fischer, Brenda Elaine
The No Child Left Behind Act of 2002 fundamentally changed the ways in which schools are held accountable for the academic achievement of all students. Each year, millions of tests are given to students in the United States to comply with the federal accountability mandates set forth by this unprecedented federal legislation. Since these tests are so plentiful and prevalent and so much time and energy is invested in gathering results, it seems it might be possible for this multitude of data to be used for purposes other than external accountability. Might school leaders be able to utilize the data from mandated standardized tests to strategically enable schools to move toward increased student achievement across curricular goals? This qualitative case study tells the story of how teachers and administrators at one Minnesota elementary school, that was labeled in need of improvement, used a variety of data available to them to increase student academic achievement scores. Findings from this study include discussions of the factors and combination of factors that led to increased academic success. This study also includes suggestions for teachers, principals, policy makers, and institutions of higher learning, based on information gained during interviews and from the literature, for creating the conditions under which data can be used as an essential component in the ongoing challenge to increase academic achievement for all students.
Why does the graduation rate productivity of U.S. public universities vary? Looking outside their boundaries for answers.
(2010-09) Asmussen, John G.
The importance of college completion has risen high on the U.S. policy agenda in recent years. An obvious strategy for increasing college completion is improving graduation rates, which for public universities have hovered around 50% for decades. Higher education scholars previously have revealed many student and institutional characteristics which help explain differences in graduation rates. This study treated the variation which could not be explained by student and institutional differences as an indicator of the level of productivity associated with an institution's graduation rate. It used punctuated equilibrium theory, which suggested that productivity would be aligned with expectations from the external environment, as a conceptual framework. Using multivariate analysis of covariance statistical techniques, the study identified several elements from the external environments of 398 public universities which had statistically significant relationships with differences in the productivity levels of their graduation rates. Statistically significant environmental elements included the type of state-level higher education plan in place, use of performance budgeting programs, existence of local governing boards, and choice of regional accrediting agency. The study found no evidence of a statistically significant effect associated with other environmental elements, notably the use of performance reporting or performance funding programs. The results suggest that public universities which have collegial relationships with their external environments have the most productive graduation rates. Public universities on the governmental agenda in their states had more productive graduation rates than universities which were absent from the governmental agenda. Productivity, though, was not further enhanced for universities subject to the most aggressive accountability mechanisms. The public universities subject to the most aggressive accountability levels, e.g., state-level plans with targeted completion rates combined with performance budgeting programs, showed the lowest productivity levels. Perhaps the inertia of low productivity was resistant to external accountability efforts. Ironically, the public universities which enjoyed the most autonomy only showed mediocre productivity in their graduation rates. State policymakers and higher education officials could use the study results as they reconsider the nature of their relationships and the design of accountability mechanisms in pursuit of improved college graduation rates.

University Digital Conservancy

Browse by Subject

Browsing by Subject "Accountability"