Evaluation

This study presents a reanalysis of a large randomized controlled trial of school-based mentoring and examines the estimated effect of mentoring as a function of mentee-reported relationship quality using a novel statistical approach.

This paper is based on the simple idea that students’ educational achievement is affected by the effort put in by those participating in the education process: schools, parents, and, of course, the students themselves.

The Benchmarks for Advanced Tiers (BAT) allows school teams to self-assess the implementation status of Tiers 2 (secondary, targeted) and 3 (tertiary, intensive) behavior support systems within their school. The BAT is based on factors drawn from the Individual Student Systems Evaluation Tool (I-SSET). School teams can use the BAT to build an action plan to delineate next steps in the implementation process.

This article examines issues in developing valid and reliable direct observation of behavior. Suggestions are made to minimize the problems that threaten validity and reliability. The discussion is concluded by an examination of costs and benefits of direct observation and who pays them and who benefits from them.

The CJAA funded the nation’s first statewide experiment concerning research-based programs for juvenile justice. The question here was whether they work when applied statewide in a “real world” setting. This report indicates that the answer to this question is yes— when the programs are competently delivered. 

This article evaluates a procedure‐based scoring system for a performance assessment (an observed paper towels investigation) and a notebook surrogate completed by fifth‐grade students varying in hands‐on science experience. 

The Hexagon Discussion and Analysis Tool helps organizations evaluate new and existing programs and practices. This tool is designed to be used by a team to ensure diverse perspectives are represented in a discussion of the six contextual fit and feasibility factors.

This paper examines how to measure teacher performance and the practices necessary for increasing teacher trust in systems designed to effectively measure performance.

In this study, the authors evaluate the efficacy of videotape analysis with structured expert consultation and self-evaluation to improve teacher candidates’ instructional delivery. A single-case, multiple-baseline, across-participants design was used to evaluate lesson components, rate of praise statements, and rate of opportunities to respond included by teacher candidates in their teaching.

This study developed a zero-to-five index of the strength of accountability in 50 states based on the use of high-stakes testing to sanction and reward schools, and analyzed whether that index is related to student gains on the NAEP mathematics test in 1996–2000.

Recent efforts to attract and retain effective educators and to improve teacher practices have focused on reforming evaluation and compensation systems for teachers and principals. In 2006, Congress established the Teacher Incentive Fund (TIF), which provides grants.

This chapter provides an integrative approach to measurement as it is likely to be applied to assessment and evaluation within an RTI framework

This overview provides information about teacher evaluation as it relates to collecting information about teacher practice and using it to improve student outcomes. The history of teacher evaluation and current research findings and implications are included.

The purpose of this overview is to provide information about the role of formal teacher evaluation, the research that examines the practice, and its impact on student outcomes.

This overview examines the current understanding of research on performance feedback as a way to improve teacher performance and student outcomes. 

The purpose of this overview is to provide information about informal evaluation as a practice in schools, and the current understanding of research related to informal teacher evaluation to improve teacher performance and student outcomes.

The purpose of this overview is to provide an understanding of the research base on professional development and its impact on student achievement, as well as offer recommendations for future teacher professional development

This brief explores research that points to the opportunities and the challenges that evaluating teacher preparation programs differently presents.

The authors examined the extent to which program integrity (i.e., the degree to which programs were implemented as planned) was verified and promoted in evaluations of primary and early secondary prevention programs published between 1980 and 1994. 

This article presents a conceptual framework for examining the design and implementation of teacher evaluation processes in school organizations. 

A growing number of researchers are studying whether value-added measures can do a good job of measuring the contribution of teachers to test score growth. Here I summarize a handful of analyses that shed light on two questions.

This article examines the politics of principal evaluation through both an extensive review of the literature and in-depth interviews with principals and superintendents. The findings reveal that the format and processes used in principal evaluation often vary from one district to another and that principals and superintendents frequently hold different perspectives about the purposes and usefulness of evaluation.

Principal evaluations can be important tools for improving leadership practice, but evaluations have often described by principals and researchers as unsystematic and lacking timely and actionable feedback. This study examines principal perceptions of the Texas Principal Evaluation and Support System.

Problems associated with the school psychologists traditional assessment functions and methodology are identified and contrasted with the need for assessment information that can contribute meaningfully to the formulation and evaluation of educational interventions. 

In the past decade, nearly all states have revised their principal evaluation policies, prompting school districts across the country to rethink how they are evaluating school leaders. The new principal evaluation systems that emerge out of these policy reforms often couple increased accountability with a greater emphasis on development in an effort to spur continuous improvement in school leadership practices.

A prominent philosophical influence in school consultation research proposes that to effectively serve children, school psychologists must work primarily and paradoxically on changing the behavior of adults. Although the consultation literature base has traditionally conceptualized this role within a dyadic model emphasizing the consultant-consultee relationship, much of school consultation today occurs in the context of teams rather than dyads.

Principals are in a paradoxical position. On one hand, they're called on to use research-based strategies to improve student achievement. On the other, they're increasingly required to micromanage teachers by observing in classrooms and engaging in intensive evaluation. The authors point out that these two positions are at odds with each other.

C-SAIL was established in July 2015 as a resource on the implementation and effects of college and career readiness standards. The Center is funded through a grant from the Institute of Education Sciences (IES) of the U.S. Department of Education.

Research strongly suggests that feedback obtained through direct observations of performance can be a powerful tool for improving teacher’s skills. This study examines a peer teacher observation method used in England. The study found no evidence that Teacher Observation improved student language and math scores.

a brief overview of findings from the Blueprints for Violence Prevention-Replication Initiative is presented, identifying factors that enhance or impede a successful implementation of these programs.

This section includes tools and resources that can help school leaders, teachers, and other stakeholders be more strategic in their decision-making about planning, implementing, and evaluating evidence-based interventions to improve the conditions for learning and facilitate positive student outcomes.

This section includes tools and resources that can help school leaders, teachers, and other stakeholders be more strategic in their decision-making about planning, implementing, and evaluating evidence-based interventions to improve the conditions for learning and facilitate positive student outcomes.

The Every Student Succeeds Act (ESSA) returns decision making for our nation’s education back where it belongs – in the hands of local educators, families, and communities – while keeping the focus on students most in need.

This book shows how principals and other school leaders can develop the skills necessary for teachers to deliver high quality instruction by introducing principals to a five-part model of effective instruction.

Part One provides the reader with information essential to understanding not only the content of the section that follow but also the wealth of material that exists in the literature on  program evaluation. Part Two introduces you to different approaches to evaluation to enlarge your understanding of the diversity of choices that evaluators and stakeholders make in undertaking evaluation.

A growing number of evidence-based psychotherapies hold the promise of substantial benefits for children, families, and society.  For the benefits of evidence-based programs to be realized on a scale sufficient to be useful to individuals and society, evidence-based psychotherapies need to be put into practice outside of controlled clinical trials. 

This article describes a school-based randomized trial in over 200 New York City public schools designed to better understand the impact of teacher incentives. 

This meta-analysis investigated the effects of formative evaluation procedures on student achievement. The data source was 21 controlled studies, which generated 96 relevant effect sizes, with an average weighted effect size of .70. The magnitude of the effect of formative evaluation was associated with publication type, data-evaluation method, data display, and use of behavior modification. Implications for special education practice are discussed.

This study examined the educational effects of repeated curriculumbased measurement and evaluation. Thirty-nine special educators, each having three to four pupils in the study, were assigned randomly to a repeated curriculum-based measurement/evaluation (experimental) treatment or a conventional special education evaluation (contrast) treatment

Educator performance evaluation systems are a potential tool for improving student achievement by increasing the effectiveness of the educator workforce. For example, recent research suggests that giving more frequent, specific feedback on classroom practice may lead to improvements in teacher performance and student achievement.

The authors have been evaluating the impact of five principal preparation programs in the United States on student outcomes. This information should be considered as one aspect of preparation program improvement and accountability. The study team lays out its recommendations in this policy paper. 

This article describes the procedures and utility of the Benchmarks of Quality as part of a comprehensive evaluation plan to assess the universal level of implementation fidelity of behavior support for a school. However, results can also be examined to determine the level of implementation fidelity across a district or state for ongoing behavioral training and technical assistance planning.

Principal evaluation shares many characteristics with the more general field of personnel evaluation. That is, evaluations may have the purpose of gathering data to help improve performance (formative), or may use the collected information to make decisions about promotion or firing (summative).

This research synthesis examines how teacher effectiveness is currently measured (i.e., formative vs. summative evaluation).

This guide is a tool designed to assist states and districts in constructing high-quality teacher evaluation systems in an effort to improve teaching and learning.

This paper report on work estimating the stability of value-added estimates of teacher effects, an important area of investigation given that new workforce policies implicitly assume that effectiveness is a stable attribute within teachers.

As performance feedback continues to become more commonplace in school settings, it will become increasingly necessary to build capacity around the processes of giving and receiving feedback. Results from this study have implications for how principals can be supported to use their evaluation data.

This paper provide some recommendations to increase the pool of potential teachers, make it tougher to award tenure to those who perform least well, and reward effective teachers who are willing to work in schools serving large numbers of low-income, disadvantaged children. 

Is dismissing poorly performing teachers truly feasible in America today? After all the political capital (and real capital) spent on reforming teacher evaluation, can districts actually terminate ineffective teachers who have tenure or have achieved veteran status?

This article examine the concept of behavioral contingency and describes NCLB as a set of contingencies to promote the use of effective educational practices. Then they describe their design of an alternate assessment, including the components designed to capitalize on the contingencies of NCLB to promote positive educational outcomes. 

The major objective of this data analysis was to estimate the relationship between variables which can be controlled by public policy and educational output. 

This discussion provides a quantitative statement of one approach to achieving the governors’ (and the nation’s) goals – teacher deselection.

This paper reports on the analysis of state statutes and department of education regulations in fifty states for changes in teacher evaluation in use since the passage of No Child Left Behind Act of 2001.

In 2011, as a part of the State Board of Education's implementation of North Carolina's Race to the Top initiative, a sixth standard—a measure of student growth, the Educational Value-Added Assessment System—was added to the existing five standards for evaluating teachers. The purpose of this report is to describe the outcomes of teacher evaluations that have occurred since the sixth standard was added and trends in those outcomes through 2013-14.

This article discusses the current focus on using teacher observation instruments as part of new teacher evaluation systems being considered and implemented by states and districts. 

Direct observation plays an important role in the assessment practices of school psychologists and in the development of evidence-based practices in general and special education. The defining psychometric features of direct observation are presented, the contributions to assessment practice reviewed, and a specific proposal is offered for evaluating the psychometric merit of direct observation in both practitioner developed and commercial/research specific applications.

NSDC opens the door to professional learning that ensures great teaching for every student every day

This book presents clear and functional techniques for deciding what students with learning disabilities should be taught and how. This book can also function as a tool to assist pre-service teachers (students) with deciding how to teach and what to teach to regular/non-special education children.

In recent years, most states have constructed elaborate accountability systems using school-level test scores. We evaluate the implications for school accountability systems. For instance, rewards or sanctions for schools with scores at either extreme primarily affect small schools and provide weak incentives to large ones.

This study used a random-assignment experiment in Los Angeles Unified School District to evaluate various non-experimental methods for estimating teacher effects on student test scores. Having estimated teacher effects during a pre-experimental period, the authors used these estimates to predict student achievement following random assignment of teachers to classrooms.

This report presents an in-depth discussion of the analytical methods and findings from the Measures of Effective Teaching (MET) project’s analysis of classroom observations.1 A nontechnical companion report describes implications for policymakers and practitioners.

This paper combines information from classroom-based observations and measures of teachers' ability to improve student achievement as a step toward addressing these challenges. The results point to the promise of teacher evaluation systems that would use information from both classroom observations and student test scores to identify effective teachers.

Considers design issues and strategies by comparative outcome studies, including the conceptualization, implementation, and evaluation of alternative treatments; assessment of treatment-specific processes and outcomes; and evaluation of the results. It is argued that addressing these and other issues may increase the yield from comparative outcome studies and may attenuate controversies regarding the adequacy of the demonstrations.

Data Evaluation of 285 previous chapters have discussed fundamental issues about assessment and experimental design for single-case research. I mentioned that the third component of methodology after assessment and design is data evaluation. The three components work in concert to permit one to draw inferences about the intervention.

Substantial variation was found in the relationship of evaluators' ratings of teachers and value-added measures of the average achievement of the teachers' students. The results did not yield a simple explanation for the differences in validity of evaluators' ratings. Instead, evaluators' decisions were found to be a complex and idiosyncratic function of motivation, skill, and context.

The purpose of this study was to examine the validity of teacher evaluation scores that are derived from an observation tool, adapted from Danielson's Framework for Teaching, designed to assess 22 teaching components from four teaching domains.

the present study was designed to learn more about how to strengthen the integrity of the problem-solving process

In this article, guidelines for evaluating the technical adequacy of that process are described. The guidelines highlight the interdependencies among assessment functions, subsumed by the goal of helping, and the role of structural factors (e.g., collaboration) in shaping the meaningfulness, appropriateness, and usefulness of the assessment-intervention process.

The authors show school and district-level administrators how to set the priorities and support the practices that will help all teachers become expert teachers. Their five-part framework is based on what research tells us about how expertise develops. 

Building on the analysis that was first reported in School Leadership That Works, the authors of Balanced Leadership identify the 21 responsibilities associated with effective leadership and show how they relate to three overarching responsibilities: 

This review addresses two different proposals for reforming teacher training, neither of which is grounded in research. The authors report three weak correlations between the performance of program participants and TNTP’s certification evaluation rubric. The report concludes with three self-evident aphorisms: practice improves teaching, teachers who master teaching skills do better, and inadequate performers should be weeded out. 

States and districts across the country are revising how they evaluate school principals. Those that are doing so face a substantial challenge: there is scant evidence on the validity and reliability of current principal evaluation tools. Pennsylvania is among states that are developing a new tool for evaluating principals and assistant principals.

In December 2016, Bellwether Education Partners and The Thomas B. Fordham Institute independently released two reports centered on teacher evaluation and its consequences. Both reports offer a glimpse into ongoing challenges and opportunities with teacher evaluation reform, but they have very different analyses. 

Full and durable implementation of school-based interventions is supported by regular evaluation of fidelity of implementation. Multiple assessments have been developed to evaluate the extent to which schools are applying the core features of school-wide positive behavioral interventions and supports (SWPBIS).

Under new frameworks, districts have better aligned their evaluations with their school-leadership standards and developed nuanced rubrics for evidence-collection and evaluation ratings. They have also altered the role of principal supervisors so that they spend more time in schools working with principals.

The edTPA is a performance assessment of teaching used variously by some state education departments and institutions of higher education for licensure testing and evaluating teaching candidates. The edTPA artifacts include written commentaries, lesson plans, video segments, and samples of student work, based on a series of three to five consecutive lessons.

This report describes how the seven Regional Educational Laboratory (REL) Central states (Colorado, Kansas, Missouri, Nebraska, North Dakota, South Dakota, and Wyoming) evaluate their teacher preparation programs and the changes they are making to improve their approaches to evaluation.

This paper summarizes validity evidence pertaining to several different implementations of the Framework. It is based primarily on reviewing the published and unpublished studies that have looked at the relationship between teacher evaluation ratings made using systems based on the Framework and value-added measures of teacher effectiveness.

This report is concerned with only one of the many causes and dimensions of the problem, but it is the one that undergirds American prosperity, security, and civility.

This study examines the impact of 2 forms of professional development on prekindergarten teachers' early language and literacy practice: coursework and coaching. 

The goal of the study was to examine the effects of coaching or professional development coursework on teacher knowledge and teacher practice.

This research evaluated procedures for training supervisors in a residential setting to provide feedback
for maintaining direct‐service staff members' teaching skills with people who have severe disabilities.

As states and districts consider potential changes to their teacher evaluation systems and policies, this paper seeks to inform those efforts by reviewing the evolution of the teacher evaluation policy movement over the last several years, identifying positive outcomes of new systems and negative consequences, and describing risks that should be considered. 

This handbook advocates a new approach to teacher evaluation as a cooperative effort undertaken by a group of professionals.

In this provocative and persuasive new book, the author asserts that the secret to high performance and satisfaction-at work, at school, and at home—is the deeply human need to direct our own lives, to learn and create new things, and to do better by ourselves and our world.

Donald Campbell was an American social psychologist and noted experimental social science researcher who did pioneering work on methodology and program evaluation. He has also become—posthumously—an unlikely hero of the anti-testing and accountability movement in the United States.

Research using student scores on standardized tests confirms the common perception that some teachers are more effective than others. It also reveals that being taught by an effective teacher has important consequences for student achievement. The best way to assess a teacher's effectiveness is to look at his or her on-the-job performance.

In order to provide accurate estimates of how much teachers affect the achievement of their students, this study used panel data covering over a decade of elementary student test scores and teacher assignment in two contiguous New Jersey school districts.

This review examined the overlap between state-created curriculum evaluation tools and The Hexagon Tool created by the National Implementation Research Network. The author followed systematic procedures while conducting a web search and visiting each state’s department of education website in search of curriculum evaluation tools.

This report is the first in a series by the National Council on Teacher Quality (NCTQ) that examines the current status of states' teacher policies. Updated on a two-year cycle, each will cover a specific area of teacher policy. This report focuses on state teacher policies governing what states require in evaluations of both teachers and principals.

This research considers relationships between student achievement (knowledge and cognitive skill), teacher efficacy (Gibson & Dembo, 1984), and interactions with assigned coaches (self-report measures) in a sample of 18 grade 7 and 8 history teachers in 36 classes implementing a specific innovation with the help of 6 coaches.

The author develop falsification tests for three widely used VAM specifications, based on the idea that future teachers cannot influence students' past achievement. 

This essay seeks to help you put the hard-earned experience of others to use through a set of practical steps, prompts, and tips for matching the right evaluator to your need.

The purpose of this study was to explore the purpose and practices of leadership evaluation as perceived by principals. The researcher wanted to identify the perceived purposes and practices of leadership evaluation as described by nine public school principals, and to respond to the apparent need expressed by administrators to receive substantive feedback.

Teacher evaluations matter a lot—both to teachers and to those holding them accountable. But how can schools measure the performance of all teachers fairly? And what should they do with the results?

Written by one of the leaders in evaluation, Evaluation Thesaurus, Fourth Edition, provides readers with a quick analysis of the leading concepts, positions, acronyms, processes, techniques, and checklists in the field of evaluation.

This book is organized around four dominant interrelated core issues: professional standards, a guide to applying the Joint Committee's Standards, ten alternative models for the evaluation of teacher performance, and an analysis of these selected models. 

A critical review of reading programs requires objective and in-depth analysis. For these reasons, the authors offer the following recommendations and procedures for analyzing critical elements of programs.

A multiple baseline across items design was used to evaluate the effects of a cover, copy, and compare (CCC) intervention on students' accuracy in identifying states on a map of the United States. The results showed the CCC intervention was effective in increasing the class mean accuracy levels in locating states.

This annual publication provides a compilation of statistical information covering the broad field of education from prekindergarten through graduate school. It has been published annually since 1962, providing over 50 years of data with which to benchmark education performance at the system level in this country.

Observational measurement of treatment adherence has long been considered the gold standard. However, little is known about either the generalizability of the scores from extant observational instruments or the sampling needed. Results suggested that reliable cognitive–behavioral therapy adherence studies require at least 10 sessions per patient, assuming 12 patients per therapists and two coders—a challenging threshold even in well-funded research. Implications, including the importance of evaluating alternatives to observational measurement, are discussed.

This paper presents the results of a rigorous experiment examining the impact of pay for performance on student achievement and instructional practice.

We report findings from a quasi-experimental evaluation of the recently implemented US$5,000 retention bonus program for effective teachers in Tennessee’s Priority Schools. We estimate the impact of the program on teacher retention using a fuzzy regression discontinuity design by exploiting a discontinuity in the probability of treatment conditional on the composite teacher effectiveness rating that assigns bonus eligibility.

Bill & Melinda Gates Foundation launched the Intensive Partnerships for Effective Teaching initiative. The initiative's goal is dramatic gains in student achievement, graduation rates, and college-going, especially for LIM students. 

This paper offers evidence that evaluation can shift the teacher effectiveness distribution through a different mechanism: by improving teacher skill, effort, or both in ways that persist long-run.

In the research reported here, the authors study one approach to teacher evaluation: practice-based assessment that relies on multiple, highly structured classroom observations conducted by experienced peer teachers and administrators. 

This report proposes six design standards that any rigorous and fair teacher evaluation system should meet. It offers a blueprint for better evaluations that can help every teacher succeed in the classroom—and give every student the best chance at success. 

The authors examine the causes and consequences of the status of teacher evaluation and its implications for the current national debate about performance pay for teachers. The report also examines a number of national, state, and local evaluation systems that offer potential alternatives to current practice.

Provides 2 hypothetical case examples that illustrate how single-case designs (alternating treatments, multiple baseline, and reversal) can be used to evaluate manipulable influences on school performance. In each case, an intervention plan is proposed for a student, and the success of the intervention is evaluated within a single-case design.

This article presents a brief description of a manual called Getting to outcomes: methods and tools for planning, evaluation, and accountability (GTO) designed to assist practitioners in formulating the planning, implementation, and evaluation strategies for programs and policies. 

In this article, we report the results from a randomized evaluation of the Safe and Civil Schools (SCS) model for school-wide positive behavioral interventions and supports. Thirty-two elementary schools in a large urban school district were randomly assigned to an initial training cohort or a wait-list control group.

This report examines the pervasive and longstanding failure to recognize and respond to variations in the effectiveness of teachers. 

The purpose of this study was to assess the degree to which behavioral intervention studies conducted with persons with mental retardation operationally defined the independent variables and evaluated and reported measures of treatment integrity. The study expands the previous work in this area reported by Gresham, Gansle, and Noell (1993) and Wheeler, Baggett, Fox, and Blevins (2006) by providing an evaluation of empirical investigations published in multiple journals in the fields of applied behavior analysis and mental retardation from 1996–2006. Results of the review indicated that relatively few of the studies fully reported data on treatment integrity.

The current review sought to describe the implementation and evaluation of trauma-focused school practices as represented in the published literature. Through a systematic literature search, we identified 39 articles describing trauma-focused practices implemented in school settings with elementary populations and coded data regarding these interventions’ characteristics as well as their implementation and evaluation procedures.