Education Drivers

Value Added

Value-added modeling (VAM) is a statistical approach that provides quantitative performance measures for monitoring and evaluating schools and other aspects of the education system. VAM comprises a collection of complex statistical techniques that use standardized test scores to estimate the effects of individual schools or teachers on student performance. Although the VAM approach holds promise, serious technical issues have been raised regarding VAM as a high-stakes instrument in accountability initiatives. The key question remains: Can VAM scores of standardized test scores serve as a proxy for measuring teaching quality? To date, research on the efficacy of VAM is mixed. There is a body of research that supports VAM, but there is also a body of studies suggesting that model estimates are unstable over time and subject to bias and imprecision. A second issue with VAM is the sole use of standardized tests as a measure of student performance. Despite these valid concerns, VAM has been shown to be valuable in performance improvement efforts when used cautiously in combination with other measures of student performance such as end-of-course tests, final grades, and structured classroom observations.

Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness

(Wing Institute Original Paper)

Value-Added Overview PDF

Cleaver, S., Detrich, R. & States, J. (2020). Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Oakland, CA: The Wing Institute. https://www.winginstitute.org/staff-value-added.

  

One goal of education is to produce students who make measurable academic progress each year. National policy (e.g, No Child Left Behind) has been built around the idea that schools consistently produce students who demonstrate increasing mastery of content. One goal of education research is to understand what educational factors (e.g., teachers, students, class size) contribute to student learning. When it comes to identifying teacher effectiveness, the core question is, how much do test scores accurately reflect a teacher’s contribution to student learning?

            The question of how teachers contribute to student achievement is crucial because teachers are the most important school-based factor in student achievement (Rivkin, Hanushek, & Kain, 2005; Sanders & Horn, 1998). Standardized test scores are related to how far students progress in school and how much they earn later in life (Chetty et al., 2011; Hanushek & Woessman, 2008; Lazear, 2003; Murnane, Willett, Duhaldeborde, & Tyler, 2000)[C1] [SC2] , so it is worth considering these types of data when looking at teacher impact. However, it has been difficult to distinguish between effective and less effective teachers in relation to raising student achievement (Toch & Rothman, 2008; Weisberg et al., 2009).

Currently, information on how teachers impact individual student results is used by teacher evaluation programs at the local and system levels. A teacher’s impact on student achievement may influence teacher retention, pay, and incentives. The problem is how to capture the growth that an individual teacher has on a student’s performance outside of other factors, such as socioeconomic status or class size, that could also impact those gains. Value-added modeling (also known as value-added measurement, value-added assessment) is an attempt to address that concern. Value-added modeling is a method of evaluation that attempts to measure a teacher’s contribution to student achievement in a given year by isolating the value added, or contribution, of the teacher and compare it to the value added of other teachers. These often involve analyzing data from multiple years and in comparison with other teachers’ data. The purpose of this paper on value-added research in education is to define this type of research, provide an overview of how it has been conducted, and discuss its benefits and limitations.

 

Ways to Measure Student Achievement

There are various ways to evaluate student achievement. Status models use the test scores of one group of students at one time. These compare the achievement of a group of students on one exam compared with the results that are expected on that assessment (Koretz, 2008). Cohort-to-cohort change models use change in achievement exams over time; for example, comparing the percentage of students who are proficient in an exam in one year with previous proficiency rates to see if there has been improvement (Koretz, 2008). In contrast, value-added models are based on individual student growth across a year of education. In this way, value-added models attempt to demonstrate student achievement, and thereby teacher effectiveness, by using information from the same group of students across time, something that the status and cohort-to-cohort change models cannot do (Koretz, 2008).

Value-added research in education involves using statistical models that control, or remove the influence, of some variables in order to isolate the effect of a teacher on his or her students’ learning (Steele, Hamilton, & Stecher, 2010). Value-added research involves statistical modeling that takes students’ test scores and, sometimes, school characteristics to create value-added scores for teachers and schools (Braun, 2015).

The purpose of value-added modeling is to show relative effectiveness in improving student scores. Value-added measures are also generally regarded as more effective than indicators based on student characteristics (e.g., student proficiency in a subject).

 

Value-Added Measures in Educational Research

Value-added measures use complicated formulas that take into account multiple factors (e.g., past and current test scores) to show how effective teachers are at producing student growth, while keeping other factors constant (e.g., assignment of students to classrooms) (David, 2010; McCaffrey & Lockwood, 2011).

When researchers measure a teacher’s ability to produce future achievement in students, they find large differences between teachers. Typical teacher quality variables (e.g., education level, licensure, experience) explain little of the variation in the results that teachers produce with students (Hanushek & Rivkin, 2010). Because observable teacher differences rarely explain teacher quality, other aspects are necessary to determine teacher quality.

Koretz (2008) has argued that, in addition to looking at how much students learn, value-added measures should take into account students’ rate of learning, or how quickly they master new skills or learn new information. Students advance through school with various student characteristics (e.g., disability, initial aptitude) and external factors (e.g., parental income) that impact their rate of learning. For example, students who know more about a topic learn new information at a faster rate than those who don’t know as much about the topic. Value-added measures, including models that take multiple years of historical student data into account, can use that information to determine a teacher’s effectiveness.

 

History of Value-Added Measures in Education

Because teacher evaluation efforts tend to focus on rewarding teachers for their contribution to student learning, it is necessary to build evaluation systems that measure student performance (Steele et al., 2010). These efforts, combined with competitive federal programs (e.g., Race to the Top) and philanthropic efforts (e.g., Bill and Melinda Gates Foundation’s Empowering Effective Teachers [EET] initiative) are shaping how state and districts recruit, evaluate, reward, and develop teachers (Steele et al., 2010). One notable example of value-added measures is the Tennessee Value-Added Assessment System (TVAAS).

 

Tennessee Value-Added Assessment System (TVAAS)

In Tennessee, education reform started with the 1984 Comprehensive Education Reform Act, which, among other efforts, provided merit pay. At the same time, two statisticians at the University of Tennessee were working to gauge the feasibility of a statistical model that would eliminate the impediments (e.g., missing student records, different modes of teaching, teacher turnover) to using student achievement data to understand education outcomes of students in grades 2 through 5 in Knox County, Tennessee. The findings indicated the following:

  • There were measurable differences among schools and teachers in regard to student learning.
  • The effects of school and teachers on student achievement were consistent from year to year.
  • Teacher effects were not school specific, meaning that a gain could not be predicted based on the location of the school.
  • There was a strong positive correlation between the teacher’s effect on student achievement and the teacher’s principal evaluation, even though the evaluations were more subjective.
  • Student gains were not related to the initial ability or achievement levels of the students at the start of the school (McLean & Sanders, 1984).

            Future research on TVAAS indicated that academic gains were unrelated to socioeconomic status (i.e., free and reduced lunch), race or ethnicity, or the mean achievement level of the school (Sanders & Horn, 1998).

The focus on growth (i.e., using students’ information across years and creating a model that made the students their own comparison group) gave researchers the ability to see how students grew over time and how much of the growth was produced by teacehrs (Sanders & Horn, 1998). This study, together with other studies that replicated the findings, indicated that statistical models could be used to isolate the impact of teachers on student progress.

 

Challenges of Value Added Research

Challenges in value-added research relate to trustworthiness, reliability, validity, and usefulness[RD3] [SC4] . These challenges also apply to other measures of student progress (Betebenner, 2009). However, when value-added models are used to make high-stakes decisions, like employment and pay incentives, it is important to have data that teachers and school leaders can trust and use effectively.

 

Trustworthiness Considerations

Research is deemed trustworthy when it to demonstrates value and allows for external judgments about procedures and findings, specifically that they are objective and unbiased.

Bias is a concern in value-added modeling. For example, Rothstein (2008) worried that test score gains are biased because they lack random assignment; students are not randomly assigned to teachers. Comparing teachers who have large numbers of students with behavior or learning concerns with teachers who have classrooms of higher achieving students is problematic as scores favor the teachers with higher achieving students.

There is also a concern that the choice of test can impact a teacher’s value-added score (McCaffrey, 2012). In analyzing value-added ratings of middle school teachers using two different math subtests, RAND researchers found discrepancies in teachers’ effectiveness depending on the subtest used (Lockwood et al., 2006). If score results are a function of the test used and not teacher behavior, this is a significant concern. It is important to choose tests that are a direct measure of what school leaders want to assess, and to be aware of any assessments that are imperfect and how they impact the interpretation of value-added measures (Lockwood et al., 2006; McCaffrey, 2012).

Stability over time is also an issue that impacts trustworthiness. In considering whether value-added analyses identify the same teachers as effective every year, Goldhaber and Hansen (2008) examined a large data set from North Carolina and found that estimates of teacher effectiveness in reading and math were not the same across years. Similarly, other researchers have questioned whether it is possible to compare gains from one year to the next using tests that may not include the same content (Koretz, 2008). This suggests that effectiveness is not a fixed quality and can vary over time depending, for example, on the makeup of a teacher’s class or the teacher’s years of experience.

 

Reliability Considerations  

Reliability is the extent to which scores are consistent across repeated measures and are free of measurement errors (AERA, APA, & NCME, 1999). Put another way, reliability lies in the consistency of tests.

Internal reliability (or internal consistency) is the extent to which test items measure the same construct, or concept or topic, that is being investigated (Crocker & Algina, 1986). For example, tests might measure student mastery of math concepts or progress in reading. Internal reliability is also the degree to which similar results occur under consistent testing conditions. When tests have internal reliability, they are accurate and consistent from one testing session to the next. When expressed quantitatively, reliability scores that are above 0.8 are considered acceptable, and scores above 0.9 quite reliable. The higher the reliability score the better, although a minimum reliability score of 0.5 or 0.6 might be considered for some assessments. In their research, Steele et al. (2010) identified three reliability considerations related to value-added modeling:

  • Internal consistency of student assessment scores
  • Consistency of ratings by individuals scoring assessments
  • Consistency of the estimates of the value-added measures generated from student scores.

            As educational research develops, larger longitudinal data sets are available to work with. These data have been used to evaluate educational inputs on student achievement. One concern is that missing data may impact the results of these value-added models. In one analysis of a large data set from an urban U.S. school district, analyzing data that included missing data (e.g., one year of student test scores) had little impact on the estimated teacher effects (McCaffrey & Lockwood, 2011).

 

Validity Considerations

Validity is how well the evidence from a test supports the interpretation of test scores and, in turn, the use of the test (AERA, APA, & NCME, 1999). In other words, it refers to the accuracy of an inference drawn from the results of a test or how well the assessment aligns with course content or what students are learning.

Concern about validity addresses the core questions of value-added modeling: To what extent do changes in students’ performance on an assessment reflect their accurate understanding of the content? How much do test scores accurately reflect the teacher’s contribution to student learning? Various aspects of instruction (e.g., teaching test taking strategies) may contribute to changes in performance on an assessment more than knowledge in content knowledge (e.g., Koretz, 2008; Koretz & Barron, 1998). This would artificially inflate student scores and impact teacher value-added scores.

Inconsistencies in the content of a test could impact the validity of an inference about student growth (McCaffrey, Lockwood, Koretz, & Hamilton, 2003). For example, gauging students’ growth in science knowledge overall by using tests in biology and then chemistry might lead to an invalid inference if the students have different levels of knowledge in the two sciences. This concern can also arise when students take tests in the same course, for example, two math tests focusing on different standards (Martineau, 2006).

Another threat to the validity of value-added modeling lies in attributing student performance to an individual teacher when the assessment covers material from multiple courses, for example, SAT or ACT tests that are cumulative across high school.

 

Usefulness Considerations

Usefulness relates to how value-added measures are received and used by teachers and school leaders. Principals have demonstrated skepticism about the usefulness of value-added measures compared with observational data, specifically in timing, validity, and utility for teachers (Goldring et al., 2015).

Teachers have also expressed concern about the lack of transparency of value-added scores (Jiang, Sporte, & Luppescu, 2015). In a study of Chicago Public Schools’ Recognizing Educators Advancing Chicago (REACH) program, which used both test scores and teacher observations, teachers—especially special education teachers—expressed concern about the reliance on test scores, which is a concern of value-added and other models (Jiang et al., 2015).

While teacher evaluation methods do have their fair share of concerns (Cleaver, Detrich, & States, 2018), the effort to find more objective evaluation systems has helped efforts to increase the usefulness of value-added measures. Value-added measures are one way to measure teacher effectiveness, and therefore may be useful for understanding teacher impacts over time. There are moderate correlations between value-added measures and other measures of teacher effectiveness (e.g., principal evaluations). For principal surveys, the correlation between value-added measures and principal surveys was 0.32 (Jacob & Lefgren, 2008). In another study, the correlation between value-added measure and principal assessment in teachers’ math scores was 0.41, and in teachers’ reading scores 0.44 (Harris & Sass, 2014). In their analysis of 201 teachers across grades 2 through 6, Jacob & Lefgren (2008) found that value-added models did a slightly better job of predicting future test scores, but both observation and value-added measures were able to predict which teachers would be in the top and bottom 20% the following year. These findings were replicated by studies of other evaluation systems across the country, for example, in Cincinnati and in Washoe County, Nevada (Milanowski, Kimball, & White, 2004).

 

Lingering Questions About Value-Added Modeling

No model answers all questions. In value-added modeling, the questions that remain include the following:

  • What does it mean to be a good teacher and how is that measured?
  • What outcomes do we want from education? For example, are there outcomes that cannot be measured on a standardized assessment that are just as important standardized test outcomes (Ruzek, Domina, Conley, Duncan, & Karabenick, 2015)?
  • Do good teachers succeed with all types of students or only certain types (Condie, Lefgren, & Sims, 2014)?
  • Is it meaningful to compare gains across grades; for example, across grades 3 and 6, even if the general content is the same (Everson, 2017)?

            There are also limitations to how value-added modeling demonstrates the impact of teachers on students who are learning English or who have disabilities. Specifically, an analysis by the Carnegie Foundation found that the value-added results for teachers who teach inclusive classrooms change very slightly whether or not students with disabilities are included (McCaffrey & Buzick, 2014). However, they also identified the following concerns with applying value-added modeling to teachers who exclusively teach students with disabilities:

  • The scores of students with disabilities can be low which may attribute those lower scores to their teachers’ performance.
  • Lower student test scores can also increase the random errors in value-added models.
  • The use of testing accommodations from year to year can create variability in growth that may be attributed to teachers incorrectly.
  • States and districts that use value-added models may find it helpful to monitor the proportion of students with disabilities in classes along with other evidence of systematic error so they can revise the models as needed (McCaffrey & Buzick, 2014).

 

Recommendations for Value-Added Measures

The use of value-added measures is relatively new in education research. Because these measures are being used to make decisions that impact teachers, decision makers should carefully consider the following factors (Martineau, 2006):

  • Reliability of assessment measures used (AERA, APA, & NCME, 1999),
  • Validity of inferences drawn from value-added measures,
  • How well student assessments compare from year to year.

            While value-added measures provide some useful information about differences in teacher performance, individual scores suffer from variance and low stability as well as undetermined bias (Braun, 2015). The American Statistical Association (2014) has recommended caution in using value-added measures for evaluation or high-stakes purposes. Because high-quality evaluations based on observation of teacher practice can provide information about teacher effectiveness, effort should be put into training teachers and principals in teacher evaluation (Braun, 2015). Both teacher observation and value-added measures provide usable information about teacher quality (Milanowski, Kimball, & White, 2004). However, a remaining question about value-added measures is whether high-quality, large-scale observation protocols can be achieved and maintained (Jiang et al., 2015).

 

Conclusion

Value-added models have expanded our ability to analyze teachers’ impact on student achievement. From value-added research, we know that there is variation in teacher performance (Aaronson, Barrow, & Sander, 2007; Rivkin et al., 2005) and that value-added models can capture the effect of teachers on student achievement (Hanushek & Rivkin, 2010; McCaffrey & Buzick, 2014; Sanders & Horn, 1998). There are concerns related to reliability, validity, efficacy, and usefulness that should be taken into account before designing and implementing a plan for evaluating teachers using value-added measures across a district. In addition, questions remain about the precision and practicality of value-added measures in schools, specifically whether value-added measures can address core questions about quality teaching, and the practicality of value-added modeling compared with observation protocols.

 

Citations

Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the Chicago Public High Schools. Journal of Labor Economics, 25(1), 95–135.

American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME). (1999). The standards for educational and psychological testing.Washington, DC: AERA Publications.

American Statistical Association. (2014). ASA statement on using value-added models for educational assessments.Alexandria, VA: Author. Retrieved from https://www.amstat.org/asa/files/pdfs/POL-ASAVAM-Statement.pdf

Betebenner, D. (2009). Growth, standards and accountability. Dover, NH: National Center for the Improvement of Educational Assessment. Retrieved from https://www.nciea.org/sites/default/files/publications/growthandStandard_DB09.pdf

Braun, H. (2015). The value in value added depends on the ecology. Educational Researcher, 44(2), 127–131. Retrieved from https://doi.org/10.3102%2F0013189X15576341

Chetty, R., Friedman, J., Hilger, N., Saez, E., Schanzenbach, D. W., & Yagan, D. (2011). How does your kindergarten classroom affect your earnings? Evidence from Project STAR. Quarterly Journal of Economics, 126(4), 1593–1660.

Cleaver, S., Detrich, R., & States, J. (2018). Overview of teacher formal evaluation. Oakland, CA: The Wing Institute. Retrieved from https://www.winginstitute.org/teacher-evaluation-formal

Condie, S., Lefgren, L., & Sims, D. (2014). Teacher heterogeneity, value-added and education policy. Economics of Education Review, 40, 76–92.

Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.

David, J. L. (2010). What research says about using value-added measures to evaluate teachers. Educational Leadership, 67(8), 81–82.

Everson, K. C. (2017). Value-added modeling and educational accountability: Are we answering the real questions? Review of Educational Research, 87(1), 35–70.

Goldhaber, D., & Hansen, M. (2008). Is it just a bad class? Assessing the stability of measured teacher performance.Working paper 2008-5. Seattle: Center on Reinventing Public Education, University of Washington.

Goldring, E., Grisson, J. A., Rubin, M., Neumerski, C. M., Cannata, M., Drake, T., Schuermann, P. (2015). Make room value added: Principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher, 44(2), 96–104.

Hanushek, E. A. & Rivkin, S. G. (2010). Generalizations about using value-added measures of teacher quality. American Economic Review100(2), 267–271.

Hanushek, E. A., & Woessman, L. (2008). The role of cognitive skills in economic development. Journal of Economic Literature, 46(3), 607–668. Retrieved from: http://hanushek.stanford.edu/sites/default/files/publications/Hanushek%2BWoessmann%202008%20JEL%2046%283%29.pdf

Harris, D. N., & Sass, T. R. (2014). Skills, productivity, and the evaluation of teacher performance. Economics of Education Review, 40, 183–204.

Jacob, B. A., & Lefgren, L. (2008). Can principals identify effective teachers? Evidence on subjective performance evaluation in education. Journal of Labor Economics, 26(1), 101–136.

Jiang, J. Y., Sporte, S. E., & Luppescu, S. (2015). Teacher perspectives on evaluation reform: Chicago’s REACH students. Educational Researcher, 44(2), 105–116.

Koretz, D. (2008). A measured approach: Value-added models are a promising improvement, but no one measure can evaluate teacher performance. American Educator, 32(3), 18–39.

Koretz, D., & Barron, S. (1998) The validity of gains on the Kentucky Instructional Results Information System (KIRIS). Santa Monica, CA: RAND Corporation.

Lazear, E. P. (2003). Teacher incentives. Swedish Economic Policy Review, 10(3), 179–214.

Lockwood, J. R., McCaffrey, D. F., Hamilton, L. S., Stecher, B. Vi-Nhuan, L., & Martinez, F. (2006). The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/reports/2009/RAND_RP1269.pdf

Martineau, J. A. (2006). Distorting value-added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavioral Statistics, 31(1), 35–62.

McCaffrey, D. (2012). Do value-added methods level the playing field for teachers? Carnegie Knowledge Network. Stanford, CA: Carnegie Foundation for the Advancement of Teaching. Retrieved from http://www.carnegieknowledgenetwork.org/wp-content/uploads/2013/06/CKN_2012-10_McCaffrey.pdf

McCaffrey, D. F., & Buzick, H. (2014). Is value-added accurate for teachers of students with disabilities? Carnegie Foundation for the Advancement of Teaching. Retrieved from: http://www.carnegieknowledgenetwork.org/wp-content/uploads/2014/01/CKN_McCaffrey_Disabilities_Fourth_formatted.pdf

McCaffrey, D. F., & Lockwood, J. R. (2011). Missing data in value-added modeling of teacher effects. Annals of Applied Statistics, 5(2A), 773–797. Retrieved from: https://projecteuclid.org/euclid.aoas/1310562205

McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003). Evaluating value-

added models for teacher accountability. RAND Corporation: Santa Monica, CA. Retrieved from: https://www.rand.org/content/dam/rand/pubs/monographs/2004/RAND_MG158.pdf

McLean, R. A., & Sanders, W. L. (1984). Objective component of teacher evaluation: A feasibility study. Working paper No. 199. Knoxville, TN: University of Tennessee, College of Business Administration.

Milanowski, A., Kimball, S., & White, B. (2004). The relationship between standards-based teacher evaluation scores and student achievement: Replication and extensions at three sites. Madison, WI: Consortium for Policy Research in Education, University of Wisconsin.

Murnane, R. J., Willett, J. B., Duhaldeborde, Y. & Tyler, J. H. (2000). How important are the cognitive skills of teenagers in predicting subsequent earnings? Journal of Policy Analysis and Management. 19(4), 547–568.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458.

Rothstein, J. (2008). Teacher quality in educational production: Tracking, decay, and student achievement. Working paper No. 14442. Cambridge, MA: National Bureau of Economic Research.

Ruzek, E. A., Domina, T., Conley, A. M., Duncan, G. J., & Karabenick, S. A. (2015). Using value-added models to measure teacher effects on students’ motivation and achievement. Journal of Early Adolescence, 35(5–6), 852–882.

Sanders, W. L., & Horn, S. P. (1998). Research findings from the Tennessee Value-Added Assessment System (TVAAS) database: Implications for educational evaluation and research. Journal of Personnel Evaluation in Education, 12(3), 247–256.

Steele, J. L., Hamilton, L. S., & Stecher, B. M. (2010). Incorporating student performance measures into teacher evaluation systems. Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/technical_reports/TR917.html

Toch, T., & Rothman, R. (2008). Rush to judgment: Teacher evaluation in public education. Washington, DC: Education Sector.

Weisberg, D., Sexton, S., Mulhern, J., Keeling, D., Schunk, J., Palcisco, A., & Morgan, K.

 (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. New York, NY: New Teacher Project.

 

Publications

TITLE
SYNOPSIS
CITATION
The Value of Interrupted Time-Series Experiments for Community Intervention Research

This paper advocates the use of time-series experiments for the development and evaluation of community interventions.

Biglan, A., Ary, D., & Wagenaar, A. C. (2000). The value of interrupted time-series experiments for community intervention research. Prevention Science1(1), 31-49.

Overview: Formal Teacher Evaluation

The purpose of this overview is to provide information about the role of formal teacher evaluation, the research that examines the practice, and its impact on student outcomes.

Cleaver, S., Detrich, R. & States, J. (2018). Overview of Teacher Formal Evaluation. Oakland, CA: The Wing Institute.https://www.winginstitute.org/teacher-evaluation-formal.

Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness

The purpose of this paper on value-added research in education is to define this type of research, provide an overview of how it has been conducted, and discuss its benefits and limitations.

Cleaver, S., Detrich, R. & States, J. (2020). Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Oakland, CA: The Wing Institute. https://www.winginstitute.org/staff-value-added.

  

Overview of value-added research in education: Reliability, validity, efficacy, and usefulness.

The purpose of this paper on value-added research in education is to define this type of research, provide an overview of how it has been conducted, and discuss its benefits and limitations.

Cleaver, S., Dietrich, R. & States, J. (2020). Overview of value-added research in education: Reliability, validity, efficacy, and usefulness. The Wing Institute.

 

Data Mining

TITLE
SYNOPSIS
CITATION
What is the relationship between teacher working conditions and school performance?
This inquiry looks at the effect of time on the job and the quality of a teacher's skills.
Keyworth, R. (2010). What is the relationship between teacher working conditions and school performance? Retrieved from what-is-relationship-between882.
TITLE
SYNOPSIS
CITATION
Teachers and student achievement in the Chicago Public High Schools

The authors estimate the importance of teachers in Chicago public high schools using matched student-teacher administrative data. 

Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the Chicago public high schools. Journal of labor Economics25(1), 95-135.

Coaching side by side: One-on-one collaboration creates caring, connected teachers

This article describes a school district administrator's research on optimal coaching experiences for classroom teachers. This research was done with the intent of gaining a better understanding of how coaching affects student learning. 

Akhavan, N. (2015). Coaching side by side: One-on-one collaboration creates caring, connected

teachers. Journal of Staff Development, 36,34-37.

 

The effectiveness of a technologically facilitated classroom-based early reading intervention: The targeted reading intervention

The purpose of this study was to evaluate the efficacy of a classroom-teacher-delivered reading intervention for struggling readers called the Targeted Reading Intervention (TRI), designed particularly for kindergarten and first-grade teachers and their struggling students in rural, low-wealth communities. 

Amendum, S. J., Vernon-Feagans, L., & Ginsberg, M. C. (2011). The effectiveness of a technologically facilitated classroom-based early reading intervention: The targeted reading intervention. The Elementary School Journal112(1), 107-131.

Teachers Matter: Evidence from Value-Added Assessments.

Value-added assessment proves that very good teaching can boost student learning and that family background does not determine a student's destiny. Students taught by highly effective teachers several years in a row earn higher test scores than students assigned to particularly ineffective teachers.

American Education Research Association (AERA). (2004). Teachers matter: Evidence from value-added assessments. Research Points, 2(2). Retrieved from http://www.aera.net/ Portals/38/docs/Publications/Teachers%20Matter.pdf

ASA statement on using value-added models for educational assessment

Value-Added Models (VAMs) has been embraced by many states and school districts as part of educational accountability systems. Value-Added Assessment (VAA) Models attempt to estimate effects of individual teachers or schools on student achievement while accounting for differences in student background. This paper provides a summary of the American Statistical Associations analysis of the efficacy of value-added modeling in education.

American Statistical Association. (2014). ASA statement on using value-added models for educational assessment. Alexandria, VA.

Instructional Coaching: Professional development strategies that improve instruction

This article discusses instructional coaching as well as the eight factors that can increase the likelihood that coaching will be a real fix for a school. Instructional coaching holds much potential for improving the way teachers teach and the way students learn, but that potential will only be realized if leaders plan their coaching program with care. 

Annenburg Institute for School Reform. (2004). Instructional Coaching: Professional development strategies that improve instruction. 

Evaluating the impact of performance-related pay for teachers in England.

This paper evaluates the impact of a performance-related pay scheme for teachers in England. 

Atkinson, A., Burgess, S., Croxson, B., Gregg, P., Propper, C., Slater, H., & Wilson, D. (2009). Evaluating the impact of performance-related pay for teachers in England. Labour Economics16(3), 251-261.

The need for assessment of maintaining variables in OBM

The authors describe three forms of functional assessment used in applied behavior analysis and explain three potential reasons why OBM has not yet adopted the use of such techniques.

Austin, J., Carr, J. E., & Agnew, J. L. (1999). The need for assessment of maintaining variables in OBM. Journal of Organizational Behavior Management19(2), 59-87.

What Do Surveys of Program Completers Tell Us About Teacher Preparation Quality?

This study uses statewide completer survey data from North Carolina to assess whether perceptions of preparation quality and opportunities to learn during teacher preparation predict completers’ value-added estimates, evaluation ratings, and retention.

Bastian, K. C., Sun, M., & Lynn, H. (2018). What do surveys of program completers tell us about teacher preparation quality? Journal of Teacher Education, November 2019.

Questioning the Author: An approach for enhancing student engagement with text

The book presents many examples of Questioning the Author (QtA) in action as children engage with narrative and expository texts to construct meaning.

Beck, I. L., & McKeown, M. G., Hamilton, R. L., & Kugan, L. (1997). Questioning the Author: An approach for enhancing student engagement with text.Newark, DE: International Reading Association.

 

Questioning the Author: An approach for enhancing student engagement with text

The book presents many examples of Questioning the Author (QtA) in action as children engage with narrative and expository texts to construct meaning.

Beck, I. L., & McKeown, M. G., Hamilton, R. L., & Kugan, L. (1997). Questioning the Author: An approach for enhancing student engagement with text.Newark, DE: International Reading Association.

 

Growth, Standards and Accountability

This paper introduces analysis techniques and results showing how student growth percentiles, a normative growth analysis technique, can be used to examine the illuminate the relationship between standards based accountability systems and the performance standards on which they are based.

Betebenner, D. (2009). Growth, standards and accountability. Dover, NH: National Center for the Improvement of Educational Assessment. Retrieved from https://www.nciea.org/sites/default/files/publications/growthandStandard_DB09.pdf

Effects of coaching on teachers’ use of function-based interventions for students with severe disabilities

This study used a delayed multiple-baseline across-participants design to analyze the effects of coaching on special education teachers’ implementation of function-based interventions with students with severe disabilities. This study also examined the extent to which teachers could generalize function-based interventions in different situations. 

Bethune, K. S., & Wood, C. L. (2013). Effects of coaching on teachers’ use of function-based interventions for students with severe disabilities. Teacher Education and Special Education, 36(2), 97-114.

 

Assessing the value-added effects of literary collaborative professional development on student learning.

This article reports on a 4-year longitudinal study of the effects of Literacy Collaborative (LC), a schoolwide reform model that relies primarily on the oneon-one coaching of teachers as a lever for improving student literacy learning.

Biancarosa, G., Bryk, A. S., & Dexter, E. R. (2010). Assessing the value-added effects of literacy collaborative professional development on student learning. The elementary school journal111(1), 7-34.

The Value of Interrupted Time-Series Experiments for Community Intervention Research

This paper advocates the use of time-series experiments for the development and evaluation of community interventions.

Biglan, A., Ary, D., & Wagenaar, A. C. (2000). The value of interrupted time-series experiments for community intervention research. Prevention Science1(1), 31-49.

Professional development and teacher learning: Mapping the terrain

Teacher professional development is essential to efforts to improve our schools. This article maps the terrain of research on this important topic. It first provides an overview of what we have learned as a field, about effective professional development programs and their impact on teacher learning. 

Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher30(8), 3–15.

Teacher Preparation and Student Achievement.

This article examined the differences in effectiveness of teacher preparation programs that supply teachers to New York City schools.  One of the important findings is that preparation directly linked to practice benefits teachers in their first year.

Boyd, D. J., Grossman, P. L., Lankford, H., Loeb, S., & Wyckoff, J. (2009). Teacher Preparation and Student Achievement. Educational Evaluation & Policy Analysis, 31(4), 416-440.

The narrowing gap in New York City teacher qualifications and its implications for student achievement in high-poverty schools.

By estimating the effect of teacher attributes using a value-added model, the analyses in this paper predict that observable qualifications of teachers resulted in average improved achievement for students in the poorest decile of schools of .03 standard deviations.

Boyd, D., Lankford, H., Loeb, S., Rockoff, J., & Wyckoff, J. (2008). The narrowing gap in New York City teacher qualifications and its implications for student achievement in high‐poverty schools. Journal of Policy Analysis and Management: The Journal of the Association for Public Policy Analysis and Management27(4), 793-818.

The value in value added depends on the ecology.

These five articles begin to build a bridge between literature. specifically, they report on how the use of the indicators derived from value-added models (VAM) actually payout in practice and give carefully consideration to how the design and implementation of teacher evaluation system could be modified to enhance the positive impact of accountability and mitigate the negative consequences,

Braun, H. (2015). The value in value added depends on the ecology. Educational Researcher, 44(2), 127–131. Retrieved from https://doi.org/10.3102%2F0013189X15576341

Using Student Progress to Evaluate Teachers: A Primer on Value-Added Models. Policy Information Perspective.

This report is a lay person’s guide to value added modeling as a means of evaluating teacher performace.

Braun, H. I. (2005). Using Student Progress to Evaluate Teachers: A Primer on Value-Added Models. Policy Information Perspective. Educational Testing Service. Retrieved from http://files.eric.ed.gov/fulltext/ED529977.pdf

 

The debate about rewards and intrinsic motivation: Protests and accusations do not alter the results.
 

In this paper, the authors show that the questions we asked are fundamental and that our meta-analytic techniques are appropriate, robust, and statistically correct. In sum, the results and conclusions of our meta-analysis are not altered by our critics’ protests and accusations.

Cameron, J., & Pierce, W. D. (1996). The debate about rewards and intrinsic motivation: Protests and accusations do not alter the results. Review of Educational Research, 66(1), 39–51.

Value-added measures: How and why the strategic data project uses them to study teacher effectiveness

This brief explains how and why Strategic Data Project (SDP) uses value-added measures for our diagnostic work. We also explain how value-added measures relate to other measures of teacher effectiveness and the limitations of value-added measures.

Center for Education Policy Research. (2011). Value-added measures: How and why the strategic data project uses them to study teacher effectiveness. Retrieved from https://hwpi.harvard.edu/files/sdp/files/sdp-va-memo_0.pdf

The Long-Term Impacts Of Teachers: Teacher Value-Added And Student Outcomes In Adulthood

This paper examines the issue of efficacy of value-added measures in evaluating teachers. This question is important in understanding whether value-added analysis provides unbiased estimates of teachers’ impact on student achievement and whether these teachers improve long-term student outcomes.

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood (No. w17699). National Bureau of Economic Research.

The Long-Term Impacts Of Teachers: Teacher Value-Added And Student Outcomes In Adulthood

This paper examines the issue of efficacy of value-added measures in evaluating teachers. This question is important in understanding whether value-added analysis provides unbiased estimates of teachers’ impact on student achievement and whether these teachers improve long-term student outcomes.

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood (No. w17699). National Bureau of Economic Research.

Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood

This paper examines the issue of the efficacy of valued-added measures in evaluating the effectiveness of teachers and long term impact on student’s lives.

Chetty, R., Friedman, J. N., & Rockoff, J. E. (in press II). Measuring the impact of teachers II: Evaluating bias in teacher value-added estimates. American Economic Review.

How does your kindergarten classroom affect your earnings? Evidence from Project STAR.

This paper evaluates the long-term impacts of STAR by linking the experimental data to administrative records.

Chetty, R., Friedman, J., Hilger, N., Saez, E., Schanzenbach, D. W., & Yagan, D. (2011). How does your kindergarten classroom affect your earnings? Evidence from Project STAR. Quarterly Journal of Economics, 126(4), 1593–1660.

Overview: Formal Teacher Evaluation

The purpose of this overview is to provide information about the role of formal teacher evaluation, the research that examines the practice, and its impact on student outcomes.

Cleaver, S., Detrich, R. & States, J. (2018). Overview of Teacher Formal Evaluation. Oakland, CA: The Wing Institute.https://www.winginstitute.org/teacher-evaluation-formal.

Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness

The purpose of this paper on value-added research in education is to define this type of research, provide an overview of how it has been conducted, and discuss its benefits and limitations.

Cleaver, S., Detrich, R. & States, J. (2020). Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Oakland, CA: The Wing Institute. https://www.winginstitute.org/staff-value-added.

  

Teacher heterogeneity, value-added and education policy.

This study examines the theoretical and practical implications of ranking teachers with a one-dimensional value-added metric when teacher effectiveness varies across subjects or student types.

Condie, S., Lefgren, L., & Sims, D. (2014). Teacher heterogeneity, value-added and education policy. Economics of Education Review40, 76-92.

Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures for teacher effectiveness in policy and practice

In this report, the author aim to provide an accessible introduction to these new measures of teaching quality and put them into the broader context of concerns over school quality and achievement gaps.

Corcoran, S. P. (2010). Can Teachers Be Evaluated by Their Students' Test Scores? Should They Be? The Use of Value-Added Measures of Teacher Effectiveness in Policy and Practice. Education Policy for Action Series. Annenberg Institute for School Reform at Brown University (NJ1).

Introduction to classical and modern test theory.

This text was written to help the reader acquire a base of knowledge about classical psychometrics and to integrate new ideas into that framework of knowledge.

Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.

Evaluating teacher evaluation: Popular modes of evaluating teachers are fraught with inaccuracies and inconsistencies

Popular modes of evaluating teachers are fraught with inaccuracies and inconsistencies, but the field has identified better approaches. Value-added models enable researchers to use statistical methods to measure changes in student scores over time while considering student characteristics and other factors often found to influence achievement.

Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation: Popular modes of evaluating teachers are fraught with inaccuracies and inconsistencies, but the field has identified better approaches. Phi Delta Kappan, 93(6), 8–15.Retrieved from https://www.edweek.org/ew/articles/2012/03/01/kappan_hammond.html

What research says about using value-added measures to evaluate teachers.

A growing number of researchers are studying whether value-added measures can do a good job of measuring the contribution of teachers to test score growth. Here I summarize a handful of analyses that shed light on two questions.

David, J. L. (2010). What research says about using value-added measures to evaluate teachers. Educational Leadership, 67(8), 81–82. Retrieved from http://www.ascd.org/publications/educational_leadership/may10/vol67/num08/Using_Value-Added_Measures_to_Evaluate_Teachers.aspx

Sharing successes and hiding failures: ‘reporting bias’ in learning and teaching research

This paper examines factors that lead to bias as well offers specific recommendations to journals, funders, ethics committees, and universities designed to reduce reporting bias.

Dawson, P., & Dawson, S. L. (2018). Sharing successes and hiding failures:‘reporting bias’ in learning and teaching research. Studies in Higher Education43(8), 1405-1416.

How important are school principals in the production of student achievement?

As school leaders, principals can influence student achievement in a number of ways, such as: hiring and firing of teachers, monitoring instruction, and maintaining student discipline, among many others. We measure the effect of individual principals on gains in math and reading achievement between grades 4 and 7 using a value-added framework

Dhuey, E., & Smith, J. (2014). How important are school principals in the production of student achievement? Canadian Journal of Economics, 47(2), 634–663.

The Effect of Career and Technical Education on Human Capital Accumulation: Causal Evidence from Massachusetts

Twenty percent of high school students take four or more courses in career and technical education (CTE). Despite this high rate of participation, little is known about what constitutes high-quality CTE and whether high-quality CTE allows participants to accumulate meaningful knowledge and skills to succeed in a career. This study from the Association for Education Finance and Policy examined the impact of participating in CTE on high school attendance, high school completion, professional certifications, and performance on standardized test scores. The evidence suggests that a high-quality CTE program boosts on-time graduation for higher income students and for lower income .

 

Dougherty, S. M. (2016). The effect of career and technical education on human capital accumulation: Causal evidence from Massachusetts. Education Finance and Policy. doi:10.1162/EDFP_a_00224.

Selecting growth measures for school and teacher evaluations: Should proportionality matter?

In this paper we take up the question of model choice and examine three competing approaches. The first approach, (SGPs) framework, eschews all controls for student covariates and schooling environments. The second approach, value-added models (VAMs), controls for student background characteristics and under some conditions can be used to identify the causal effects of schools and teachers. The third approach, also VAM-based, fully levels the playing field so that the correlation between school- and teacher-level growth measures and student demographics is essentially zero. We argue that the third approach is the most desirable for use in educational evaluation systems.

Ehlert, M., Koedel, C., Parsons, E., & Podgursky, M. (2013). Selecting growth measures for school and teacher evaluations: Should proportionality matter?. National Center for Analysis of Longitudinal Data in Education Research, 21.

Value-added modeling and educational accountability: Are we answering the real questions?

Value-added estimates of teacher or school quality are increasingly used for both high- and low-stakes accountability purposes, making understanding of their limitations critical.

Everson, K. C. (2017). Value-added modeling and educational accountability: Are we answering the real questions?. Review of Educational Research87(1), 35-70.

Stand by me: What teachers say about unions, merit pay, and other professional matters

This paper exams teachers' views on unions, tenure, pay-for-performance, alternative certification, and other issues and finds that while most teachers are strong supporters of standards, a sense of vulnerability, along with fears of politics and favoritism, make them loyal to the tenure system, loyal to their unions, and highly skeptical about pay tied to student test scores.

Farkas, S., Johnson, J., & Duffett, A. (2003). Stand by me: What teachers say about unions,

merit pay, and other professional matters. New York: Public Agenda.

Coaching middle-level teachers to think aloud improves comprehension instruction and student reading achievement
In an effort to improve student achievement, a group of middle-school teachers at an underperforming school developed a school-wide literacy plan. As part of the plan, they agreed to model their thinking while reading aloud. Eight teachers were selected for coaching related to thinking aloud in which they exposed students to comprehension strategies that they used while reading. 

Fisher, D., Frey, N., & Lapp, D. (2011). Coaching middle-level teachers to think aloud improves comprehension instruction and student reading achievement. The Teacher Educator, 46(3), 231-243.

Evidence for the need to more closely examine school effects in value-added modeling and related accountability policies

This paper evaluates the reasonableness of assumptions of weighted approaches to account for building level effects when value added modeling is used to evaluate teachers.  In urban schools and ultimately teachers in those schools were negatively impacted by using weighted models to account for building level effects.

Franco, M. S., & Seidel, K. (2014). Evidence for the need to more closely examine school effects in value-added modeling and related accountability policies. Education and Urban Society. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0013124511432306

Strategies for Effective Classroom Coaching

This article aimed to present frameworks and practices coaches can use with classroom teachers to facilitate the implementation of evidence-based interventions in schools.

Garbacz, S. A., Lannie, A. L., Jeffrey-Pearsall, J. L., & Truckenmiller, A. J. (2015). Strategies for effective classroom coaching. Preventing School Failure: Alternative Education for Children and Youth59(4), 263-273.

Is this just a bad class? Assessing the stability of measured teacher performance

This paper report on work estimating the stability of value-added estimates of teacher effects, an important area of investigation given that new workforce policies implicitly assume that effectiveness is a stable attribute within teachers.

Goldhaber, D. D., & Hansen, M. (2008). Is it Just a Bad Class?: Assessing the Stability of Measured Teacher Performance. Seattle, WA: Center on Reinventing Public Education.

Is it just a bad class? Assessing the stability of measured teacher performance.

This paper report on work estimating the stability of value-added estimates of teacher effects, an important area of investigation given that new workforce policies implicitly assume that effectiveness is a stable attribute within teachers. 

Goldhaber, D. D., & Hansen, M. (2008). Is it Just a Bad Class?: Assessing the Stability of Measured Teacher Performance. Seattle, WA: Center on Reinventing Public Education.

Teacher career paths, teacher quality, and persistence in the classroom: Are public schools keeping their best?

In this paper we examine the mobility of early-career teachers of varying quality, measured using value-added estimates of teacher performance.

Goldhaber, D., Gross, B., & Player, D. (2011). Teacher career paths, teacher quality, and persistence in the classroom: Are public schools keeping their best?. Journal of Policy Analysis and Management30(1), 57-87.

Make room value added: Principals’ human capital decisions and the emergence of teacher observation data.

Interview and survey data from six school districts that have recently implemented new evaluation systems with classroom observations provide evidence that principals tend to rely less on test scores in their human capital decisions. 

Goldring, E., Grissom, J. A., Rubin, M., Neumerski, C. M., Cannata, M., Drake, T., & Schuermann, P. (2015). Make room value added: Principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher44(2), 96-104.

The legal and policy implications of value-added teacher assessment policies

This argument makes the case that policies that require states’ student test scores to account for 40-50% of the overall score of teacher performance in value added modeling may produce a significant number of teachers to be falsly identified as ineffective.  Using such value added models may leave school districts legally vulnerable to law suits.

Green, P. C., Baker, B. D., & Oluwole, J. (2012). The legal and policy implications of value-added teacher assessment policies. BYU Educ. & LJ. Retrieved fromhttp://digitalcommons.law.byu.edu/elj/vol2012/iss1/2

What works in professional development?

A research synthesis confirms the difficulty of translating professional development into student achievement gains despite the intuitive and logical connection. Those responsible for planning and implementing professional development must learn how to critically assess and evaluate the effectiveness of what they do.

Guskey, T. R., & Yoon, K. S.(2009). What works in professional development? Phi Delta Kappan.doi: 10.1177003172170909000709.

Reliability and Validity of Inferences about Teachers Based on Student Scores

Policymakers and school administrators have embraced value-added models of teacher effectiveness as tools for educational improvement. Teacher value-added estimates may be viewed as complicated scores. This Paper examines the use of value-added modeling as a tool to identify effective teachers from ineffective instructors.

Haertel, E. H. (2013). Reliability and Validity of Inferences about Teachers Based on Student Scores. William H. Angoff Memorial Lecture Series. Educational Testing Service.

Generalizations about using value-added measures of teacher quality.

The precise method of attributing differences in classroom achievement to teachers is the
subject of considerable discussion and analysis.

Hanushek, E. A., & Rivkin, S. G. (2010). Generalizations about using value-added measures of teacher quality. American Economic Review100(2), 267-71.

The role of cognitive skills in economic development.

This paper reviews the role of cognitive skills in promoting economic well-being, with a particular focus on the role of school quality and quantity.

Hanushek, E. A., & Woessmann, L. (2008). The role of cognitive skills in economic development. Journal of economic literature46(3), 607-68.

Value-added measures in education: What every educator needs to know

In" Value-Added Measures in Education", Douglas N. Harris takes on one of the most hotly
debated topics in education. Drawing on his extensive work with schools and districts, he
sets out to help educators and policymakers understand this innovative approach to
assessment and the issues associated with its use.

Harris, D. N. (2011). Value-Added Measures in Education: What Every Educator Needs to Know. Harvard Education Press. 8 Story Street First Floor, Cambridge, MA 02138.

Skills, productivity, and the evaluation of teacher performance.

The authors examine the relationships between observational ratings of teacher performance, principals’ evaluations of teachers’ cognitive and non-cognitive skills and test-score based measures of teachers’ productivity. 

Harris, D. N., & Sass, T. R. (2014). Skills, productivity and the evaluation of teacher performance. Economics of Education Review40, 183-204.

A Review of value-added models

This paper examines issues of value-added modeling. It describes what value-added modeling is and how it works in education.

Hibpshman, T. L. (2004). A Review of value-added models. Kentucky Education Professional Standards Board.

Do Principals Fire the Worst Teachers?

This paper examines how principals make decisions regarding teacher dismissal. The study estimates the relative weight that school administrators place on a variety of teacher characteristics and finds evidence that principals do consider teacher absences and value-added measures, along with several demographic characteristics, in determining which teachers to dismiss.

Jacob, B. A. (2010). Do principals fire the worst teachers? (No. w15715). National Bureau of Economic Research.

Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education

This paper examines how well principals can distinguish between more and less effective teachers. To put principal evaluations in context, we compare them with the traditional determinants of teacher compensation-education and experience-as well as value-added measures of teacher effectiveness.

Jacob, B. A., & Lefgren, L. (2008). Can principals identify effective teachers? Evidence on subjective performance evaluation in education. Journal of Labor Economics, 26(1), 101-136.

Teacher perspectives on evaluation reform: Chicago’s REACH students.

This study draws on 32 interviews from a random sample of teachers and 2 years of survey data from more than 12,000 teachers per year to measure their perceptions of the clarity, practicality, and cost of the new system.

Jiang, J. Y., Sporte, S. E., & Luppescu, S. (2015). Teacher perspectives on evaluation reform: Chicago’s REACH students. Educational Researcher44(2), 105-116.

Student Achievement through Staff Development

This book provides research as well as case studies of successful professional development strategies and practices for educators.

Joyce, B. R., & Showers, B. (2002). Student achievement through staff development. ASCD.

Estimating teacher impacts on student achievement: An experimental evaluation

This study used a random-assignment experiment in Los Angeles Unified School District to evaluate various non-experimental methods for estimating teacher effects on student test scores. Having estimated teacher effects during a pre-experimental period, the authors used these estimates to predict student achievement following random assignment of teachers to classrooms.

Kane, T. J., & Staiger, D. O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation (No. w14607). National Bureau of Economic Research.

Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment.

In this study the authors designed the Measures of Effective Teaching (MET) project to test replicable methods for identifying effective teachers. In past reports, the authors described three approaches to measuring different aspects of teaching: student surveys, classroom observations, and a teacher's track record of student achievement gains on state tests.

Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment. Research Paper. MET Project. Bill & Melinda Gates Foundation.

Identifying effective classroom practices using student achievement data

This paper combines information from classroom-based observations and measures of teachers' ability to improve student achievement as a step toward addressing these challenges. The results point to the promise of teacher evaluation systems that would use information from both classroom observations and student test scores to identify effective teachers.

Kane, T. J., Taylor, E. S., Tyler, J. H., & Wooten, A. L. (2011). Identifying effective classroom practices using student achievement data. Journal of human Resources, 46(3), 587-613.

 

Examining teacher evaluation validity and leadership decision making within a standards-based evaluation system

Substantial variation was found in the relationship of evaluators' ratings of teachers and value-added measures of the average achievement of the teachers' students. The results did not yield a simple explanation for the differences in validity of evaluators' ratings. Instead, evaluators' decisions were found to be a complex and idiosyncratic function of motivation, skill, and context.

Kimball, S. M., & Milanowski, A. (2009). Examining teacher evaluation validity and leadership decision making within a standards-based evaluation system. Educational Administration Quarterly45(1), 34-70.

Assessing the cost of instructional coaching.

this study presents and apply a framework for measuring the cost of coaching programs to 3 schools. Then the study discusses strategies for reducing the average cost of instructional coaching. 

Knight, D. S. (2012). Assessing the cost of instructional coaching. Journal of Education Finance, 52-80.

Value-added modeling: A review

This article provides a review of areas of agreement and disagreement of vaious aspects of value added modeling.

Koedel, C., Mihaly, K., & Rockoff, J. E. (2015). Value-added modeling: A review. Economics of Education Review. Retrieved from http://faculty.smu.edu/millimet/classes/eco7321/papers/koedel et al 2015.pdf

 

A measured approach: Value-added models are a promising improvement, but no one measure can evaluate teacher performance

The education policy community is abuzz with interest in value-added modeling as a way to estimate the effectiveness of schools and especially teachers. Value-added models provide useful information, but that information is error-prone and has a number of other important limitations.

Koretz, D. (2008). A measured approach. American Educator32(2), 18-39.

The validity of gains on the Kentucky Instructional Results Information System (KIRIS).

This study evaluated the extent to which the large performance gains shown on KIRIS represented real improvements in student learning rather than inflation of scores. 

Koretz, D. M., & Barron, S. I. (1998). The validity of gains on the Kentucky instructional results information system. KIRIS). Santa Monica: RAND.

The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence

This study review the empirical literature on teacher coaching and conduct meta-analyses to estimate the mean effect of coaching programs on teachers’ instructional practice and students’ academic achievement.

Kraft, M. A., Blazar, D., & Hogan, D. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research88(4), 547-588.

Using Coaching to improve the Fidelity of Evidence-Based Practices: A Review of Studies

The authors conducted a comprehensive review of research to identify the impact of coaching on changes in preservice and in-service teachers’ implementation of evidence-based practices.

Kretlow, A. G., & Bartholomew, C. C. (2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education33(4), 279-299.

Using in-service and coaching to increase teachers’ accurate use of research-based strategies

This study examined the effects of in-service plus follow-up coaching on first grade teachers’ accurate delivery of three research-based strategies during math instruction.

Kretlow, A. G., Cooke, N. L., & Wood, C. L. (2012). Using in-service and coaching to increase teachers’ accurate use of research-based strategies. Remedial and Special Education33(6), 348-361.

Using in-service and coaching to increase kindergarten teachers’ accurate delivery of group instructional units.

This study examined the effects of in-service support plus coaching on kindergarten teachers’ accurate delivery of group instructional units in math.

Kretlow, A. G., Wood, C. L., & Cooke, N. L. (2011). Using in-service and coaching to increase kindergarten teachers’ accurate delivery of group instructional units. The Journal of Special Education44(4), 234-246.

What matters for elementary literacy coaching? Guiding principles for instructional improvement and student achievement

The seven guiding principles in this manuscript offer research-based directions for literacy coaching.

L’Allier, S., Elish-Piper, L., & Bean, R. M. (2011). What matters for elementary literacy coaching? Guiding principles for instructional improvement and student achievement. The Reading Teacher, 63,544-554. doi: 10.1598/RT.63.7.2

Teacher Incentives

Three questions are addressed. First, what are the principles behind creating optimal teacher incentives, and how close do the actual structures in Sweden and the US conform to the ideal ones? Second, how much is performance affected by creating incentives for current teachers, and how much by changing the pool of teacher applicants? Third, do teacher preferences align with those of their students and of society in general, and if not, why not? Associated with each of these questions are policy implications that may remedy existing distortions.

Lazear, E. P. (2003). Teacher incentives. Swedish Economic Policy Review10(2), 179-214.

The politics and statistics of value-added modeling for accountability of teacher preparation programs

This paper reviews the evaluation of a value added model for evaluating teacher preparation programs in Texas.  The model produced statistically meaningful data but the results were sensitive to decisions about accountability critieria, the selection of teachers, and the selection of control variables.  Different decisions across each of these variables would impact the results of the value added modeling.

Lincove, J. A., Osborne, C., & Dillon…, A. (2014). The politics and statistics of value-added modeling for accountability of teacher preparation programs. … of Teacher Education. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0022487113504108

The Characteristics and Experiences of Youth in Special Education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups

The United States has committed to improving the lives of students with disabilities for over 40 years. Since the advent of Federal Law PL 94-142 in 1975 that mandated a free and appropriate education for all students regardless of ability and six reauthorizations of legislation, the federal government has emphasized the need to prepare students with disabilities for post-secondary education, careers, and independent living. The federal investment in funding special education services exceeds $15 Billion annually. It is reasonable to ask, are student with disabilities substantially benefiting from these efforts? The National Longitudinal Transition Study (NLTS) provides the most recent data on youth with disabilities and efforts to address their needs. The study used surveys in 2012 and 2013 on a nationally representative set of nearly 13,000 students. The student included were mostly those with an individualized education program (IEP) and expected to receive special education services. The data reveal participation in key transition activities by youth with an IEP and their parents have declined, although they are just as likely to have gone to an IEP meeting. The findings from this report suggest a closer examination of current practices is warranted with a focus on achieving the stated outcomes the laws were designed to remedy.

Lipscomb, S., Hamison, J., Liu Albert, Y., Burghardt, J., Johnson, D. R., & Thurlow, M. (2018). Preparing for Life after High School: The Characteristics and Experiences of Youth in Special Education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups. Full Report. NCEE 2017-4018. National Center for Education Evaluation and Regional Assistance.

The sensitivity of value-added teacher effect estimates to different mathematics achievement measures.

Using longitudinal data from a cohort of middle school students from a large school district,
we estimate separate “value‐added” teacher effects for two subscales of a mathematics
assessment under a variety of statistical models varying in form and degree of control for
student background characteristics.

Lockwood, J. R., McCaffrey, D. F., Hamilton, L. S., Stecher, B., Le, V. N., & Martinez, J. F. (2007). The sensitivity of value‐added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement44(1), 47-67.

Los Angeles teacher ratings

About 11,500 Los Angeles Unified elementary school teachers and 470 elementary schools are included in The Times' updated database of "value-added" ratings.

Los Angeles Times. (2021). Los Angeles teacher ratings.

Education Pays 2016: The Benefits of Higher Education for Individuals and Society.

This report documents differences in the earnings and employment patterns of U.S. adults with different levels of education. It also compares health-related behaviors, reliance on public assistance programs, civic participation, and indicators of the well-being of the next generation. This year's report also presents data on variation in earnings by different characteristics such as gender, race/ethnicity, occupation, college major, and sector. 

Ma, J., Pender, M., & Welch, M. (2016). Education Pays 2016: The Benefits of Higher Education for Individuals and Society. Trends in Higher Education Series. College Board.

Distorting value-added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability.

This study demonstrates mathematically that the use of such “construct-shifting” vertical scales in longitudinal, value-added models introduces remarkable distortions in the value-added estimates of the majority of educators

Martineau, J. A. (2006). Distorting value-added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavioral Statistics, 31(1), 35–62.

The effect of content-focused coaching on the quality of classroom text discussions

This study examines the effect of a comprehensive literacy-coaching program focused on enacting a discussion-based approach to reading comprehension instruction (content-focused coaching [CFC]) on the quality of classroom text discussions over 2 years.

Matsumura, L. C., Garnier, H.E., Spybrook, J. (2012). The effect of content-focused coaching on the quality of classroom text discussions. Journal of Teacher Education, 63,214-228.

Do value-added methods level the playing field for teachers? Carnegie Knowledge Network

In this brief, we discuss what is and is not known about how well value‐added measures level the playing field for teachers by controlling for student characteristics. 

McCaffrey, D. F. (2012). Do value-added methods level the playing field for teachers. Carnegie Knowledge Network.

Is value-added accurate for teachers of students with disabilities

In this brief, we discuss the challenges of using value-added to evaluate teachers of students with disabilities.

McCaffrey, D. F., & Buzick, H. (2014). Is value-added accurate for teachers of students with disabilities. Carnegie Knowledge Network Brief, (14).

Missing data in value-added modeling of teacher effects

The current study extends recent value-added modeling approaches for longitudinal student achievement data Lockwood et al. [J. Educ. Behav. Statist. 32 (2007) 125–150] to allow data to be missing not at random via random effects selection and pattern mixture models, and applies those methods to data from a large urban school district to estimate effects of elementary school mathematics teachers. 

McCaffrey, D. F., & Lockwood, J. R. (2011). Missing data in value-added modeling of teacher effects. The Annals of Applied Statistics, 773-797.

The promise and peril of using value-added modeling to measure teacher effectiveness

This article addresses the potential sources of bias that can be introduced into value added modeling by the decisions that are made about the  details of the model.  There is a call for a refinement of procedures used when applying value added modeling.

McCaffrey, D. F., Koretz, D., Lockwood, J. R., & Hamilton, L. S. (2004). The promise and peril of using value-added modeling to measure teacher effectiveness. rand.org. Retrieved from http://www.rand.org/pubs/research_briefs/RB9050

Evaluating Value-Added Models for Teacher Accountability. Monograph.

Value added modeling has become of interest to policymakers interested in evaluating teacher performance.  The authors argue that the models work well when the schools in the sample are homogenous but as heterogeneity of the student population  increases estimates of teacher effects are likely to confounded.

McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003). Evaluating Value-Added Models for Teacher Accountability. Monograph. ERIC. Retrieved from http://eric.ed.gov/?id=ED529961

Models for value-added modeling of teacher effects

Value added modeling has become of interest to policymakers interested in evaluating teacher performance.  The authors argue that the models work well when the schools in the sample are homogenous but as heterogeneity of the student population  increases estimates of teacher effects are likely to confounded.

McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T. A., & Hamilton, L. (2004). Models for Value-Added Modeling of Teacher Effects. Journal of Educational and Behavioral Statistics, 29(1), 67-101. doi:10.3102/10769986029001067

Alternative student growth measures for teacher evaluation: Implementation experiences of early-adopting districts

This study examines implementation of alternative student growth measures in a sample of eight school districts that were early adopters of the measures. It builds on an earlier Region­ al Educational Laboratory Mid-Atlantic report that described the two types of alterna­tive student growth measures—alternative assessment–based value-added models and student learning objectives—in the early-adopting districts.

McCullough, M., English, B., Angus, M. H., & Gill, B. (2015). Alternative student growth measures for teacher evaluation: Implementation experiences of early-adopting districts (No. 8a9dfcb1bc6143608448114ea9b69d06). Mathematica Policy Research.

Early intervention in reading: From research to practice

This study documents the implementation of research-based strategies to minimize the occurrence of reading difficulties in a first-grade population. Three strategies were implemented. 

Menzies, H. M, Mahdavi, J. N., & Lewis, J. L. (2008). Early intervention in reading: From research to practice. Remedial and Special Education, 29(2), 67-77.

Validity research on teacher evaluation systems based on the framework for teaching.

This paper summarizes validity evidence pertaining to several different implementations of the Framework. It is based primarily on reviewing the published and unpublished studies that have looked at the relationship between teacher evaluation ratings made using systems based on the Framework and value-added measures of teacher effectiveness.

Milanowski, A. T. (2011). Validity Research on Teacher Evaluation Systems Based on the Framework for Teaching. Online Submission.

The relationship between standards-based teacher evaluation scores and student achievement: Replication and extensions at three sites

This paper reports on the results of the analysis of an additional year of evaluation and student achievement data at some research.

Milanowski, A. T., Kimball, S. M., & White, B. (2004). The Relationship Between Standards-Based Teacher Evaluation Scores and Student Achievement: Replication and Extensions at Three Sites Consortium for Policy Research in Education (CPRE)-University of Wisconsin Working Paper Series. TC4(01).

How important are the cognitive skills of teenagers in predicting subsequent earnings?

How important are teenagers' cognitive skills in predicting subsequent labor market success? Do cognitive skills pay off in the labor market only for students who go to college? Does college benefit only students who enter with strong basic skills? These questions are often part of current policy debates about how to improve the earnings prospects for young Americans. 

Murnane, R. J., Willett, J. B., Duhaldeborde, Y., & Tyler, J. H. (2000). How important are the cognitive skills of teenagers in predicting subsequent earnings?. Journal of Policy Analysis and Management19(4), 547-568.

Where is the value in value-added modeling

This paper reviews the various issues and challenges associated with value added modeling.  It concludes with a discussion of how value added modeling  can be used in conjuction with other measures to identify teacher strengths and weaknesses.

Murphy, D. (2012). Where is the value in value-added modeling. 24–26 September 2014. Retrieved from http://educatoreffectiveness.pearsonassessments.com/downloads/viva_v1.pdf

Promoting language and literacy development for early childhood educators: A mixed-methods study of coursework and coaching

This study examines the impact of 2 forms of professional development on prekindergarten teachers' early language and literacy practice: coursework and coaching. 

Neuman, S. B., & Wright, T. S. (2010). Promoting language and literacy development for early childhood educators: A mixed-methods study of coursework and coaching. Elementary School Journal, 11,63-86. No Child Left Behind Act of 2001, P.L. 107-110, 20 U.S.C. § 6319 (2002).

No Child Left Behind Act of 2001

No Child Left Behind Act of 2001 ESEA Reauthorization

No child left behind act of 2001. Publ. L, 107-110. (2002)

Value-Added Assessment of Teacher Preparation: An Illustration of Emerging Technology

This paper describes one effort to use value added modeling to evaluate teacher preparation program in one state.

Noell, G. H., & Burns, J. L. (2006). Value-Added Assessment of Teacher Preparation: An Illustration of Emerging Technology. Journal of Teacher Education, 57(1), 37-50. doi:10.1177/0022487105284466

doi: 10.1177/0022487105284466

Effects of an early literacy professional development intervention on Head Start teachers and children

Effects of a 1-semester professional development (PD) intervention that included expert coaching with Head Start teachers were investigated in a randomized controlled trial with 88 teachers and 759 children. 

Powell, D. R., Diamond, K. E., Burchinal, M. R., & Koehler, M. J. (2010). Effects of an early literacy professional development intervention on Head Start teachers and children. Journal of Educational Psychology, 102, 299-312.

Using Coaching to Support Teacher Implementation of Classroom-based Interventions.

This study evaluted the impact of coaching on the implementation of an intervention.  Coaching with higher rates of performance feedback resulted in the highest level of treatment integrity.

Reinke, W., Stormont, M., Herman, K., & Newcomer, L. (2014). Using Coaching to Support Teacher Implementation of Classroom-based Interventions. Journal of Behavioral Education, 23(1), 150-167.

Teachers, schools, and academic achievement.

This paper disentangles the impact of schools and teachers in influencing achievement with special attention given to the potential problems of omitted or mismeasured variables and of student and school selection. 

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica73(2), 417-458.

How teacher turnover harms student achievement

This study used a version of value added modeling to evaluate the impact of teacher turnover has on student achievement.

Ronfeldt, M., Lankford, H., Loeb, S., & Wyckoff, J. (2011). How Teacher Turnover Harms Student Achievement. National Bureau of Economic Research Working Paper Series, No. 17176. doi:10.3386/w17176

Teacher quality in educational production: Tracking, decay, and student achievement.

The author develop falsification tests for three widely used VAM specifications, based on the idea that future teachers cannot influence students' past achievement. 

Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. The Quarterly Journal of Economics125(1), 175-214.

A randomized controlled trial of COMPASS web-based and face-to-face teacher coaching in autism

Most children with autism rely on schools as their primary source of intervention, yet research has suggested that teachers rarely use evidence-based practices. To address the need for improved educational outcomes, a previously tested consultation intervention called the Collaborative Model for Promoting Competence and Success was evaluated in a 2nd randomized controlled trial, with the addition of a web-based group. 

Ruble, L. A., McGrew, J. H., Toland, M. D., Dalrymple, N. J., & Jung, L. (2013). A randomized controlled trial of COMPASS web-based and face-to-face teacher coaching in autism. Journal of Consulting and Clinical Psychology, 81, 566-572.

The Hidden Cost of California’s Harsh School Discipline: And the Localized Economic Benefits from Suspending Fewer High School Students

This research from the Center for Civil Rights Remedies at the Civil Rights Project, UCLA, and California Dropout Research Project shows that the overuse of suspensions in California schools is harming student achievement and graduation rates, and causing billions of dollars in economic damage. The financial consequences of school suspensions, including both additional costs borne by taxpayers as a result of suspensions and lost economic benefit, are quantified. The impact of school suspension varies widely by school district, with California’s largest districts incurring the greatest losses. For example, suspensions in the Los Angeles Unified School District for a 10th grade cohort are estimated to cause $148 million in economic damage. The report calculates a total statewide economic burden of $2.7 billion over the lifetime of the single 10th grade cohort.

Rumberger, R., & Losen, D. (2017). The Hidden Cost of California’s Harsh School Discipline: And the Localized Economic Benefits from Suspending Fewer High School Students. The Center for Civil Rights Remedies at the Civil Rights Project, UCLA, and California Dropout Research Project.

Using value-added models to measure teacher effects on students’ motivation and achievement

Using data from 35 seventh-grade teachers and 2,026 students across seven schools, we employ VA methods to measure teacher contributions to students’ motivational orientations (mastery and performance achievement goals) and their mathematics performance. 

Ruzek, E. A., Domina, T., Conley, A. M., Duncan, G. J., & Karabenick, S. A. (2015). Using value-added models to measure teacher effects on students’ motivation and achievement. The Journal of Early Adolescence35(5-6), 852-882.

Professional development for cognitive reading strategy instruction

In this article, we describe and report on the results of a study in Texas that tested 2 models of professional development for classroom teachers as a way of improving their practices and increasing the reading achievement of their students. 

Sailors, M., & Price, L. (2010). Professional development for cognitive reading strategy instruction. Elementary School Journal, 110,301-323.

 

The Tennessee value-added assessment system (TVAAS): Mixed-model methodology in educational assessment

This paper describes the mixed method model Tennessee developed to evaluate teacher contributions to student achievement.  It desribes how they resolved some of the challenges to using value added modeling.

Sanders, W. L., & Horn, S. P. (1994). The Tennessee value-added assessment system (TVAAS): Mixed-model methodology in educational assessment. Journal of Personnel Evaluation in education. Retrieved from https://eric.ed.gov/?id=EJ498467

Research Findings from the Tennessee Value-Added Assessment System (TVAAS) Database: Implications for Educational Evaluation and Research

The Tennessee Value-Added Assessment System determines the effectiveness of school systems, schools, and teachers based on student academic growth over time.

Sanders, W. L., & Horn, S. P. (1998). Research findings from the Tennessee Value-Added Assessment System (TVAAS) database: Implications for educational evaluation and research. Journal of Personnel Evaluation in Education12(3), 247-256.

Cumulative and residual effects of teachers on future student academic achievement.

The Tennessee Value-Added Assessment System determines the effectiveness of school systems, schools, and teachers based on student academic growth over time. Research conducted utilizing data from the TVAAS database has shown that race, socioeconomic level, class size, and classroom heterogeneity are poor predictors of student academic growth. Rather, the effectiveness of the teacher is the major determinant of student academic progress.

Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student academic achievement.

Measuring teaching using value-added modeling: The imperfect panacea

This paper attempts to untangle some of the competing claims about the value of value added modeling.  It concludes that it should not be used as the sole measure of teacher performance but should be part of a larger accountability system.

Scherrer, J. (2011). Measuring teaching using value-added modeling: The imperfect panacea. NASSP Bulletin. Retrieved from https://eric.ed.gov/?id=EJ938929.

Effects of multilevel support on first-grade teachers’ use of research-based strategies during beginning reading instruction

The purpose of this study was to examine the effects of multilevel support on first-grade teachers' accurate use of research-based strategies during beginning reading instruction and the extent to which teachers maintained use of these strategies. 

Schnorr, C. I. (2013). Effects of multilevel support on first-grade teachers' use of research-based strategies during beginning reading instruction (Doctoral dissertation, The University of North Carolina at Charlotte).

The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation

A recent large-scale evaluation of Reading Recovery, a supplemental reading program for young struggling readers, supports previous research that found it to be effective.  In a 4 year, federally funded project, almost 3,500 students in 685 schools found that generally students benefitted from the intervention. Students receiving Reading Recovery receive supplemental services in a 1:1 instructional setting for 30 minutes 5 days a week from an instructor trained in Reading Recovery.  In the study reported here, students who received Reading Recovery had effect sizes of .35-.37 relative to a control group across a number of measures of reading.  These represent moderate effect sizes and account for about a 1.5 month increase in skill relative to the control group.  Even though the research supports the efficacy of the intervention, it also raises questions about its efficiency.  The schools that participated in the study served about 5 students and the estimated cost per student has ranged from $2,000-$5,000.  These data raise questions about the wisdom of spending this much money per student for growth of about a month and a half.

Sirinides, P., Gray, A., & May, H. (2018). The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation. Educational Evaluation and Policy Analysis, 0162373718764828.

Teacher job satisfaction and motivation to leave the teaching profession: Relations with school context, feeling of belonging, and emotional exhaustion.

This study examines the relations between school context variables and teachers’ feeling of belonging, emotional exhaustion, job satisfaction, and motivation to leave the teaching profession. Six aspects of the school context were measured: value consonance, supervisory support, relations with colleagues, relations with parents, time pressure, and discipline problems.

Skaalvik, E. M., & Skaalvik, S. (2011). Teacher job satisfaction and motivation to leave the teaching profession: Relations with school context, feeling of belonging, and emotional exhaustion. Teaching and teacher education27(6), 1029-1038.

Incorporating student performance measures into teacher evaluation systems.

the authors examine how the five profiled systems are addressing assessment quality, evaluating teachers in nontested subjects and grades, and assigning teachers responsibility for particular students. The authors also examine what is and is not known about the quality of various student performance measures used by school systems.

Steele, J. L., Hamilton, L. S., & Stecher, B. M. (2010). Incorporating Student Performance Measures into Teacher Evaluation Systems. Technical Report. Rand Corporation.

Evaluating special educator effectiveness: Addressing issues inherent to value-added modeling

This paper addresses the unique challenges posed by special education when using value added modeling to evaluate teacher effectiveness.

Steinbrecher, T. D., Selig, J. P., Cosbey, J., & Thorstensen, B. I. (2014). Evaluating Special Educator Effectiveness. Exceptional Children, 80(3), 323-336. doi:10.1177/0014402914522425

 

The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half-Century of Research

A soon to be published meta-analysis of Direct Instruction (DI) curricula that reviews research on DI curricula between 1966-2016 reports that DI curricula produced moderate to large effect sizes across the curriculum areas reading, math, language, and spelling.  The review is notable because it reviews a much larger body of DI research than has occurred in the past and covers a wide range of experimental designs (from single subject to randomized trials).  328 studies were reviewed and almost 4,000 effects were considered.  Given the variability in research designs and the breadth of the effects considered, it suggests that DI curricula produce robust results.  There was very little decline during maintenance phases of the study and greater exposure to the curricula resulted in greater effects.

Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C. (2018). The effectiveness of direct instruction curricula: A meta-analysis of a half century of research. Review of Educational Research88(4), 479-507.

Multitiered support framework for teachers’ classroom-management practices: Overview and case study of building the triangle for teachers

In this article, the authors describe key features of the multi-tiered support (MTS) continuum of intervention and assessment and present a case study to illustrate implementation of some components of the framework with four middle school teachers.

Sugai, G. (2014). Multitiered support framework for teachers’ classroom-management practices: Overview and case study of building the triangle for teachers. Journal of Positive Behavior Interventions, 16(3), 179-190.

Targeted reading intervention: A coaching model to help classroom teachers with struggling readers

This study examined the effectiveness of a classroom teacher intervention, the Targeted Reading Intervention (TRI), in helping struggling readers in kindergarten and first grade. This intervention used biweekly literacy coaching in the general education classroom to help classroom teachers use diagnostic strategies with struggling readers in one-on-one 15-min sessions.  

Targeted reading intervention: A coaching model to help classroom teachers with struggling readers. Learning Disability Quarterly, 35, 102-114.

The Mirage: Confronting the truth about our quest for teacher development

"The Mirage" describes the widely held perception among education leaders that they already know how to help teachers improve, and that they could achieve their goal of great teaching in far more classrooms if they just applied what they knew more widely.

TNTP. (2015). The Mirage: Confronting the truth about our quest for teacher development. Retrieved from: https://tntp.org/publications/view/the-mirage-confronting-the-truth-about-our-quest-for-teacher-development

Rush to judgment: Teacher evaluation in public education

The authors examine the causes and consequences of the status of teacher evaluation and its implications for the current national debate about performance pay for teachers. The report also examines a number of national, state, and local evaluation systems that offer potential alternatives to current practice.

Toch, T., & Rothman, R. (2008). Rush to Judgment: Teacher Evaluation in Public Education. Education Sector Reports. Education Sector.

The coaching of teachers: Results of five training studies.

In this study, the results of five training studies evaluating the effects of a coaching program for use in Dutch primary and secondary schools are described.

Veenman, S, & Denessen, E. (2001). The coaching of teachers: Results of five training studies.

Educational Research and Evaluation, 7(4), 385–417.

The Widget Effect: Our National Failure to Acknowledge and Act on Differences in Teacher Effectiveness.

This report examines the pervasive and longstanding failure to recognize and respond to variations in the effectiveness of teachers. 

Weisberg, D., Sexton, S., Mulhern, J., Keeling, D., Schunck, J., Palcisco, A., & Morgan, K. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. New Teacher Project.

Making the case for evidence-based policy

U.S. public policy has increasingly been conceived, debated, and evaluated through the lenses of politics and ideology. The fundamental question -- Will the policy work? -- too often gets short shrift or even ignored. A remedy is an evidence-based policy--a rigorous approach that draws on careful data collection, experimentation, and both quantitative and qualitative analysis to determine what the problem is, which ways it can be addressed, and the probable impacts of each of these ways. 

Wesley, P. W., & Buysse, V. (2006). Making the case for evidence- based policy. In V. Buysse & P. W. Wesley (Eds.), Evidence-based practice in the early childhood field (pp. 117–159). Washington, DC: Zero to Three.

Role of professional development and multi-level coaching in promoting evidence-based practice in education

 Due to the increased need to support teachers' use of evidence-based practices in multi-tiered systems of support such as RTI [Response to Intervention] and PBIS [Positive Behavior Interventions and Support], coaching can extend and strengthen professional development. This paper describes a multi-level approach to coaching and provides implications for practice and research.

Wood, C. L., Goodnight, C. I., Bethune, K. S., Preston, A. I., Cleaver, S. L. (2016). Role of professional development and multi-level coaching in promoting evidence-based practice in education. Learning Disabilities: A Contemporary Journal, 14,159-170.

Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement. Issues & Answers.

The purpose of this study is to examine research to answer the question, What is the impact of teacher professional development on student achievement.

Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. L. (2007). Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement. Issues & Answers. REL 2007-No. 033. Regional Educational Laboratory Southwest (NJ1).

TITLE
SYNOPSIS
University of Pennsylvania: A Grand Bargain for Education Reform

This web site provides resources for educators to better under what value-added modeling is and how it works.

Back to Top