Does professional development make a difference in student performance?

Why is this question important? The history of education reform is littered with structural interventions that contribute little to improving schools. Although popular, interventions such as class size reduction, school size reduction, charter schools, school vouchers, and accountability through high-stakes testing have fallen short when it comes to increasing student test scores. Research suggests that teachers make a difference in improving student performance (Sanders & Rivers, 1996; Mendro, Jordon, Gomez, Anderson, & Bembry, 1998; Rowan, Correnti, & Miller, 2002). Given the importance of what happens in the classroom, can staff development contribute to maximizing teachers’ impact on student achievement?

See further discussion below.

858.jpg

Source(s): The Effects of Teachers’ Professional Development on Student Achievement: Findings From a Systematic Review of Evidence, 2008

Result(s): The purpose of the paper was to research any empirical links between professional development and student achievement. An average effect size of 0.54 in mathematics, science, and reading and English/language arts was reported. Consistency across the three academic domains suggests that professional development has a moderate effect on student achievement. Achievement increased an average 21% for students whose teachers were provided professional development. Because of the limited number of studies included in the paper, the study results applied only to elementary school students and teachers.

Implication(s): As a limited number of studies met What Works Clearinghouse’s (WWC) evidence standards, only nine were included in the paper. To build a robust evidence base for the efficacy of professional development, more research needs to be conducted. Despite the shortage of studies, the paper does suggest that professional development can be a valuable tool to improve student achievement.

Unfortunately, the variability in the professional development approaches in the studies preclude any conclusions about the effectiveness of specific professional development programs or the effectiveness of professional development by form, content, or intensity.

Author(s): Kwang Suk Yoon, Teresa Duncan, Sylvia Lee, and Kathy Shapley

Publisher(s): The Elementary School Journal

Study Description: The paper examined studies on the topic of staff development and/or teacher in-service training. From a total of over 1,300 studies, 9 met all three criteria: rigorous research meeting WWC standards (empirically based and using randomized controlled trials or some form of quasi-experimental design), direct examination of the effect of training on student achievement, and focus on the three core academic subjects of mathematics, science, and reading and English/language arts.

The nine studies were:

  • Carpenter, Fennema, Peterson, Chiang, and Loef, 1989
  • Cole, 1992
  • Duffy et al., 1986
  • Marek and Methven, 1991
  • McCutchen et al., 2002
  • McGill-Franzen, Allington, Yokoi, and Brooks, 1999
  • Saxe, Gearhart, and Nasir, 2001
  • Sloan, 1993
  • Tienken, 2003

The paper identified 20 effect sizes across the nine studies and concluded the following:

  • The average effect size across the nine studies was 0.54, ranging from –0.53 to 2.39.
  • The average improvement index was 21 (which can be translated to boosting students’ achievement by 21%), ranging from –20 to 49.
  • Only one effect was negative (Saxe et al., 2001), and only one effect was zero (Duffy et al., 1986). The other 18 effects were positive, with effect sizes ranging from 0.12 to 2.39 (and improvement indices from 5 to 49).
  • Of the 20 effects, 12 were not statistically significant after applying necessary corrections. However, 9 of those 12 were substantively important, according to WWC conventions. WWC stipulates that statistical significance informs us if results are mathematically large, but it may tells us very little about practical significance or relative impact. It is important that statistical significance not be used exclusively to measure the results of an intervention. Effect sizes that are not statistically significant but have an effect size of at least 0.25 are considered substantively important in WWC standards.

Definition(s):
Professional development: Training provided to teachers to increase teacher knowledge and classroom skills. The form of training includes in-service instruction, workshops, and summer institutes. The instruction pedagogy includes didactic presentation of information, classroom demonstrations, and/or peer coaching. In all nine studies, professional development training was provided directly to the teachers rather than through a train-the-trainer approach.

Statistical significance: A finding of statistical significance using a two-tailed t-test with α = .05 for a single measure or mean effect within each domain.

Structural intervention: A systemic educational intervention designed to indirectly improve student achievement by targeting factors that support educational efforts but do not directly impact actual classroom instruction (what is taught and how it is taught). Examples of structural interventions include increased teacher pay, reduced class size, school size, and charter schools. By contrast instructional interventions include: progress monitoring, curriculum design, homework, lesson pacing, active responding, and sequenced skill presentation and acquisition.

Substantively important: The smallest positive value at or above which the effect is deemed substantively important with relatively high confidence for the outcome domain at issue. Effect sizes at least this large will be taken as a qualified positive effect even though they may not reach statistical significance in a given study. The suggested default value is a student-level effect size greater than or equal to 0.25.

Citation:
Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C. P., & Loef, M. (1989). Using knowledge of children’s mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26(4), 499–531.

Cole, D. C. (1992). The effects of a one-year staff development program on the achievement of test scores of fourth-grade students. Dissertation Abstracts International, 53(06), 1792A. (UMI No. 9232258).

Duffy, G. G., Roehler, L. R., Meloth, M. S., Vavrus, L. G., Book, C., Putnam, J., & Wesselman, R. (1986). The relationship between explicit verbal explanations during reading skill instruction and student awareness and achievement: A study of reading teacher effects. Reading Research Quarterly, 21(3), 237–252.

Marek, E. A., & Methven, S. B. (1991). Effects of the learning cycle upon student and classroom teacher performance. Journal of Research on Science in Teaching, 28(1), 41–53.

Mendro, R., Jordan, H., Gomez, E., Anderson, M., & Bembry, K. (1998). An application of multiple linear regression in determining longitudinal teacher effectiveness. Dallas, TX: Dallas Public Schools.

McCutchen, D., et al. (2002). Beginning literacy: Links among teacher knowledge, teacher practice, and student learning. Journal of Learning Disabilities, 35(1), 69–86.

McGill-Franzen, A., Allington, R. L., Yokoi, L., & Brooks, G. (1999). Putting books in the classroom seems necessary but not sufficient. Journal of Reading Research, 93(2), 67–74.

Rowan, B., Correnti, R., & Miller, R. (2002). What large-scale research tells us about teacher effects on student achievement: Insights from the prospects study of elementary schools. Consortium for Policy Research in Education. Retrieved February 9, 2011, from http://cw.marianuniversity.edu/mreardon/755/document%20repository/Teacher%20Effects%20on%20Student%20Achievement.pdf

Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student academic achievement. University of Tennessee Value-Added Research and Assessment Center. Retrieved September 18, 2010, from http://www.mccsc.edu/~curriculum/cumulative%20and%20residual%20effects%20of%20teachers.pdf

Saxe, G. B., Gearhart, M., & Nasir, N. S. (2001). Enhancing students’ understanding of mathematics: A study of three contrasting approaches to professional support. Journal of Mathematics Teacher Education, 4, 55–79.

Sloan, H. A. (1993). Direct instruction in fourth and fifth grade classrooms. Dissertation Abstracts International, 54(08), 2837A. (UMI No. 9334424)

Tienken, C. H. (2003). The effect of staff development in the use of scoring rubrics and reflective questioning strategies on fourth-grade students’ narrative writing performance. Dissertation Abstracts International, 64(02), 388A. (UMI No. 3081032)

What Works Clearinghouse (2008). What Works Clearinghouse
Procedures and Standards Handbook (Version 2.0). Retrieved May 18, 2011, from does-professional-development-make

Yoon, K. S., Duncan, T., Lee, S., & Shapley K, (2008). The effects of teachers’ professional development on student achievement: Findings from a systematic review of evidence. American Education Research Association (AERA). Retrieved May 16, 2011, from http://www.pdal.net/inc/docs/AERA%202008%20Paper_final_PD%20research%20review.pdf