To produce better outcomes for students two things are necessary: (1) effective, scientifically supported interventions (2) those interventions implemented with high integrity. Typically, much greater attention has been given to identifying effective practices. This review focuses on features of high quality implementation.
Detrich, R. (2014). Treatment integrity: Fundamental to education reform. Journal of Cognitive Education and Psychology, 13(2), 258-271.
Schools are often expected to implement innovative instructional programs. Most often these initiatives fail because what we know from implementation science is not considered as part of implementing the initiative. This chapter reviews the contributions implementation science can make for improving outcomes for students.
Detrich, R. Innovation, Implementation Science, and Data-Based Decision Making: Components of Successful Reform. Handbook on Innovations in Learning, 31.
Reform efforts tend to come and go very quickly in education. This paper makes the argument that the sustainability of programs is closely related to how well those programs are implemented.
Detrich, R., Keyworth, R. & States, J. (2010). Treatment Integrity: A Fundamental Unit of Sustainable Educational Programs. Journal of Evidence-Based Practices for Schools, 11(1), 4-29.
This study evaluated the effects of performance feedback to increase the implementation of skills taught during in-service training.
Auld, R. G., Belfiore, P. J., & Scheeler, M. C. (2010). Increasing Pre-service Teachers’ Use of Differential Reinforcement: Effects of Performance Feedback on Consequences for Student Behavior. Journal of Behavioral Education, 19(2), 169-183.
This study evaulates the effects of performance feedback as part of proffessional development across three studies.
Barton, E. E., Pribble, L., & Chen, C.-I. (2013). The Use of E-Mail to Deliver Performance-Based Feedback to Early Childhood Practitioners. Journal of Early Intervention, 35(3), 270-297.
One of the primary goals of implementation science is to insure that programs are implemented with integrity. This paper presents an integrated model of implementation that emphasizes treatment integrity.
Berkel, C., Mauricio, A. M., Schoenfelder, E., Sandler, I. N., & Collier-Meek, M. (2011). Putting the pieces together: An Integrated Model of program implementation. Prevention Science, 12, 23-33.
This study investigated the effects of performance feedback as a means for increasing the integrity of problem-solving teams following a decision protocol.
Burns, M. K., Peters, R., & Noell, G. H. (2008). Using performance feedback to enhance implementation fidelity of the problem-solving team process. Journal of School Psychology, 46(5), 537-550.
This review assesses the effectiveness of school-based curricula, finance, management, and teacher’s decision-making. This report has implications for the impact of charter schools, as the primary intervention in this model is local control. The report finds limited evidence of the effectiveness of these reforms, especially from low-income countries.
Carr-Hill, R., Rolleston, C., Pherali, T., & Schendel, R. (2014). The effects of school-based decision making on educational outcomes in low-and middle-income contexts: A systematic review.
Incidental teaching is often a component of early childhood intervention programs. This study evaluated the use of grahical feedback to increase the use of incidental teaching.
Casey, A. M., & McWilliam, R. A. (2008). Graphical Feedback to Increase Teachers’ Use of Incidental Teaching. Journal of Early Intervention, 30(3), 251-268.
One of the challenges for increasing treatment integrity is finding effective methods for doing so. This study evaluated the use of checklist-based training to increase treatment integrity.
Casey, A. M., & McWilliam, R. A. (2011). The impact of checklist-based training on teachers’ use of the zone defense schedule. Journal of Applied Behavior Analysis, 44(2), 397-401.
This study evaluated the impact of performance feedback on how well problem-solving teams implemeted a structured decision-making protocal. Teams performed better when feedback was provided.
Codding, R. S., & Smyth, C. A. (2008). Using Performance Feedback To Decrease Classroom Transition Time And Examine Collateral Effects On Academic Engagement. Journal of Educational & Psychological Consultation, 18(4), 325-345. Retrieved from http://content.epnet.com/ContentServer.asp?T=P&P=AN&K=35383935&EbscoContent=dGJyMNXb4kSeprY4zdnyOLCmr0yep7VSr6m4Ta6WxWXS&ContentCustomer=dGJyMOzprk%2BvqrdPuePfgeyx%2BEu3q64A&D=eue
This study investigated the effects of performance feedback to increase treatment integrity.
Codding, R. S., Feinberg, A. B., & Dunn, E. K. (2005). Effects of Immediate Performance Feedback on Implementation of Behavior Support Plans. Journal of Applied Behavior Analysis, 38(2), 205-219.
This study evaluated the effects of performance feedback in increasing treatment integrity. It also evaluated the possible reactivitiy effects of being observed.
Codding, R. S., Livanis, A., Pace, G. M., & Vaca, L. (2008). Using Performance Feedback to Improve Treatment Integrity of Classwide Behavior Plans: An Investigation of Observer Reactivity. Journal of Applied Behavior Analysis, 41(3), 417-422.
This study evaluated the effects of video modeling on staff implementation of a problem solving intervention.
Collins, S., Higbee, T. S., & Salzberg, C. L. (2009). The effects of video modeling on staff implementation of a problem-solving intervention with adults with developmental disabilities. Journal of applied behavior analysis, 42(4), 849-854.
This study evaluated the effects of allowing teachers to “test drive” interventions and then select the intervention they most preferred. The result was an increase in treatment integrity.
Dart, E. H., Cook, C. R., Collins, T. A., Gresham, F. M., & Chenier, J. S. (2012). Test Driving Interventions to Increase Treatment Integrity and Student Outcomes. School Psychology Review, 41(4), 467-481.
This study comared the effects of goal setting about student performance and feedback about student performance with daily written feedback about student performance, feedback about accuracy of implementation, and cancelling meetings if integrity criterion was met.
DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers' treatment implementation integrity and students' inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40(3), 447-461.
This study evaluated the impact of allowing teachers to miss coaching meetings if their treatment integrity scores met or exceeded criterion.
DiGennaro, F. D., Martens, B. K., & McIntyre, L. L. (2005). Increasing Treatment Integrity Through Negative Reinforcement: Effects on Teacher and Student Behavior. School Psychology Review, 34(2), 220-231.
This study evaluated the effects of video modeling on how well teachers implemented interventions. There was an increase in integrity but it remained variable. More stable patterns of implementation were observed when teachers were given feedback about their peroformance.
Digennaro-Reed, F. D., Codding, R., Catania, C. N., & Maguire, H. (2010). Effects of video modeling on treatment integrity of behavioral interventions. Journal of Applied Behavior Analysis, 43(2), 291-295.
This study evaluated the impact of public feedback in RtI team meetings on the quality of implementation. Feedback improved poor implementation and maintained high level implementation.
Duhon, G. J., Mesmer, E. M., Gregerson, L., & Witt, J. C. (2009). Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance. Journal of School Psychology, 47(1), 19-37.
This paper describes the scaling up and dissemination of a partent training program in Norway while maintaining fidelity of implementation.
Forgatch, M. S., & DeGarmo, D. S. (2011). Sustaining fidelity following the nationwide PMTO implementation in Norway. Prevention Science, 12(3), 235-246. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3153633
Garbacz, L., Brown, D., Spee, G., Polo, A., & Budd, K. (2014). Establishing Treatment Fidelity in Evidence-Based Parent Training Programs for Externalizing Disorders in Children and Adolescents. Clinical Child & Family Psychology Review, 17(3).
This article evaluated the effects of response dependent feedback on accurate implementation of an intervention. Whenever teachers failed to meet 100% accuracy criterion they were given feedback about their performance.
Gilbertson, D., Witt, J., Singletary, L., & VanDerHeyden, A. (2007). Supporting teacher use of interventions: effects of response dependent performance feedback on teacher implementation of a math intervention. Journal of Behavioral Education, 16(4), 311-326.
Organizations house many individuals. Many of them are responsible implementing the same practice. If organizations are to meet their goal it is important for the organization have systems for assuring high levels of treatment integrity.
Gottfredson, D. C. (1993). Strategies for Improving Treatment Integrity in Organizational Consultation. Journal of Educational & Psychological Consultation, 4(3), 275.
Founded more than 45 years ago at Johns Hopkins University, the Center for Social Organization of Schools (CSOS), now part of the Hopkins’ School of Education, concentrates its considerable research and development resources on improving low-performing schools and the education they offer their students.