What Does it Take to Assure High-Quality Implementation?

A fundamental assumption of evidence-based practice is that interventions will produce benefit only if there are high treatment integrity levels. High levels cannot be assumed in the usual course of practice in education. It must be planned for and routinely monitored. Often, there is not the time and resources to do that in schools, so effective interventions fail to produce the expected benefits for students. The standard “train and hope” is not sufficient to assure adequate levels of treatment integrity. The question becomes what is sufficient? George Noell, Kristin Gansle, and Veronic Gulley (2021) recently addressed this question. Teachers were assigned to either a weekly follow-up consultation meeting or an Integrated Support condition that included social influence, planning, and performance feedback. After an initial four-week consultation period in which problems were identified, intervention plans were developed, and staff were trained to implement, teachers in each group were followed for four additional weeks to determine their level of treatment integrity and effects on student behavior (either behavior or academic). Implementation scores for the participants in the Weekly follow-up meeting were relatively low the first week and declined across the rest of the four weeks.  

Participants in the Integrated Support group had high levels of treatment integrity the first week and scores decreased very little across the rest of the study. Students in the Integrated Support group had much greater improvements in behavior than students in the Weekly Follow-up condition. 

The authors reported that three school climate variables were related to plan implementation and child outcomes in the Integrated Support condition. For treatment plan implementation, the variables were (1) student relations (2) resources (3) time. For child outcomes, the only school climate factor was time. There were no school climate variables that influenced the Weekly Follow-up condition outcomes at either the level of treatment plan implementation or child outcomes.

These data highlight the importance of continuous monitoring of implementation and supporting educators as they implement intervention plans. Failure to do so results in very limited outcomes for students, does not use implementers time most effectively, and yields a very poor return on investment. Separating monitoring of implementation from intervention will almost always result in poor outcomes for students.

The challenge for schools is to reconfigure services so that monitoring treatment integrity is considered a part of services as it generates best outcomes for students. 

Citation: 

Noell, G., Gansle, K., & Gulley, V. (2021). The Impact of Integrated Support and Context on Treatment Implementation and Child Outcomes Following Behavioral Consultation. Behavior Modification, 01454455211054020.