Categories for Implementation
October 4, 2021
Electronically Delivered Support to Promote Intervention Implementation Fidelity: A Research Synthesis. Fundamental to any intervention outcome is the fidelity of implementation of the intervention. The ultimate goal of implementation science is to assure that innovations are implemented well enough for students to benefit. Failure to implement well can minimize the effectiveness of even the most powerful intervention. One of the challenges involved in insuring high quality implementation is that most approaches are resource intensive and often are not seen as feasible in school settings even though failure to achieve adequate implementation fidelity may result in a very poor benefit to cost ratio. One possible alternative is to utilize technology to reduce the resource demands. Fallon and colleagues (2021) conducted a systematic review to evaluate the effectiveness of technology-based supports to promote implementation fidelity. For the purposes of this review, “electronically delivered implementation supports (EDIS) was support delivered to an implementer electronically (e.g., via email, social media, video conferencing) for the purpose of improving educators’ implementation fidelity of a student intervention.” Fifteen studies met inclusion criteria and were judged to be of sufficient methodological rigor to warrant further analysis. The electronically delivered implementation supports ranged from video modeling, electronically delivered performance feedback, emailed intervention prompts, coaching via video conference, and online training modules. All of the studies were based on single participant designs. Since there are no agreed upon methods for calculating effect sizes for single participant designs, the authors calculated several different effect sizes (Tau-U, Standard Mean Difference, Hedges’ g, and a variation of Hedges’ g. In most of the studies, the effect sizes ranged from moderate to large regardless of the calculation method used. After completing the review, the authors provided guidance to educators about when to use the various methods of electronically delivered implementation supports. This article is a valuable resource to any educator considering implementing an intervention but is concerned about the resource requirements required for insuring high quality implementation. This article suggests that technology-based alternatives can be effective in supporting implementation and may reduce the overall demands on resources.
Citation: Fallon, L. M., Collier-Meek, M. A., Famolare, G. M., DeFouw, E. R., & Gould, K. M. (2020). Electronically Delivered Support to Promote Intervention Implementation Fidelity: A Research Synthesis. School Psychology Review, 1-16.
Link: https://doi.org/10.1080/2372966X.2020.1853485
October 4, 2021
School-Based Interventions for Middle School Students With Disruptive Behaviors: A Systematic Review of Components and Methodology. Middle school students are more likely than elementary or high school students to be disruptive (Erikson & Gresham, 2019). This presents difficult problems for classroom teachers trying to provide instruction and maintain order in the classroom. It has been estimated that 2.5 hours per week are lost to disruptive behavior each week (Education Advisory Board, 2019). This level of disruptive also contributes to teacher turnover with 39% of teachers reporting that disruptive behavior was one of the primary reasons for resigning (Bettini et al., 2020). To address challenges presented by middle school students, Alperin and colleagues (2021) completed a systematic review to identify programs that had a positive effect on disruptive behavior and the characteristics of those programs. They identified 51 studies that met their inclusion criteria. Of those 51 studies, 40 of them specified the function of behavior (gain attention or escape demands) that the program addressed; 16 of the studies included a home-based component with 7 of the studies providing parent training; 22 of the interventions had a manual guiding the implementation of the intervention; and encouragingly, 42 of the studies assessed intervention implementation. Effect sizes for seven of the studies were computed for intervention that involved class-wide intervention strategies. The effects ranged from small to large across the studies. Fourteen of the studies evaluated skill acquisition for small groups or individuals and the effect sizes again ranged from small to large. Seven of the studies evaluated reinforcement strategies for reinforcement-based interventions for small groups or individuals and reported effect sizes that ranged from small to large. Two studies evaluated interventions for escape from demands for small groups or individuals. Both of these studies reported large effect sizes. The data from this study are important as they can provide guidance to educators seeking to reduce disruptive behavior of middle school students. Ultimately, the educators will have to consider the contextual fit for each of these interventions for the settings in which they work. This study narrows the range of options to those that have some demonstrated level of effectiveness rather than leaving the educator to choose from all available options.
Citation: Alperin, A., Reddy, L. A., Glover, T. A., Bronstein, B., Wiggs, N. B., & Dudek, C. M. (2021). School-Based Interventions for Middle School Students With Disruptive Behaviors: A Systematic Review of Components and Methodology. School Psychology Review, 1-26.
Link: https://doi.org/10.1080/2372966X.2021.1883996
October 4, 2021
Scaling and Disseminating Brief Bullying Prevention Programming: Strengths, Challenges, and Considerations. One of the persistent problems in education and other human service disciplines is the research to practice gap (some would call it a chasm). In an effort to disseminate an effective bullying program (Free2B), Leff and colleagues applied the logic of Diffusion of Innovations (Rogers, 2003). This logic proposes that innovations are more likely to be adopted if the innovation has (1) a relative advantage over current practices, (2) is easy to use, (3) is compatible with the values, beliefs, experiences of the users, (4) can be implemented on a trial basis before large scale implementation, and (5) the opportunity for others to observe implementation and the effects of implementation. Leff and colleagues followed these recommendations in implementing the Free2B anti-bullying program in 40 middle schools. The authors concluded that it was an attractive alternative to many anti-bullying programs because the intervention was delivered in a school assembly that schools were already providing, so it required no additional time allocation. Additionally, the video format made the delivery very easy compared to school-wide programs that are more time and resource intensive. The students reported that it addressed important topics. Prior to implementation, Leff and colleagues presented pilot data to key stakeholders at the state’s Office of Safe Schools who were able to leverage adoption by schools across the state. In addition to measuring adoption they also measured the impact on students and founds positive effects across all measures.
Citations: Leff, S. S., Waasdorp, T. E., Paskewich, B. S., & Winston, F. K. (2020). Scaling and Disseminating Brief Bullying Prevention Programming: Strengths, Challenges, and Considerations. School Psychology Review, 1-15.
Rogers, E. M. (1962). Diffusion of innovations. New York: Free Press of Glencoe.
Link: https://doi.org/10.1080/2372966X.2020.1851612
October 4, 2021
Overview of Professional Judgment. Educators make many decisions regarding services for students. Even when there is abundant evidence to guide their decisions, educators must use their judgment about what is appropriate in a given situation. Only on rare occasion does the available evidence perfectly match the service context of concern to the educator. To bridge the gap between research and local circumstance, the educator must make a series of judgments such as defining the problem, determining which evidence is relevant, and deciding which features of the local context are likely to require adaptations to the selected evidence-based intervention. Professional judgment is a cornerstone of evidence-based practice, as are best available evidence, stakeholder values, and the context in which services are provided. In this definition of evidence-based practice, the integration of these variables influences decisions. No one cornerstone can be substituted for the others. Judgment must be informed and constrained by the best available evidence, stakeholder values, and context.
Citation: Guinness, K., and Detrich, R. (2021). Overview of Professional Judgment. Oakland, CA: The Wing Institute. https://www.winginstitute.org/evidence-based-decision-making-professional-judgment.
Link: https://www.winginstitute.org/evidence-based-decision-making-professional-judgment
June 18, 2021
Cost-Effectiveness Analysis: A Component of Evidence-Based Education. Including cost-effectiveness data in the evaluation of programs is the next step in the evolution of evidence-based practice. Evidence-based practice is grounded in three complementary elements: best available evidence, professional judgment, and client values and context. To fully apply the cost-effectiveness data, school administrators will have to rely on all three of these elements. The function of cost-effectiveness data is to guide decisions about how limited financial resources should be spent to produce the best educational outcomes. To do so, it is necessary for decision makers to choose between options with known cost-effectiveness ratios while working within the budget constraints. In this article, I discuss some of the considerations that have to be addressed in the decision-making process and implications of including cost-effectiveness analyses in data-based decision making.
Citation: Detrich, R. (2020). Cost-effectiveness analysis: A component of evidence-based education. School Psychology Review, 1-8.
Link:
https://www.tandfonline.com/doi/abs/10.1080/2372966X.2020.1827864
June 18, 2021
How could evidence-based reform advance education? This article presents a definition and rationale for evidence-based reform in education, and a discussion of the current state of evidence-based research, focusing on China, the U.S., and the UK. The article suggests ways in which Chinese, U.S., UK, and other scholars might improve the worldwide quality of evidence-based reform in education. One indicator of this partnership is an agreement among the Chinese University of Hong Kong, Nanjing Normal University, and Johns Hopkins University to work together on Chinese and English versions of the website Best Evidence in Brief and a collaboration between Johns Hopkins and the ECNU Review of Education at East China Normal University.
The Wing Institute would like to acknowledge the contributions of Robert Slavin to the field of education. Our condolences go out to Robert Salvin’s family on the loss of one of America’s premier proponents of evidence-based education, who recently passed away on April 24, 2021. Robert Slavin was an education researcher who sought to translate the science of learning into effective teaching practices. Dr. Slavin was a distinguished professor at Johns Hopkins University’s School of Education, where he directed the Center for Research and Reform in Education
Citation: Slavin, R. E., Cheung, A. C., & Zhuang, T. (2021). How could evidence-based reform advance education?. ECNU Review of Education, 4(1), 7-24.
Link:
https://journals.sagepub.com/doi/full/10.1177/2096531120976060
June 17, 2021
Evidence-based decision-making: A team effort toward achieving goals. Implementing evidence-based practices requires not only knowledge of various interventions and practices but also professional judgment in selecting and applying an intervention that best meets the needs of the child and the family. Previous work on decision-making in evidence-based practices has focused on describing evidence-based practices, how the identification of evidence-based practices has affected the field of education (and, specifically, special education), and strategies for implementing evidence-based practices. The next logical step is in addressing how practitioners might make decisions about how to select evidence-based practices that match strengths and needs as well as contexts for children.
Citation: McCollow, M. M., & Hoffman, H. H. (2020). Evidence-based decision-making: A team effort toward achieving goals. Young Exceptional Children, 23(1), 15-23.
Link: https://journals.sagepub.com/doi/pdf/10.1177/1096250618801287
June 17, 2021
A Cost Analysis of the Innovation–Decision Process of an Evidence-Based Practice in Schools. The translation of evidence-based practices (EBPs) to improve students’ social, emotional, behavioral, and academic out- comes into authentic school settings has posed significant challenges for both researchers and practitioners. Among the many barriers to the adoption and use of EBPs are their associated costs. This study presents a framework for integrating the diffusion of innovation theory into an economic evaluation utilizing a societal perspective, which affords the capturing of costs of all phases from adoption through implementation of EBPs for all stakeholders.
Citation: Barrett, C. A., Pas, E. T., & Johnson, S. L. (2020). A Cost Analysis of the Innovation–Decision Process of an Evidence-Based Practice in Schools. School Mental Health, 12(3), 638-649.
Link: A Cost Analysis of the Innovation–Decision Process of an Evidence-Based Practice in Schools
May 19, 2021
Standardized Testing and the Controversy Surrounding It. The purpose of this paper is to provide a general understanding of standardized testing as well as the current controversy surrounding it, particularly in the context of performance-based accountability systems. The overview addresses the following questions related to standardized testing:
- What do stakeholders need to understand about standardized testing?
- What is the history of standardized tests and how have the tests been used?
- What are the reasons for the current controversy over standardized testing?
Citation: Polster, P.P., Detrich, R., & States, J., (2021). Standardized Testing: The Controversy Surrounding It. Oakland, CA: The Wing Institute. https://www.winginstitute.org/student-standardized-tests.
Link: https://www.winginstitute.org/student-standardized-tests
May 12, 2021
Back-to-school metrics: How to assess conditions for teaching and learning and to measure student progress during the COVID-19 pandemic. In response to the COVID-19 pandemic, the Regional Educational Laboratory (REL) Midwest is featuring a blog series addressing the many challenges that educators, caregivers, and students are facing. In this post, Susan Bowles Therriault, Ed.D., a managing researcher at the American Institutes for Research (AIR), describes school- and classroom-level metrics that administrators and teachers can use to assess teaching and learning conditions and measure student progress and engagement in a remote or hybrid learning setting.
Citation: Therriault, S. B. (2020). Back-to-school metrics: How to assess conditions for teaching and learning and to measure student progress during the COVID-19 pandemic. Regional Education Laboratory Program (REL).
Link: https://ies.ed.gov/ncee/edlabs/regions/midwest/blogs/back-to-school-metrics-covid.aspx
