Author Archives for Jack States
November 5, 2021 11:45 am
Published by Jack States
One of the great challenges in education is training all staff to implement interventions. There is considerable reliance on para-professionals, especially in special education, to support students. Many of the para-professionals have minimal training in educational practices. In many cases, the training that does occur is the traditional didactic model and there is little evidence that it produces the outcomes it is supposed to yield. An alternative model of training that holds great promise is coaching; however, there are limitations to it because it often relies on outside coaches which makes it cost-prohibited for many districts. A recent report by Sallese and Vannest (2021) offers an alternative that may make coaching more cost-effective. In their research, they utilized classroom teachers to coach the para-professionals working in the classroom to increase the use of behavior specific praise. Many teachers report that they have little or no pre-service or in-service training focused on paraprofessional training and support (Douglas, Chapin & Nolan, 2016). To address this issue, the teachers were provided a manual to guide their coaching efforts. The components of the coaching package included self-monitoring, performance feedback, goal setting, modeling, and action planning. In surveys of paraprofessionals one of the most cited concerns is lack of training and support in behavior management (Mason, et al., 2021). Behavior specific praise has been identified as an evidence-based component of classroom behavior management (Simonsen, Fairbanks, Briesch, Myers, & Sugai (2008); however, it has often been a challenge to increase behavior specific praise and maintain it over time. In this study, all four of the para-professionals that received coaching increased their rate of behavior specific praise. In addition, 100% of the participants agreed that the procedures were appropriate and feasible in terms of time and effort required to implement.
This was a small-scale study but it holds promise as a method for coaching implementers to carry out effective practices. From an implementation perspective, this provides a cost-effective approach to increase the internal capacity of a system to implement adopted practices. Building internal capacity is critical if effective interventions are to be sustained over generations of implementers.
Citation: Sallese, M. R., & Vannest, K. J. (2021). Effects of a Manualized Teacher-Led Coaching Intervention on Paraprofessional Use of Behavior-Specific Praise. Remedial and Special Education, 07419325211017298.
Link: https://journals-sagepub-com.dist.lib.usu.edu/doi/pdf/10.1177/07419325211017298?casa_token=SxqgzIctDbYAAAAA:m6SYJpowkUZud_eynTS6oZUX0Bbn2ExL87kZ7PO8fzmZz7Di2CSKl08A9KO2Wv1h32uaf68TQuja
References:
- Douglas, S. N., Chapin, S. E., & Nolan, J. F. (2016). Special education teachers’ experiences supporting and supervising paraeducators: Implications for special and general education settings. Teacher Education and Special Education, 39(1), 60–74. https://doi.org/gf86tz
- Mason, R. A., Gunersel, A. B., Irvin, D. W., Wills, H. P., Gregori, E., An, Z. G., & Ingram, P. B. (2021). From the frontlines: Perceptions of paraprofessionals’ roles and responsibilities. Teacher Education and Special Education, 44(2), 97–116. https://doi.org/fwn6
- Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and treatment of children, 351-380.
November 5, 2021 11:36 am
Published by Jack States
One of the persistent problems in education is the gap between what we know about effective educational practices and the practices that are frequently used in public schools. Many of these practices do not have empirical support. The challenge for all educators is how do we close the gap? The flow of research to practice is often perceived as being a one way flow from researchers that develop effective interventions and disseminate them to practitioners who are expected to adopt them (Ringeisen, Henderson, & Hoagwood, 2003). Ringeisen et al., argue that this is not likely to result in widespread adoption of effective practices. McLaughlin and colleagues (1997) have made the argument that having an array of effective practices is not sufficient for closing the research to practice gap. In many instances, the practices developed by researchers are not a good contextual fit for the school settings because training and experience requirements for implementers are unreasonable within the school setting, the resources necessary for implementation are not present, and the time demands to implement are unrealistic. If the dominant model of disseminating empirically-supported interventions is not impacting the research to practice gap, what should we do? The goal is important but we need effective alternatives to the common approach. Recently a report from the William T. Grant Foundation, (Farrell, Penuel, Coburn, Daniel, Steup (2021) entitled, Research-Practice Partnerships in Education: The State of the Field. In this report, the authors define research-practice partnerships as “intentionally organized to connect diverse forms of expertise and shift power relations in the research endeavor to ensure that all partners have a say in the joint work.” This is a significant shift from usual practice in the development and dissemination of effective practices. There are five principles associated with these partnerships: (1) they are long term collaborations (2) they work toward educational improvement or equitable transformation (3) they feature engagement with research as a leading activity (4) they are intentionally organized to bring together a diversity of expertise (5) they employ strategies to shift power relations in research endeavors to ensure that all participants have a say. This is an important shift. Practitioners are now partners with researchers. It is a movement away from the researcher as expert model to a model in which practitioners are equally expert as researchers. Each is an expert in different domains of improving educational practices.
If practitioners are involved from the beginning in guiding research then the practices are more likely to be seen as usable by educators when considering interventions to adopt. The development of research-practice partnerships has the potential to increase the adoption of empirically-supported practices.
Citation: Farrell, C.C., Penuel, W.R., Coburn, C., Daniel, J., & Steup, L. (2021). Research-practice partnerships in education: The state of the field. William T. Grant Foundation.
Link: http://wtgrantfoundation.org/research-practice-partnerships-in-education-the-state-of-the-field
References: McLaughlin, M. J., & Leone, P. E., Meisel, S., & Henderson, K. (1997). Strengthen school and community capacity. Journal of Emotional and Behavioral Disorders, 5(1), 15-24.
Ringeisen, H., Henderson, K., & Hoagwood, K. (2003). Context matters: Schools and the “research to practice gap” in children’s mental health. School Psychology Review, 32(2), 153-168.
October 5, 2021 10:35 am
Published by Jack States
A review of the evidence for real-time performance feedback to improve instructional practice. Performance feedback, including real time performance feedback, has been implemented across many contexts including educational settings. Sinclair, Gesel, LeJeune, and Lemons (2020) evaluated 32 studies that met their inclusion criteria to determine the effectiveness of real-time performance feedback in improving the instructional practices of educators. Interestingly, all but two of the studies utilized bug-in-the-ear technology in which the teacher wore an ear piece and a coach provided immediate feedback to the teacher as implementation of an intervention was occurring. In this review, teachers implementing both academic instruction and behavior management interventions were considered. Based on this review, the authors concluded that real time performance feedback was an evidence-based practice and could be used as a method for improving the performance of pre-service teachers, teachers, and paraprofessionals. The bug-in-the-ear technology offers several advantages. First, it is less intrusive than other methods for providing real time feedback. With current technology, the coach can view implementation in the classroom without being in the classroom. Technologies such as Go Pro and Swivl have sufficient flexibility for the coach to get a good sample of what is happening in the classroom. A second advantage is that because the feedback is immediate it is more likely to be effective compared to when the feedback is delayed. Finally, the bug-in-the-ear technology is time saving because the feedback is delivered in real-time. Brief follow-up meetings to discuss issues related to the intervention can be scheduled. Interestingly, bug-in-the-ear technology has been around for decades but has been under-utilized in educational settings. An analysis of the barriers to utilizing this technology more broadly is warranted. The potential for impact on implementation of an intervention is significant.
Citation: Sinclair, A. C., Gesel, S. A., LeJeune, L. M., & Lemons, C. J. (2020). A review of the evidence for real-time performance feedback to improve instructional practice. The Journal of Special Education, 54(2), 90-100.
Link: https://www.researchgate.net/publication/336116611_A_Review_of_the_Evidence_for_Real-Time_Performance_Feedback_to_Improve_Instructional_Practice
October 4, 2021 10:54 am
Published by Jack States
Electronically Delivered Support to Promote Intervention Implementation Fidelity: A Research Synthesis. Fundamental to any intervention outcome is the fidelity of implementation of the intervention. The ultimate goal of implementation science is to assure that innovations are implemented well enough for students to benefit. Failure to implement well can minimize the effectiveness of even the most powerful intervention. One of the challenges involved in insuring high quality implementation is that most approaches are resource intensive and often are not seen as feasible in school settings even though failure to achieve adequate implementation fidelity may result in a very poor benefit to cost ratio. One possible alternative is to utilize technology to reduce the resource demands. Fallon and colleagues (2021) conducted a systematic review to evaluate the effectiveness of technology-based supports to promote implementation fidelity. For the purposes of this review, “electronically delivered implementation supports (EDIS) was support delivered to an implementer electronically (e.g., via email, social media, video conferencing) for the purpose of improving educators’ implementation fidelity of a student intervention.” Fifteen studies met inclusion criteria and were judged to be of sufficient methodological rigor to warrant further analysis. The electronically delivered implementation supports ranged from video modeling, electronically delivered performance feedback, emailed intervention prompts, coaching via video conference, and online training modules. All of the studies were based on single participant designs. Since there are no agreed upon methods for calculating effect sizes for single participant designs, the authors calculated several different effect sizes (Tau-U, Standard Mean Difference, Hedges’ g, and a variation of Hedges’ g. In most of the studies, the effect sizes ranged from moderate to large regardless of the calculation method used. After completing the review, the authors provided guidance to educators about when to use the various methods of electronically delivered implementation supports. This article is a valuable resource to any educator considering implementing an intervention but is concerned about the resource requirements required for insuring high quality implementation. This article suggests that technology-based alternatives can be effective in supporting implementation and may reduce the overall demands on resources.
Citation: Fallon, L. M., Collier-Meek, M. A., Famolare, G. M., DeFouw, E. R., & Gould, K. M. (2020). Electronically Delivered Support to Promote Intervention Implementation Fidelity: A Research Synthesis. School Psychology Review, 1-16.
Link: https://doi.org/10.1080/2372966X.2020.1853485
October 4, 2021 10:45 am
Published by Jack States
School-Based Interventions for Middle School Students With Disruptive Behaviors: A Systematic Review of Components and Methodology. Middle school students are more likely than elementary or high school students to be disruptive (Erikson & Gresham, 2019). This presents difficult problems for classroom teachers trying to provide instruction and maintain order in the classroom. It has been estimated that 2.5 hours per week are lost to disruptive behavior each week (Education Advisory Board, 2019). This level of disruptive also contributes to teacher turnover with 39% of teachers reporting that disruptive behavior was one of the primary reasons for resigning (Bettini et al., 2020). To address challenges presented by middle school students, Alperin and colleagues (2021) completed a systematic review to identify programs that had a positive effect on disruptive behavior and the characteristics of those programs. They identified 51 studies that met their inclusion criteria. Of those 51 studies, 40 of them specified the function of behavior (gain attention or escape demands) that the program addressed; 16 of the studies included a home-based component with 7 of the studies providing parent training; 22 of the interventions had a manual guiding the implementation of the intervention; and encouragingly, 42 of the studies assessed intervention implementation. Effect sizes for seven of the studies were computed for intervention that involved class-wide intervention strategies. The effects ranged from small to large across the studies. Fourteen of the studies evaluated skill acquisition for small groups or individuals and the effect sizes again ranged from small to large. Seven of the studies evaluated reinforcement strategies for reinforcement-based interventions for small groups or individuals and reported effect sizes that ranged from small to large. Two studies evaluated interventions for escape from demands for small groups or individuals. Both of these studies reported large effect sizes. The data from this study are important as they can provide guidance to educators seeking to reduce disruptive behavior of middle school students. Ultimately, the educators will have to consider the contextual fit for each of these interventions for the settings in which they work. This study narrows the range of options to those that have some demonstrated level of effectiveness rather than leaving the educator to choose from all available options.
Citation: Alperin, A., Reddy, L. A., Glover, T. A., Bronstein, B., Wiggs, N. B., & Dudek, C. M. (2021). School-Based Interventions for Middle School Students With Disruptive Behaviors: A Systematic Review of Components and Methodology. School Psychology Review, 1-26.
Link: https://doi.org/10.1080/2372966X.2021.1883996
October 4, 2021 10:42 am
Published by Jack States
Scaling and Disseminating Brief Bullying Prevention Programming: Strengths, Challenges, and Considerations. One of the persistent problems in education and other human service disciplines is the research to practice gap (some would call it a chasm). In an effort to disseminate an effective bullying program (Free2B), Leff and colleagues applied the logic of Diffusion of Innovations (Rogers, 2003). This logic proposes that innovations are more likely to be adopted if the innovation has (1) a relative advantage over current practices, (2) is easy to use, (3) is compatible with the values, beliefs, experiences of the users, (4) can be implemented on a trial basis before large scale implementation, and (5) the opportunity for others to observe implementation and the effects of implementation. Leff and colleagues followed these recommendations in implementing the Free2B anti-bullying program in 40 middle schools. The authors concluded that it was an attractive alternative to many anti-bullying programs because the intervention was delivered in a school assembly that schools were already providing, so it required no additional time allocation. Additionally, the video format made the delivery very easy compared to school-wide programs that are more time and resource intensive. The students reported that it addressed important topics. Prior to implementation, Leff and colleagues presented pilot data to key stakeholders at the state’s Office of Safe Schools who were able to leverage adoption by schools across the state. In addition to measuring adoption they also measured the impact on students and founds positive effects across all measures.
Citations: Leff, S. S., Waasdorp, T. E., Paskewich, B. S., & Winston, F. K. (2020). Scaling and Disseminating Brief Bullying Prevention Programming: Strengths, Challenges, and Considerations. School Psychology Review, 1-15.
Rogers, E. M. (1962). Diffusion of innovations. New York: Free Press of Glencoe.
Link: https://doi.org/10.1080/2372966X.2020.1851612
October 4, 2021 10:36 am
Published by Jack States
Overview of Professional Judgment. Educators make many decisions regarding services for students. Even when there is abundant evidence to guide their decisions, educators must use their judgment about what is appropriate in a given situation. Only on rare occasion does the available evidence perfectly match the service context of concern to the educator. To bridge the gap between research and local circumstance, the educator must make a series of judgments such as defining the problem, determining which evidence is relevant, and deciding which features of the local context are likely to require adaptations to the selected evidence-based intervention. Professional judgment is a cornerstone of evidence-based practice, as are best available evidence, stakeholder values, and the context in which services are provided. In this definition of evidence-based practice, the integration of these variables influences decisions. No one cornerstone can be substituted for the others. Judgment must be informed and constrained by the best available evidence, stakeholder values, and context.
Citation: Guinness, K., and Detrich, R. (2021). Overview of Professional Judgment. Oakland, CA: The Wing Institute. https://www.winginstitute.org/evidence-based-decision-making-professional-judgment.
Link: https://www.winginstitute.org/evidence-based-decision-making-professional-judgment
August 31, 2021 10:24 am
Published by Jack States
Principal Evaluation. The field of principal evaluation, while gaining increased research interest in recent years, lags behind teacher evaluation in terms of conclusions that can be made regarding effective practice. Prior to Race to the Top and ESEA waivers, principal evaluation was implemented inconsistently and evaluation systems lacked instruments with validity and/or reliability, had a tenuous relationship with leadership standards, failed to include measures of student/school outcomes, and had mixed purposes as to their intended use (e.g., sometimes as formative information to help principals improve, while other times as summative information to make personnel decisions). However, today’s evaluation systems have evolved to incorporate multiple measures of principal performance that evaluate principals on research-based principles of effective leadership, often include student outcomes (which is often controversial, however), and are used both to help principals improve and to hold them accountable for their performance. Ongoing and more frequent observations, often conducted by the principal supervisor, who often also serves as a coach/mentor and directs the principal towards needed professional learning, show promise as an effective practice. Using the results from principal evaluations for personnel decisions, such as offering incentives through pay-for-performance programs, yields mixed results and warrants further research attention.
Citation: Donley, J., Detrich, R., States, J., & Keyworth, (2021). Principal Evaluation Oakland, CA: The Wing Institute. https://www.winginstitute.org/quality-leadership-principal-evaluation
Link: https://www.winginstitute.org/quality-leadership-principal-evaluation
August 25, 2021 10:19 am
Published by Jack States
Effective teacher retention bonuses: Evidence from Tennessee. The data are clear that students in high poverty schools perform worse on most measures of educational attainment; however, the discrepancy between high poverty schools and more affluent schools is reduced when there are quality teachers in the high poverty schools. The challenge is that teachers leave these schools at a higher rate. This turnover contributes to the poor outcomes for students in high poverty schools. Recruiting and training replacement teachers is an expensive proposition for districts. One approach to increasing retention in high poverty schools is to offer retention bonuses to teachers in these schools. There are two questions with respect to the use of retention bonuses: 1) are they effective over the long term, and 2) does having a more stable teachering corps increase student outcomes? A recent report examined the impact of teacher retention bonuses in Tennessee (Springer, Swain, & Rodriguez, 2016). The main findings are that teachers that participated in the retention bonus program were significantly more likely to stay in their school than teachers who did not participate. Importantly, the students in the classrooms of participating teachers had significantly higher academic gains than students of non-participating teachers. Looking at these data through the Active Implementation Frameworks lens, the retention bonus represents a usable innovation. The teacher retention bonuses are also an element of the Competency driver, specifically Selection. Finally, this innovation links to the Organizational driver since to effectively implement it, the innovation has to be considered a system level intervention.
Citation: Springer, M. G., Swain, W. A., & Rodriguez, L. A. (2016). Effective teacher retention bonuses: Evidence from Tennessee. Educational Evaluation and Policy Analysis, 38(2), 199-221.
Link: https://journals.sagepub.com/doi/abs/10.3102/0162373715609687
August 25, 2021 10:15 am
Published by Jack States
Driven by Data: Using Licensure Tests to Build a Strong, Diverse Teacher Workforce. Essential to improving educational outcomes for students is to assure that well prepared teachers are in every classroom. Teacher preparation programs are primarily responsible for preparing candidates. One measure of how well institutions are preparing teachers is the percentage of candidates that pass state licensure tests. The National Council on Teacher Quality (https://www.nctq.org/) recently released a report examining the pass rate of elementary education teachers by state, by de-identified teacher preparation institutions, and disaggregated data for candidates of color and socio-economic status. Different states have different standards, rely on different methods to assess performance, and have different criteria for passing scores. Thirty-four states provided complete data for this report, eight provided partial data, and nine states provided no data. Based on the available data, nationally 55% of teacher candidates failed the exam on their first try. The data vary considerably across states and across institutions within and across states. One of the conclusions of this report is that elementary teacher candidates, regardless of race and ethnicity, are “too often poorly prepared and supported to pass their state licensure tests.” The authors of the report identified a number of issues with how states are currently assessing teacher competency. The report concludes with a number of recommendations for improving teacher preparation programs so that more teachers pass the licensure test. These data are directly relevant to the competency implementation driver in the Active Implementation Frameworks. Implementation efforts are not likely to be successful if competent personnel are not available to implement the innovation. Competency is primarily the responsibility of the teacher preparation programs. These programs would be well served to attend to the recommendations of this report. In addition, education policy makers should review their state’s current methods for assessing the competency of teacher candidates.
Citation: Putman, H. & Walsh, K. (2021). Driven by Data: Using Licensure Tests to Build a Strong, Diverse Teacher Workforce. Washington, D.C.: National Council on Teacher Quality
Link: https://www.nctq.org/publications/Driven-by-Data:-Using-Licensure-Tests-to-Build-a-Strong,-Diverse-Teacher-Workforce/
