Instructional Effectiveness

For the best chance of a positive impact on educational outcomes, two conditions must be met: (a) Effective interventions must be adopted, and (b) those interventions must be implemented with sufficient quality (treatment integrity) to ensure benefit.  To date, emphasis in education has been on identifying effective interventions and less concern with implementing the interventions. The research on the implementation of interventions is not encouraging. Often, treatment integrity scores are very low and, in practice, implementation is rarely assessed. If an intervention with a strong research base is not implemented with a high level of treatment integrity, then the students do not actually experience the intervention and there is no reason to assume they will benefit from it. Under these circumstances, it is not possible to know if poor outcomes are the result of an ineffective intervention or poor implementation of that intervention. Historically, treatment integrity has been defined as implementing an intervention as prescribed. More recently, it has been conceptualized as having multiple dimensions, among them dosage and adherence which must be measured to ensure that it is occurring at adequate levels.

Summative assessment is an appraisal of learning at the end of an instructional unit or at a specific point in time. It compares student knowledge or skills against standards or benchmarks. Summative assessment includes midterm exams, final project, papers, teacher-designed tests, standardized tests, and high-stakes tests. 

The purpose of this analysis is to examine data from the National Schools and Staffing Survey on principal years of experience on the job.

Three 8th grade English teachers participated in this single-case multiple baseline experiment. These teachers were observed daily during classes that were inclusive to students with disabilities. Observations were conducted using the Classroom Teaching Scan (www.classroomteachingscan.com/ctscan/). Within the Classroom Teaching Scan, a checklist of quality indicators for modeling was the primary dependent variable. Additionally, observations were scored using the Protocol for Language Arts Teaching Observations (PLATO, 2017). Participating students responded to curriculum-based measurement writing prompts throughout the study.

Minor changes in performance on the PLATO and CBM measures were demonstrated. However, these measures were descriptive in nature, not experimental. Therefore, more research over a sustained period of time is necessary to determine the effect of this professional development package on distal measures of teacher quality and student outcomes.