Categories for Effective Instruction
June 27, 2018
Four Domains for Rapid School Improvement: An Implementation Framework
The National Implementation Research Network (NIRN) and the National Center for School Turnaround published the “Four Domains for Rapid School Improvement: An Implementation Framework” as a companion to the Center for School Turnaround’s publication of “The Four Domains for Rapid School Improvement: A Systems Framework“. This paper describes “how” to effectively implement lasting school improvement initiatives that maximize leadership, develop talent, amplify instructional transformation, and shift the culture.
Citation:Jackson, K., R., Fixsen, D., and Ward, C. (2018). Four Domains for Rapid School Improvement: An Implementation Framework. The Center on School Turnaround.
Link: https://files.eric.ed.gov/fulltext/ED583980.pdf
June 26, 2018
The Impact of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation
A recent large-scale evaluation of Reading Recovery, a supplemental reading program for young struggling readers, supports previous research that found it to be effective. In a 4 year, federally funded project, almost 3,500 students in 685 schools found that generally students benefitted from the intervention. Students receiving Reading Recovery receive supplemental services in a 1:1 instructional setting for 30 minutes 5 days a week from an instructor trained in Reading Recovery. In the study reported here, students who received Reading Recovery had effect sizes of .35-.37 relative to a control group across a number of measures of reading. These represent moderate effect sizes and account for about a 1.5 month increase in skill relative to the control group. Even though the research supports the efficacy of the intervention, it also raises questions about its efficiency. The schools that participated in the study served about 5 students and the estimated cost per student has ranged from $2,000-$5,000. These data raise questions about the wisdom of spending this much money per student for growth of about a month and a half.
Citation: Sirinides, P., Gray, A., & May, H. (2018). The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation. Educational Evaluation and Policy Analysis, 0162373718764828.
Link: http://journals.sagepub.com/doi/abs/10.3102/0162373718764828
May 11, 2018
2017-18 Wing Institute Student Research Grant Results
Title: Evaluating a Multimedia Professional Development Package for Improving Implementation of Evidence-Based Instructional Practices
Author: John Elwood Romig, Ph.D.
Research Proposal: Explicit modeling is an effective instructional strategy in a wide spectrum of academic and behavioral skills (Hughes, Morris, Therrien, & Benson, 2017). Specifically, in the area of writing instruction, modeling is widely regarded as an evidence-based instructional strategy to improve students’ achievement (Troia, 2014). However, teachers rarely use modeling or other high quality instructional practices during writing instruction (Graham, Capizzi, Harris, Hebert, & Morphy, 2014; Gillespie, Graham, Kiuhara, & Hebert, 2014; Grisham & Wolsey, 2011).
The purpose of this study was to examine the effect of a professional development package designed using cognitive apprenticeship (i.e., Collins, Brown, & Newman, 1989) as a guiding framework. This professional development package, Content Acquisition Podcast – Professional Development (CAP-PD), consisted of modeling videos, performance feedback, and instructional scaffolds in the form of customizable curricular materials. To determine the effectiveness of this package, I conducted a single case multiple baseline across participants experiment. Read more
May 1, 2018
The National Assessment of Educational Progress (NAEP) has been assessing what American students know and can do in various subject areas since 1969. Subjects include: mathematics, reading, science, writing, the arts, civics, economics, geography, U.S. history, and in Technology and Engineering Literacy (TEL). NAEP generates the Nation’s Report Card every two years summarizing test scores by subject area, grade levels (4th, 8th, and 12th), student demographics, school characteristics, etc. With very few exceptions, the 2017 national results did not show any gains in test scores from previous years. One of the more critical metrics is the percentage of students who score at or above proficiency. Proficiency is the target academic performance benchmark for students. The 2017 results show that few students are at or above proficiency in all subjects. In reading, only 37% of fourth graders, 36% of eighth graders, and 357% of twelfth grader were at or above proficiency. In math, each subsequent assessed grade showed lower percentages of students at or above proficiency (4thgrade at 40%, 8thgrade at 34%, 12thgrade at 25%). Such scores are disappointing given the efforts made towards school improvement.
Citation: Institute of Education Sciences. (2018). The Nation’s Report Card, 2017. NCES 2006-451. National Center for Education Statistics.
Link: https://www.nationsreportcard.gov
April 23, 2018
A Synthesis of Quantitative Research on Reading Programs for Secondary Students
This review of the research on secondary reading programs focuses on 69 studies that used random assignment (n=62) or high-quality quasi-experiments (n=7) to evaluate outcomes of 51 programs on widely accepted measures of reading. Reading performance of students in America’s middle and high schools is one of the most important problems in education. In 2015, the National Assessment of Educational Progress (NAEP; NCES, 2016) reported that only 34% of eighth graders scored at or above proficiency. At the twelfth-grade level only 37% of students scored at or above proficiency. Given the importance of mastering reading for success in school it is important that schools adopt programs to help bridge the gap between current performance and expectations.
The study found programs using one-to-one and small-group tutoring (+0.14 to +0.28 effect size), cooperative learning (+0.10 effect size), whole-school approaches including organizational reforms such as teacher teams (+0.06 effect size), and writing-focused approaches (+0.13 effect size) showed positive outcomes. Individual approaches in a few other categories also showed positive impacts. These include programs emphasizing social studies/science, structured strategies, and personalized and group/personalization rotation approaches for struggling readers.
An important finding was programs providing a daily extra period of reading and those utilizing technology were no more effective, on average, than programs that did not provide these resources. This might not be as surprising as it appears on the surface. If students are struggling in reading, just placing them in a setting to read may only produce additional frustration and failure for those currently struggling with reading.
A few commonalities among programs that achieved positive outcomes are worth noting. One of these factors was that programs with positive outcomes tended to emphasize student motivation, student-to-student and student-to-teacher relationships, and social-emotional learning. An additional factor found in many of the promising programs is individualizing the intervention.
The findings are important suggesting interventions for secondary readers to improve struggling student’s chances of experiencing greater success in high school and better opportunities after graduation. Although, these effect sizes are small, given the large number of participants, smaller effect sizes would be anticipated and still may be of interest to school administrators.
Citation: Baye, A., Lake, C., Inns, A. & Slavin, R. E. (2018, January). A Synthesis of Quantitative Research on Reading Programs for Secondary Students. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.
Link: https://ila.onlinelibrary.wiley.com/doi/abs/10.1002/rrq.229
March 22, 2018
Promoting Educator Effectiveness: The Effects of Two Key Strategies
The National Center for Education Evaluation, a division of the Institute of Education Sciences has released a new research brief that evaluated two strategies for improving educator effectiveness as measured by improvements in student outcomes. The two strategies evaluated were performance feedback to educators about several dimensions of their performance for a period of two years and a pay-for-performance system that was in place for four years. In the performance feedback project teachers were given feedback four times per year on their classroom practices and principals received feedback two times per year. The impact on student outcomes were small. After the first year, there was a statistically significant difference between students in math but not in reading in the feedback schools compared to the schools that served as the control group as measured by end of year scores. At the end of the second year there were no statistically significant effects for either reading or math. The net gain in math achievement for the students in the feedback schools was about 4 weeks compared to the control group.
The pay-for-performance study teachers were eligible for performance bonuses based on their ratings across multiple dimensions of their performance. The students in the pay-for-performance schools outperformed the students in the control group schools in both math and reading. Statistically significant scores were obtained in reading beginning the first year and each subsequent year through the third year. Students math scores in the pay-for-performance schools achieved statistically significant scores only at the end of the third year. There was no additional benefit in reading or math for pay-for-performance in the fourth year. The overall benefit of the gains by the students in the pay-for-performance schools was estimated to be 3-4 weeks. Again, this is a relatively small impact. It was noted that the quality of implementation may have reduced the impact of the two projects. Across both studies, there were discrepancies between how the programs were intended to be implemented and how they were actually implemented. Further limitations of the studies are that the performance feedback for teachers was only four times per year. This is generally considered to be far too infrequent to have meaningful impact. In the pay-for-performance study, 40% of the teachers reported that they were not aware they were eligible for bonuses, limiting the motivational properties of the bonus system. Given these results it is clear that we must continue searching for effective approaches to improving educator performance and ways to assure high quality implementation.
Citation: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences (March 2018). Promoting Educator Effectiveness: The Effects of Two Key Strategies.
Link: https://ies.ed.gov/ncee/pubs/20184009/index.asp
March 21, 2018
The Effectiveness of School-Based Mental Health Services for Elementary-Aged Children: A Meta-Analysis
News Summary: This meta-analysis examines the effects of school-based mental health services for elementary school-age children delivered by school personnel. Forty-three controlled trials evaluating 49,941 elementary school-age children met criteria for inclusion in this study. The study used a randomized, between-subjects, controlled comparison or quasi-experimental design using matched samples to minimize selection bias. The study finds school-based mental health services had a small to medium effect size (Hedges g = 0.39) in decreasing mental health problems. The largest effect size was for targeted intervention, (Hedges g = 0.76), followed by selective prevention (Hedges g = 0.67) compared with universal prevention (Hedges g = 0.29). Interventions integrated into student’s academic instruction using contingency management were found to have positive impacts (Hedges g = 0.57), and interventions implemented multiple times per week (Hedges g = 0.50) were also shown to have a notable impact for improving student’s lives. These results are promising considering the normal barriers that impede students from receiving mental health care outside of school and the fact 80% of mental health service are provided in schools by personnel who are readily available and are shown to be effective in addressing student’s mental health needs (Ringeisen, Henderson, and Hoagwood, 2003).
Definitions
- Targeted Intervention: interventions provided only to students identified as having mental health problems.
- Universal prevention: interventions provided to all students in a classroom
- Selective prevention: interventions provided only to students at risk for mental health problems according to a teacher referral or mental health screening
Citation: Sanchez, A. L., Cornacchio, D., Poznanski, B., Golik, A. M., Chou, T., & Comer, J. S. (2018). The effectiveness of school-based mental health services for elementary-aged children: a meta-analysis. Journal of the American Academy of Child & Adolescent Psychiatry, 57(3), 153-165.
Link: http://www.jaacap.com/article/S0890-8567(17)31926-3/fulltext
March 15, 2018
Research on Mathematics Instruction with Students with Significant Cognitive Disabilities: Has Anything Changed?
News Summary: This research examines mathematics instruction for learners of significant cognitive disabilities. This study builds on the previous meta-analysis by Browder et al. (2008) and has added an additional 29 studies. The purpose of this literature review was to identify research of teaching mathematics skills published since 2006 and to evaluate the evidence of instructional practices used in these studies. The review also attempts to examine if any progress has been made in implementing five strands of mathematics instruction identified in the National Council for Teachers of Mathematics (NCTM; (2000) recommendations. The five strands for effective instruction of mathematics are: (1) Number and Operations, (2) Algebra, (3) Geometry, (4) Measurement, (5) Data Analysis and Probability. The criteria for quality of research developed by Horner and colleagues for single subject designed research was used to review the studies (Horner et al., 2005). These standards require that to be included in this review a minimum of five single-case studies must be conducted by a minimum of three different researchers across a minimum of three different geographical regions with no less than 20 participants be required for a practice to be considered evidence-based. The data from both reviews were combined as well as they were compared. The results show more studies since 2008 taught skills from Number and Operations, Geometry, and Algebra. Additionally, the study found that the teaching of Measurement decreased and Data Analysis and Probability remained unchanged. The systematic analysis conducted by the study of specific instructional practices found systematic instruction, in vivo instruction, system of least prompts strategy, constant time delay strategy, and task-analytic instruction met criteria for being considered evidence-based practices for teaching mathematics to learners with significant cognitive disabilities.
Browder, D. M., Spooner, F., Ahlgrim-Delzell, L., Harris, A. A., & Wakemanxya, S. (2008). A meta-analysis on teaching mathematics to students with significant cognitive disabilities. Exceptional children, 74(4), 407-432.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional children, 71(2), 165-179.
National Council of Teachers of Mathematics (Ed.). (2000). Principles and standards for school mathematics (Vol. 1). National Council of Teachers of.
Citation: Hudson, M. E., Rivera, C. J., & Grady, M. M. (2018). Research on Mathematics Instruction with Students with Significant Cognitive Disabilities: Has Anything Changed?. Research and Practice for Persons with Severe Disabilities, 1540796918756601.
Link: https://eric.ed.gov/?q=education+meta-analysis&ff1=dtyIn_2018&id=EJ1169270
https://www.researchgate.net/profile/Christopher_Rivera2/publication/322762194_Research_on_Mathematics_Instruction_with_Students_with_Significant_Cognitive_Disabilities_Has_Anything_Changed/links/5a84d7a1aca272c99ac38c01/Research-on-Mathematics-Instruction-with-Students-with-Significant-Cognitive-Disabilities-Has-Anything-Changed.pdf
March 8, 2018
What We Know About Literal and Inferential Comprehension in Reading
In 2000, the National Reading Panel identified five practice elements with a sufficient evidence base to be deemed essential for mastery of reading (National Institute of Child Health and Human Development, 2000). These elements consist of systematic teaching of phonemic awareness, phonics instruction, vocabulary, fluency, and exposure to reading comprehension strategies. Of these skill sets, reading comprehension has received far less attention in the literature, but is indispensable for success as student’s progress through the grades and use reading in almost every course. Being able to make effective inferences from materials that a student read is considered central to effective reading comprehension. This meta-analysis of 25 studies evaluates the impact of inference instruction in grades K-12. The study reported that inference instruction had an effect size d=0.58 on general comprehension and d= 0.68 on literal comprehension. These are “moderate to large” effects of instruction on general comprehension and to making inferences for both skilled and less skilled readers. The pattern differed for the literal measure, however, with skilled readers showing almost no gain but unskilled readers showing sizable gains. These findings support work by Daniel Willingham and Gail Lovette titled, “Can Reading Comprehension be Taught”? Their interpretation of the effect of comprehension instruction is that it signals to students the significance of inferential thinking. Willingham and Lovette conclude that practicing inferences does not lead to increases in general inferencing for the following reasons; inferencing depends on the particular text, and whatever cognitive processes contribute to inferencing are already well practiced in oral language as we are constantly drawing inferences in daily conversation.
Citation: Elleman, A. M. (2017). Examining the impact of inference instruction on the literal and inferential comprehension of skilled and less skilled readers: A meta-analytic review. Journal of Educational Psychology, 109(6), 761-781.
Willingham, D. T., & Lovette, G. (2014). Can reading comprehension be taught. Teachers College Record, 116, 1-3
Link: http://psycnet.apa.org/record/2017-06326-001
http://www.danielwillingham.com/uploads/5/0/0/7/5007325/willingham&lovette_2014_can_reading_comprehension_be_taught_.pdf
February 28, 2018
WWC Examines the Evidence on Two Teacher Training Programs
News Summary: If teachers are to have a significant impact on student learning it is necessary for them to be well trained and prepared for the role of teacher. This report examined the effectiveness of the National Board for Professional Teaching Standards (NBPTS) and My TeachingPartner Pre-K. The NBPTS is a professional certification program for teachers that have taught at least three years and can meet the NBPTS standards. My TeachingPartner Pre-K incorporates multiple media and coaching to prepare early education teachers. The results of the What Works Clearinghouse review of NBPTS is that it had mixed effects in mathematics in grades 3-8 and no discernable effect on English language arts achievement. There were no studies that met WWC standards for review so no judgment can be made about its effectiveness. The results of this review highlight the necessity of evaluating the effectiveness of teacher training programs. The stakes are very high for the students and families being served by teachers and nationally very large amount of money is spent on training teachers. It would be nice to know which approaches to teacher professional development are effective and which have no beneficial effect.
Citation: What Works Clearinghouse, Institute for Education Science (2018). National Board for Professional Teaching Standards Certification Intervention Report. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/689
Link: https://ies.ed.gov/ncee/wwc/InterventionReport/689
