Can the Knowledge Is Power Program (KIPP) and Promise Academies of the Harlem Children's Zone (HCZ) charter school model be replicated on a national scale?

Why is this question important? For more than 30 years a spotlight has been aimed at school reform in the United States. Initiatives such as No Child Left Behind have raised expectations and made school systems more accountable for raising student test scores and finding ways to bridge the achievement gap between economically disadvantaged students and students who come from higher socio-economic backgrounds. Unfortunately, improvements have been limited, and all too often school districts have failed to meet the challenge. Indicators such as the National Assessment of Education Progress (NAEP) and international comparisons such as the Program for International Student Assessment (PISA) have underscored the lack of improvement. The public, policy makers, and educators are now looking to charter schools such as KIPP and HCZ as a model for changing this discouraging picture. Despite indications that such a model can make a difference, important questions remain to be answered; for example, is hope warranted and can this model be replicated across the nation? A recent study, A Re-Analysis of the Effects of KIPP and the Harlem Promise Academies (Yeh, 2013), seeks to answer these questions.

See further discussion below.

Results:
Knowledge Is Power Program (KIPP) academic performance

Yeh's analysis of KIPP academic performance is based on the research conducted by Tuttle, Teh, Nichols-Barrer, Gill, and Gleason (2010). Yeh found KIPP's impact on student achievement to be socially significant. He reported that after 3 years KIPP students had gained an average of 0.14 standard deviations (SD) per year in mathematics and 0.08 per year in reading. In their study, Tuttle et al. reported a cumulative gain of 0.48 standard deviations in mathematics. This is equivalent to moving a student from the 30th percentile to the 48th percentile on a standard test distribution or 1.2 years of accumulated added growth in mathematics.

According to the Tuttle study, the reported effect on reading scores was also meaningful but not as dramatic as the effect on mathematics. The Tuttle analysis identified a 3-year effect of 0.28 SD, resulting in an estimated 0.9 years of additional learning, or about one third of the black-white gap in reading achievement in fourth and eighth grades.

Harlem Children's Zone (HCZ) Performance Academies academic performance

After reviewing rigorous research by Dobbie and Fryer (2011) on HCZ's impact on student achievement randomly compared with the achievement of students who did not attend HCZ schools, Yeh concluded that the effects were both statistically and socially significant. The data showed that attending an HCZ charter middle school was enough to close the black-white achievement gap in mathematics and reduce it by nearly half in reading.

HCZ elementary students gained 0.191 SD per year in mathematics and 0.095 per year in reading. Students cumulatively gained approximately .8 to 1.5 SD in both mathematics and reading by third grade.

HCZ middle-school students gained 0.229 SD per year in mathematics and 0.047 per year in reading. The cumulative gain in mathematics at an HCZ school was more than .8 SD by sixth grade, and between .25 and .3 SD in reading by eighth grade.

To keep these gains in perspective, a student whose performance improves by 1 SD can expect to earn 10% to 15% more after entering the work force (Dobbie & Fryer, 2011).

Teacher attrition at KIPP and HCZ

Yeh examined the issue of teacher retention, which has often been highlighted as a positive in the literature of both KIPP and HCZ. Over the past decade, the average annual attrition rate for public school teachers across the United States is between 14% and 16%. Available data for KIPP and HCZ turnover rates reveal annual losses between 30% and 40%. Yeh has raised this significant difference in teacher turnover rates as an issue of concern.

The very high demands that both KIPP and HCZ place on teachers have been noted as a reason for high turnover. At KIPP, teachers are expected to be available 24 hours a day by cell phone to respond to student concerns. Teachers are expected to work 9½-hour days, teach on Saturday, and work during the summer, "all of which are non-negotiable" (Carter, 1999).

Demands on HCZ teachers are also weighty. HCZ teachers work several hundred more hours than New York City teachers in traditional public schools. These extra hours result from longer workdays, longer school years, and a summer session that runs through the first week of August (Tough, 2008).

Implications: The effects on mathematics and reading achievement for students attending KIPP and HCZ schools are meaningful and substantiated through studies using rigorous research methods (Dobbie & Fryer, 2011; Tuttle et al., 2010).

Unfortunately, studies of KIPP and HCZ suggest a parallel story that takes much of the luster off the promising picture that these studies reveal. To achieve their results, both KIPP and HCZ rely heavily on highly dedicated teachers working long hours under exhausting conditions, resulting in high teacher attrition. Such heavy demands on teachers are core to the philosophy and practices embraced by these two charter programs.

Yeh contends that the hope represented by implementation of the KIPP and HCZ model in every school across the nation falls short because it is based on a "heroic teacher" model. His analysis suggests that gains from the KIPP and HCZ model will fall to zero once the model is taken to scale. The Achilles heel dooming this model as a solution to the larger challenges confronting American schools is high teacher attrition and hoarding of a disproportionate share of the nation's limited pool of highly dedicated teachers. Yeh's analysis demonstrates how current gains are unlikely to be sustained when the programs are scaled and implemented nationwide, draining the pool of the most highly qualified and dedicated teachers who make it possible for KIPP and HCZ to achieve their results. Such significant teacher turnover as a method for weeding out less competent teaching staff will not hold up as a model for American schools.

Study Description: The study design included (1) review of the results of the key impact studies, (2) review of the evidence of teacher attrition, (3) review of statements by the founders of KIPP and HCZ articulating their core assumptions, (4) a narrative explanation of the internal contradiction in this model, and (5) a formal analysis demonstrating the consequences of this contradiction.

To be included in Yeh's analysis, a study on student achievement had to use a matched comparison group design. The KIPP analysis used a matched comparison group design that included a nationwide sample of 22 KIPP schools (Tuttle et al., 2010). The HCZ study used a lottery-based randomization method to achieve matched comparison groups (Dobbie & Fryer, 2011).

To evaluate teacher attrition, Yeh selected a Stanford Research Institute (SRI) study that investigated teacher attrition at five San Francisco Bay Area KIPP schools in 2006–07 (Woodworth, David, Guha, Wang, & Lopez-Torkos, 2008). Yeh examined a 3-year period of attrition, 2006–08, to establish an turnover rate for HCZ schools (Dobbie & Fryer, 2011).

Citation:
Carter, S.C. (1999). No excuses: Seven principals of low-income schools who set the standard for highachievement. Washington, DC: Heritage Foundation.

* Cobey, W. et al., (2013). Report to the North Carolina General Assembly: 2012–2013 annual report on teachers leaving the profession. G.S. 115C–12 (22). North Carolina Department Of Public Instruction, Educator Effectiveness Division. Retrieved November 10, 2014 from https://eboard.eboardsolutions.com/meetings/TempFolder/Meetings/Attachment%201%20-%202012-13%20Teacher%20Turnover%20Report_19785qy0dwx45ftvhsa55k1f0mjyd.pdf

Dobbie, W., & Fryer, R. G., Jr. (2011). Are high-quality schools enough to increase achievement among the poor? Evidence from the Harlem Children's Zone. American Economic Journal: Applied Economics, 3(3), 158–187.

* Goldring, R., Taie, S., & Riddles, M. (2014). Teacher attrition and mobility: Results From the 2012–13 Teacher Follow-up Survey (NCES 2014-077). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved November 10, 2014 from http://nces.ed.gov/pubs2014/2014077.pdf

* Teacher turnover report: Annual report on the reasons teachers leave, 2007-2008. (2008). Public Schools of North Carolina, Department of Public Instruction, Talent Management and Development Division. Retrieved November 12, 2014 from http://www.ncpublicschools.org/newsroom/news/2007-08/.

* Teacher turnover report: Annual report on the reasons teachers leave, 2012–2013. (2013). Public Schools of North Carolina, Department of Public Instruction, Talent Management and Development Division. Retrieved November 12, 2014 from http://www.ncleg.net/documentsites/committees/JLEOC/Reports%20Received/Archives/2008%20Reports%20Received/Teacher%20Turnover%20Report.pdf.

Tough, P. (2008). Whatever it takes: Geoffrey Canada's quest to change Harlem and America. Boston, MA: Houghton Mifflin Harcourt.

Tuttle, C. C., Teh, B., Nichols-Barrer, I., Gill, B. P., & Gleason, P. ( 2010). Student characteristics and achievement in 22 KIPP middle schools. Washington, DC: Mathematica Policy Research.

Woodworth, K. R., David, J. L., Guha, R., Wang, H., & Lopez-Torkos, A. (2008). San Francisco Bay Area KIPP schools: A study of early implementation and achievement. Final report. Menlo Park, CA: SRI International.

* Yeh, S. S. (2013). A re-analysis of the effects of KIPP and the Harlem Promise Academies. Teachers College Record, 115(4).
* KIPP: 2013 Report card (2013). Retrieved November 12, 2014 from http://www.kipp.org/reportcard.
* Studies from which graphed data were derived