Education Drivers

Continuum of Evidence

At a time of diminishing student performance and constrained education resources, it is critical that policy makers, educators, and parents make informed decisions about education interventions on the basis of sound science and thoughtful social influence. The Wing Institute, an independent and non-profit organization, was created to promote evidence-based education policies and practices to address these serious challenges facing education today.

The Wing Institute is named in memory of Ernie Wing, who championed evidenced-based education as an educator and child advocate. Ernie set the standards for professionalism, integrity, effectiveness, and caring as he helped thousands of children gain access to effective educational services.

Publications

TITLE
SYNOPSIS
CITATION
Evidence-Based, Empirically Supported, OR Best Practice?

Evidence-based, empirically-supported, and best practice are often used interchangeably. A case is made that for clarity each term should have a separate and distinct meaning.

Detrich, R. (2008). Evidence-Based, Empirically Supported, OR Best Practice?. Effective practices for children with autism, 1.

A roadmap to evidence-based education: Building an evidence-based culture

Increasing education’s reliance on evidence to guide decisions requires a significant change in the culture of districts and schools. This paper reviews the implications of moving toward evidence-based education.

Detrich, R., Keyworth, R., & States, J. (2007). A Roadmap to Evidence-based Education: Building an Evidence-based Culture. Journal of Evidence-based Practices for Schools, 8(1), 26-44.

Evidence-Based Education and Best Available Evidence: Decision-Making Under Conditions of Uncertainty

Evidence-based practice is a framework for decision making.  Even with high quality evidence there are likely sources of uncertainty that practitioners must confront.

Detrich, R., Slocum, T. A., & Spencer, T. D. (2013). Evidence-based education and best available evidence: decision-making under conditions of uncertainty. Evidence-Based Practices, 26, 21.

Roles and responsibilities of researchers and practitioners for translating research to practice

This paper outlines the best practices for researchers and practitioners translating research to practice as well as recommendations for improving the process.

Shriver, M. D. (2007). Roles and responsibilities of researchers and practitioners for translating research to practice. Journal of Evidence-Based Practices for Schools, 8(1), 1-30.

Evaluating the Validity of Systematic Reviews to Indentify Empirically Supported Treatments

The systematic review process is an assessment and, as such, concerns about validity of the assessment are paramount.  In this paper, we review the considerations that are important in reaching conclusions about the adequacy of a systematic review.

Slocum, T. A., Detrich, R., & Spencer, T. D. (2012). Evaluating the validity of systematic reviews to identify empirically supported treatments. Education and Treatment of Children, 35(2), 201-233.

The Evidence-Based Practice of Applied Behavior Analysis

Applied behavior analysis emphasizes being scientifically-based In this paper, we discuss how the core features of evidence-based practice can be integrated into applied behavior analysis.

Slocum, T. A., Detrich, R., Wilczynski, S. M., Spencer, T. D., Lewis, T., & Wolfe, K. (2014). The Evidence-Based Practice of Applied Behavior Analysis. The Behavior Analyst, 37(1), 41-56.

Best Available Evidence: Three Complementary Approaches

The notion of best available evidence implies that some evidence is better than other. This paper reviews different sources of evidence and the relative strengths and limitations of each type.

Slocum, T. A., Spencer, T. D., & Detrich, R. (2012). Best available evidence: Three complementary approaches. Education and Treatment of Children, 35(2), 153-181.

Evidence-based Practice: A Framework for Making Effective Decisions

Synopsis: Evidence-based practice is characterized as a framework for decision-making integrating best available evidence, clinical expertise, and client values and context.  This paper reviews how these three dimensions interact to inform decisions.

Spencer, T. D., Detrich, R., & Slocum, T. A. (2012). Evidence-based practice: A framework for making effective decisions. Education and Treatment of Children, 35(2), 127-151.

Framework for Improving Education Outcomes

Multitiered system of support (MTSS) is a framework for organizing service delivery. At the core of MTSS is the adoption and implementation of a continuum of evidence-based interventions that result in improved academic and behavioral outcomes for all students. MTSS is a data-based decision making approach based on the frequent screening of progress for all students and intervention for students who are not making adequate progress.

 

States, J., Detrich, R., and Keyworth, R. (2017). Multitiered System of Support Overview. Oakland, Ca. The Wing Institute.

School-wide positive behavior support: Establishing a continuum of evidence-based practices

The purpose of this article is to describe how effective practices are incorporated into an approach termed schoolwide positive behavior supports (SWPBS)

Sugai, G., & Horner, R. H. (2010). School-wide positive behavior support: Establishing a continuum of evidence based practices. Journal of Evidence-Based Practices for Schools11(1), 62-83.

Identifying research-based practices for response to intervention: Scientifically-based instruction

This paper examines the types of research to consider when evaluating programs, how to know what “evidence’ to use, and continuums of evidence (quantity of the evidence, quality of the evidence, and program development).

Twyman, J. S., & Sota, M. (2008). Identifying research-based practices for response to intervention: Scientifically based instruction. Journal of Evidence-Based Practices for Schools, 9(2), 86-101.

 

Presentations

TITLE
SYNOPSIS
CITATION
Roles and Responsibilities of Researchers and Practitioners Translating Research to Practice

This paper outlines the best practices for researchers and practitioners translating research to practice as well as recommendations for improving the process.

Shriver, M. (2006). Roles and Responsibilities of Researchers and Practitioners Translating Research to Practice [Powerpoint Slides]. Retrieved from 2006-wing-presentation-mark-shriver.

Evolution of the Revolution: How Can Evidence-based Practice Work in the Real World?
This paper provides an overview of the considerations when introducing evidence-based services into established mental health systems.
Chorpita, B. (2008). Evolution of the Revolution: How Can Evidence-based Practice Work in the Real World? [Powerpoint Slides]. Retrieved from 2008-wing-presentation-bruce-chorpita.
If We Want More Evidence-based Practice, We Need More Practice-based Evidence
This paper discusses the importance, strengths, and weaknesses of using practice-based evidence in conjunction with evidence-based practice.
Cook, B. (2015). If We Want More Evidence-based Practice, We Need More Practice-based Evidence [Powerpoint Slides]. Retrieved from 2015-wing-presentation-bryan-cook.
From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education
Evidence-based practice is a decision making framework. This talk reviews the types of evidence that can be used in decision-making and when each source of evidence is best used.
Detrich, R. (2006). From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education [Powerpoint Slides]. Retrieved from 2006-calstatefresnoaba-presentation-ronnie-detrich.
An Expanded Model of Evidence-based Practice in Special Education
This paper reviews the types of evidence that can used to guide decision-making in special education as well as the necessity for high quality implementation, and monitoring the effects of intervention.
Detrich, R. (2006). An Expanded Model of Evidence-based Practice in Special Education [Powerpoint Slides]. Retrieved from 2006-campbell-presentation-ronnie-detrich.
Single Subject Research and Evidence-based Interventions: Are SSDs Really the Ugly Stepchild?
In most discussions about high quality research, single participant designs have been relegated to a lower status. This paper reviews the characteristics of SSDs and the contributions they can make to the evidence-base.
Detrich, R. (2007). Single Subject Research and Evidence-based Interventions: Are SSDs Really the Ugly Stepchild? [Powerpoint Slides]. Retrieved from 2007-aba-presentation-ronnie-detrich.
IDEIA and Evidence-based Interventions: Implications for Practitioners
The reauthorization of special education law (IDEIA) emphasizes using scientifically supported programs. This talk reviews the implications for special education practitioners.
Detrich, R. (2008). IDEIA and Evidence-based Interventions: Implications for Practitioners [Powerpoint Slides]. Retrieved from 2008-apbs-txint-presentation-ronnie-detrich.
Evidence-based Practice: More than a List
Evidence-based practice has at least two meanings. This paper argues that it is best thought of as a decision-making framework.
Detrich, R. (2011). Evidence-based Practice: More than a List [Powerpoint Slides]. Retrieved from 2011-apbs-presentation-ronnie-detrich.
Workshop: Evidence-based Practice of Applied Behavior Analysis.
Evidence-based practice is a decision-making framework that integrates best available evidence, professional judgement, and client values and context. This workshop described the relationship across these three dimensions of decision-making.
Detrich, R. (2015). Workshop: Evidence-based Practice of Applied Behavior Analysis. [Powerpoint Slides]. Retrieved from 2015-missouriaba-workshop-presentation-ronnie-detrich.
TITLE
SYNOPSIS
CITATION
Creating Single-Subject Design in Microsoft Excel
The article provides a task analyses for constructing various types of commonly used single-subject design graphs in Microsoft Excel
Dixon, M. R., Jackson, J. W., Small, S. L., Horner?King, M. J., Lik, N. M. K., Garcia, Y., & Rosales, R. (2009). CCreating Single-Subject Design in Microsoft Excel™ 2007. Journal of applied behavior analysis, 42(2), 277-293.
Evaluation in clinical practice: Clinically sensitive and systematic methods of treatment delivery

 The present paper makes the case for systematic assessment and evaluation in clinical practice. The purpose of systematic evaluation is to enhance client care and to improve the basis for drawing inferences about treatment and therapeutic change.

Kazdin, A. E. (1993). Evaluation in clinical practice: Clinically sensitive and systematic methods of treatment delivery. Behavior Therapy24(1), 11-45.

Inequality and Economic Growth: The Perspective of the New Growth Theories

We analyze the relationship between inequality and economic growth from two directions. The first part of the survey examines the effect of inequality on growth. The second part analyzes several mechanisms whereby growth may increase wage inequality, both across and within education cohorts.

Aghion, P., Caroli, E., & Garcia-Penalosa, C. (1999). Inequality and economic growth: The perspective of the new growth theories. Journal of Economic literature37(4), 1615-1660.

Randomized Trials and Quasi-Experiments in Education Research

This paper examines the benefits and challenges inherent in using randomized clinical trials and quasi-experimental designs in the field of education research.

Angrist, J. D. (2003). Randomized trials and quasi-experiments in education research. NBER Reporter Online, (Summer 2003), 11-14.

Client satisfaction, clinical significance, and meaningful change in psychotherapy.

Evaluated the use of the N. S. Jacobson et al (see record 1985-00073-001) criteria for clinical significance in psychotherapy data analysis.

Ankuta, G. Y., & Abeles, N. (1993). Client satisfaction, clinical significance, and meaningful change in psychotherapy. Professional Psychology: Research and Practice, 24(1), 70-74.

Treatment alternatives for Attention-Deficit/ Hyperactivity Disorder (ADHD)

Objective: To review alternative treatments (Tx) of Attention-Deficit/Hyperactivity Disorder (ADHD)those other than psychoactive medication and behavioral/psychosocial Tx-for the November, 1998 National Institute of Health (NIH) Consensus Development Conference on ADHD.

Arnold, L. E. (1999). Treatment alternatives for attention-deficit! hyperactivity disorder (ADHD). Journal of attention disorders3(1), 30-48.

Synthesis of behavioral science learnings about technology transfer

This chapter reviews a set of behavioral science findings derived from the November 1993 NIDA Technical Review, “Reviewing the Behavioral Science Knowledge Base on Technology Transfer.” This is not intended to be a complete recapitulation of the arguments and conclusions drawn by the authors of the 14 papers presented in this monograph.

Backer, T. E., & David, S. L. (1995). Synthesis of behavioral science learnings about technology transfer. NIDA research monograph155, 262-279.

The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.

In this article, we attempt to distinguish between the properties of moderator and mediator variables at a number of levels.

Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of personality and social psychology51(6), 1173.

Establishing Specificity in Psychotherapy: A Meta-Analysis of Structural Equivalence of Placebo Controls

s. In this study, the authors used meta-analytic procedures to test one possible factor contributing to the attenuation of effects: structural inequalities between placebo and active treatments.

Baskin, T. W., Tierney, S. C., Minami, T., & Wampold, B. E. (2003). Establishing specificity in psychotherapy: a meta-analysis of structural equivalence of placebo controls. Journal of consulting and clinical psychology71(6), 973.

What Psychology Has to Offer Education—Now

The author discusses what he believes to be the promise of this approach, its influence on the role of the school psychologist, and what educators can do if they choose to pursue the leads offered by this group. 

Bijou, S. W. (1970). What psychology has to offer education—now. Journal of Applied Behavior Analysis3(1), 65.

Intervention-based school psychology services: Training for child-level accountability; preparing for program-level accountability

This research evaluated the outcomes of a school psychology training practicum by replicating intervention-based service delivery procedures established in prior research. 

Bonner, M., & Barnett, D. W. (2004). Intervention-based school psychology services: Training for child-level accountability; preparing for program-level accountability. Journal of School Psychology42(1), 23-43.

Generalizability and decision studies: An example using conversational language samples

Observational data collected in ecologically valid measurement contexts are likely to be influenced by contextual factors irrelevant to the research question. Using multiple sessions and raters often improves the stability of scores for variables from such contexts.

Bruckner, C. T., Yoder, P. J., & McWilliam, R. A. (2006). Generalizability and decision studies: An example using conversational language samples. Journal of Early Intervention28(2), 139-153.

Dramatic Cures for Juvenile Crime: An Evaluation of a Prisoner-Run Delinquency Prevention Program

This article describes an evaluation of a prisoner-run delinquency prevention program at Hawaii's major prison.

Buckner, J. C., & Chesney-Lind, M. (1983). Dramatic cures for juvenile crime: An evaluation of a prisoner-run delinquency prevention program. Criminal Justice and Behavior10(2), 227-247.

Effective Treatment for Mental Disorders in Children and Adolescents

As pressure increases for the demonstration of effective treatment for children with mental disorders, it is essential that the field has an understanding of the evidence base. To address this aim, the authors searched the published literature for effective interventions for children and adolescents and organized this review

Burns, B. J., Hoagwood, K., & Mrazek, P. J. (1999). Effective treatment for mental disorders in children and adolescents. Clinical child and family psychology review2(4), 199-254.

Implementation research: Finding common ground on what, how, why, where, and who

Over many decades, educators have developed countless interventions and theories about how to create lasting change. In other words, implementation research is an endeavor to understand if and how educational efforts are accomplishing their goals. 

Century, J., & Cassata, A. (2016). Implementation research: Finding common ground on what, how, why, where, and who. Review of Research in Education40(1), 169-215.

Treatments that work with children: Empirically supported strategies for managing childhood problems.

Eight comprehensive chapters cover the common problems of disruptive behavior, anxiety, sleep disorders, nocturnal enuresis, encopresis, habit disorders (such as tics and thumbsucking), the treatment of pain and, finally, helping children adhere to medical regimens. The book describes diagnosis and treatment, with an emphasis on practicality. 

Christophersen, E. R., & Mortweet, S. L. (2001). Treatments that work with children: Empirically supported strategies for managing childhood problems. Washington, DC, US: American Psychological Association.

Outcomes and Incomes: How to Evaluate, Improve, and Market Your Psychotherapy Practice by Measuring Outcomes

This guide presents the tools therapists need to incorporate outcomes measurement effectively and meaningfully into everyday clinical work.

Clement, P. W. (1999). Outcomes and incomes: How to evaluate, improve, and market your psychotherapy practice by measuring outcomes. Guilford Press.

How does cognitive therapy work? Cognitive change and symptom change in cognitive therapy and pharmacotherapy for depression.

The effects of changes in depression-relevant cognition were examined in relation to subsequent change in depressive symptoms for outpatients with major depressive disorder randomly assigned to cognitive therapy (COT; n = 32) vs those assigned to pharmacotherapy only (NoCT; n = 32). 

DeRubeis, R. J., Evans, M. D., Hollon, S. D., Garvey, M. J., Grove, W. M., & Tuason, V. B. (1990). How does cognitive therapy work? Cognitive change and symptom change in cognitive therapy and pharmacotherapy for depression. Journal of Consulting and Clinical Psychology, 58(6), 862-869.

When interventions harm: Peer groups and problem behavior.

This article explored developmental and intervention evidence relevant to iatrogenic effects in peer-group interventions. Longitudinal research revealed that "deviancy training" within adolescent friendships predicts increases in delinquency, substance use, violence, and adult maladjustment.

Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American psychologist54(9), 755.

Evidence-based mental health practice: A textbook.

This comprehensive textbook is an essential primer for all practitioners and students who are grappling with the new age of evidence-based practice. The contributors explore some of the complex challenges in implementing EBPs, and highlight the meaningful opportunities that are inherent in this paradigm shift.

Drake, R. E., Merrens, M. R., & Lynde, D. W. (Eds.). (2005). A Norton professional book. Evidence-based mental health practice: A textbook. New York, NY, US: W W Norton & Co.

Can Randomized Trials Answer the Question of What Works?

This article discusses the use of randomized controlled trials as required by the Department of Education in evaluating the effectiveness of educational practices.

EDUC, A. R. O. (2005). Can randomized trials answer the question of what works?.

Developing evidence-based practice: The role of case-based research.

The authors argue that important evidence about best practice comes from case-based research, which builds knowledge in a clinically useful manner and complements what is achieved by multivariate research methods. 

Edwards, D. J., Dattilio, F. M., & Bromley, D. B. (2004). Developing evidence-based practice: The role of case-based research. Professional Psychology: Research and Practice35(6), 589.

Cover-copy-compare and spelling: One versus three repetitions

Cover, copy, compare (CCC) has been used with success to improve spelling skills. This study adds to existing research by completing an analysis of the rewriting component of the intervention. The impact of varying the number of times a subject copied a word following an error was examined with four elementary age students.

Erion, J., Davenport, C., Rodax, N., Scholl, B., & Hardy, J. (2009). Cover-copy-compare and spelling: One versus three repetitions. Journal of Behavioral Education18(4), 319-330.

The St. Louis conundrum: The effective treatment of antisocial youths

This book analyzes the findings of a treatment program which integrated antisocial and delinquent youths into prosocial peer groups in a suburban community center in St. Louis.

Feldman, R. A., Caplinger, T. E., & Wodarski, J. S. (1983). The St. Louis conundrum: The effective treatment of antisocial youths. Englewood Cliffs, NJ: Prentice-Hall.

Creating new realities: Program development and dissemination

In this paper we will review some of the examples from industrial innovation and dissemination, provide some data on replications of the Achievement Place/Teaching-Family Model over 20 years, and try to share some of the philosophical, practical, and technological guidelines we have come to accept.

Fixsen, D. L., & Blase, K. A. (1993). Creating new realities: Program development and dissemination. Journal of Applied Behavior Analysis26(4), 597-615.

Implementation Science: Fidelity Predictions and Outcomes

Predictions are made in the form of if-then statements. If reliable predictions define science and testing predictions is the work of scientists, then implementation science is a science to the extent that 1) predictions are made and 2) those predictions are tested in practice.

Fixsen, D. L., Van Dyke, M., & Blase, K. A. (2019). Implementation science: Fidelity predictions and outcomes. Retrieved from Chapel Hill, NC: Active Implementation Research Network: www. activeimplementation. org/resources.

What works for whom?: a critical review of treatments for children and adolescents

The standard reference in the field, this acclaimed work synthesizes findings from hundreds of carefully selected studies of mental health treatments for children and adolescents.

Fonagy, P., Cottrell, D., Phillips, J., Bevington, D., Glaser, D., & Allison, E. (2014). What works for whom?: a critical review of treatments for children and adolescents. Guilford Publications.

Introduction to the special section on stepped care models in psychotherapy.

This article introduces a special section addressing these resource allocation issues in the context of prevalent disorders 

Haaga, D. A. F. (2000). Introduction to the special section on stepped care models in psychotherapy. Journal of Consulting and Clinical Psychology, 68(4), 547-548.

Will the “principles of effectiveness” improve prevention practice? Early findings from a diffusion study

This study examines adoption and implementation of the US Department of Education's new policy, the `Principles of Effectiveness', from a diffusion of innovations theoretical framework. In this report, we evaluate adoption in relation to Principle 3: the requirement to select research-based programs.

Hallfors, D., & Godette, D. (2002). Will the “principles of effectiveness” improve prevention practice? Early findings from a diffusion study. Health Education Research, 17(4), 461–470.

Visible learning: A synthesis of over 800 meta-analyses relating to achievement

Hattie’s book is designed as a meta-meta-study that collects, compares and analyses the findings of many previous studies in education. Hattie focuses on schools in the English-speaking world but most aspects of the underlying story should be transferable to other countries and school systems as well. Visible Learning is nothing less than a synthesis of more than 50.000 studies covering more than 80 million pupils. Hattie uses the statistical measure effect size to compare the impact of many influences on students’ achievement, e.g. class size, holidays, feedback, and learning strategies.

Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.

 

Visible learning: A synthesis of over 800 meta-analyses related to achievement.

This unique and ground-breaking book is the result of 15 years research and synthesises over 800 meta-analyses on the influences on achievement in school-aged students. It builds a story about the power of teachers, feedback, and a model of learning and understanding.

Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. routledge.

Implementing Randomized Field Trials in Education: Report of a Workshop

This book examines the use of randomized controlled trial (RCT) studies in education.

Hilton, M., & Towne, L. (Eds.). (2004). Implementing Randomized Field Trials in Education:: Report of a Workshop. National Academies Press.

Quasi-experimentation: Design & Analysis Issues for Field Settings

A statistics textbook appropriate for graduate students and researchers conducting quasi-experimental design and analysis.

Hyman, R. (1982). Quasi-experimentation: design and analysis issues for field settings (book). Journal of Personality Assessment46(1), 96-97.

Why Most Published Research Findings Are False

This essay discusses issues and concerns that too many research findings may be false. The paper examines reasons a study may prove inaccurate including: the study power and bias, the number of other studies on the same question, and the ratio of true to no relationships. Finally, it considers the implications these problems create for conducting and interpreting research.

Ioannidis, J. P. (2005). Why most published research findings are false. PLoS medicine, 2(8), e124.

Educational research: Quantitative, qualitative, & mixed approaches

A comprehensive, easily digestible introduction to research methods for undergraduate and graduate students. Readers will develop an understanding of the multiple research methods and strategies used in education and related fields, including how to read and critically evaluate published research and how to write a proposal, construct a questionnaire, and conduct an empirical research.

Johnson, R. B., & Christensen, L. (2019). Educational research: Quantitative, qualitative, and mixed approaches. Sage publications.

Hyperactivity and Diet Treatment: A Meta-Analysis of the Feingold Hypothesis

This paper is a review of primary research investigating the Feingold hypothesis which suggests diet modification as an efficacious treatment for hyperactivity. 

Kavale, K. A., & Forness, S. R. (1983). Hyperactivity and diet treatment: A meta-analysis of the Feingold hypothesis. Journal of Learning Disabilities16(6), 324-330.

History of Behavior Modification

This chapter traces the history of behavior modification as a general movement. Individual conceptual approaches and techniques that comprise behavior modification are obviously important in tracing the history, but they are examined as part of the larger development rather than as ends in their own right. 

Kazdin, A. E. (1982). History of behavior modification. In International handbook of behavior modification and therapy (pp. 3-32). Springer, Boston, MA.

The Meanings and Measurement of Clinical Significance

The previous articles in this special section make the case for the importance of evaluating the clinical significance of the therapeutic change, present key measures and innovative ways in which they are applied, and more generally provide important guidelines for evaluating therapeutic change. 

Kazdin, A. E. (1999). The meanings and measurement of clinical significance.

Almost Clinically Significant (p < .10): Current Measures May Only Approach Clinical Significance

The review by Sheldrick et al. evaluates treatments for children and adolescents with conduct disorder and whether they produce clinically significant change

Kazdin, A. E. (2001). Almost clinically significant (pClinical psychology: Science and practice, 8(4), 455-462.

Mechanisms of Change in Psychotherapy: Advances, Breakthroughs, and Cutting-Edge Research (Do Not Yet Exist)

The focus of this chapter is on psychotherapy research and a call for research on mechanisms of therapeutic change. 

Kazdin, A. E. (2006). Mechanisms of Change in Psychotherapy: Advances, Breakthroughs, and Cutting-Edge Research (Do Not Yet Exist).

Delineating mechanisms of change in child and adolescent therapy: methodological issues and research recommendations

In this article, we discuss the importance of studying mechanisms, the logical and methodological requirements, and why almost no studies to date provide evidence for why or how treatment works.

Kazdin, A. E., & Nock, M. K. (2003). Delineating mechanisms of change in child and adolescent therapy: Methodological issues and research recommendations. Journal of Child Psychology and Psychiatry44(8), 1116-1129.

How Do Risk Factors Work Together? Mediators, Moderators, and Independent, Overlapping, and Proxy Risk Factors

The authors developed a methodological basis for investigating how risk factors work together. Better methods are needed for understanding the etiology of disorders, such as psychiatric syndromes, that presumably are the result of complex causal chains.

Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. American journal of psychiatry158(6), 848-856.

Mediators and Moderators of Treatment Effects in Randomized Clinical Trials

This paper describes an analytic framework to identify and distinguish between moderators and mediators in RCTs when outcomes are measured dimensionally. 

Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of general psychiatry59(10), 877-883.

What Works Clearinghouse: Single-Case Design Technical Documentation

This paper by a What Works Clearinghouse the panel provides an overview of singlr-subject designs (SCDs), specifies the types of questions that SCDs are designed to answer, and discusses the internal validity of SCDs. The panel then proposes standards to be implemented by the WWC.

Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. What Works Clearinghouse.

Cognitive change processes in a group cognitive behavior therapy of depression

The present study attempted to examine the causal relationships among changes in automatic thoughts, dysfunctional attitudes, and depressive symptoms in a 12-week group cognitive behavior therapy (GCBT) program for depression.

Kwon, S. M., & Oei, T. P. (2003). Cognitive change processes in a group cognitive behavior therapy of depression. Journal of Behavior Therapy and Experimental Psychiatry34(1), 73-85.

Bergin and Garfield's Handbook of Psychotherapy and Behavior Change

This bestselling resource presents authoritative thinking on the pressing questions, issues, and controversies in psychotherapy research and practice today.

Lambert, M. J., Garfield, S. L., & Bergin, A. E. (2004). Handbook of psychotherapy and behavior change. New York: John Wiley & Sons.

Clinical Outcome, Consumer Satisfaction, and Ad Hoc Ratings of Improvement in Children's Mental Health

In the present correlational study of 199 treated adolescents, the authors used a multitrait-multimethod analysis to examine psychometrically measured pathology change (pre- and postassessment of symptoms and functioning), consumer satisfaction, and perceived improvement reported by multiple informants.

Lambert, W., Salzer, M. S., & Bickman, L. (1998). Clinical outcome, consumer satisfaction, and ad hoc ratings of improvement in children's mental health. Journal of Consulting and Clinical Psychology66(2), 270-278.

Educational/Psychological Intervention Research Circa 2012

The chapter focuses on the historically perceived poor methodological rigor and low scientific credibility of most educational/psychological intervention research.

Levin, J. R., & Kratochwill, T. R. (2012). Educational/psychological intervention research circa 2012. Handbook of Psychology, Second Edition7.

Facts are more important than novelty: Replication in the education sciences

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.

Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316.

A meta‐analysis of bibliotherapy studies

Used meta‐analysis to examine the efficacy of bibliotherapy. Bibliotherapy treatments were compared to control groups and therapist‐administered treatments. 

Marrs, R. W. (1995). A meta‐analysis of bibliotherapy studies. American journal of community psychology23(6), 843-870.

Allergic disorders and attention deficit disorder in children

In a large sample of children from the general population this research found no association between parent, teacher, and self-reports of ADDH behaviors and a history of allergic disorders (asthma, eczema, rhinitis, and urticaria) at ages 9 or 13 years.

McGee, R., Stanton, W. R., & Sears, M. R. (1993). Allergic disorders and attention deficit disorder in children. Journal of abnormal child psychology21(1), 79-88.

Measuring the fidelity of implementation of a mental health program model.

Developed a fidelity index of program implementation for assertive community treatment (ACT). In Study 1, 20 experts rated the importance of 73 elements proposed as critical ACT ingredients, also indicating ideal model specifications for elements.

McGrew, J. H., Bond, G. R., Dietzen, L., & Salyers, M. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62(4), 670-678.

Treatment integrity of school‐based interventions with children

This paper examines school-based experimental studies with individuals 0 to 18 years between 1991 and 2005.  Only 30% of the studies provided treatment integrity data. Nearly half of studies (45%) were judged to be at high risk for treatment inaccuracies.

McIntyre, L. L., Gresham, F. M., DiGennaro, F. D., & Reed, D. D. (2007). Treatment integrity of school‐based interventions with children in the Journal of Applied Behavior Analysis 1991–2005. Journal of Applied Behavior Analysis, 40(4), 659–672.

The magical number seven, plus or minus two: Some limits on our capacity for processing information.

A variety of researches are examined from the standpoint of information theory. It is shown that the unaided observer is severely limited in terms of the amount of information he can receive, process, and remember. However, it is shown that by the use of various techniques, e.g., use of several stimulus dimensions, recoding, and various mnemonic devices, this informational bottleneck can be broken.

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review63(2), 81.

Expanding the frontier of treatment research.

This article covers current efforts by the National Institute of Mental Health to bridge this gap. Included are discussions of problems with the current research portfolio and new efforts in expanding the research portfolio, innovative methodological research, and expansion of training programs. 

Norquist, G., Lebowitz, B., & Hyman, S. (1999). Expanding the frontier of treatment research. Prevention & Treatment, 2(1). Article ID 1a.

Relationship of satisfaction to symptom change, follow-up adjustment, and clinical significance.

The relationship of client satisfaction to outcome was investigated for adult outpatients (N  =  152) from 3 urban community mental health centers. Clients completed a problem self-rating and the Brief Symptom Inventory (BSI) at intake, 10 weeks later, and 5 months later.

Pekarik, G., & Wolff, C. B. (1996). Relationship of satisfaction to symptom change, follow-up adjustment, and clinical significance. Professional Psychology: Research and Practice, 27(2), 202-208.

Handbook of Psychology: Educational psychology

This award-winning twelve-volume reference covers every aspect of the ever-fascinating discipline of psychology and represents the most current knowledge in the field. This ten-year revision now covers discoveries based in neuroscience, clinical psychology's new interest in evidence-based practice and mindfulness, and new findings in social, developmental, and forensic psychology.

Pianta, R. C., Hamre, B., Stuhlman, M., Reynolds, W. M., & Miller, G. E. (2003). Handbook of psychology: Educational psychology.

Empirically supported comprehensive treatments for young children with autism.

Describes treatment of autism, a severe, chronic developmental disorder that results in significant lifelong disability for most persons, with few persons ever functioning in an independent and typical lifestyle.

Rogers, S. J. (1998). Empirically supported comprehensive treatments for young children with autism. Journal of clinical child psychology27(2), 168-179.

Psychology should list empirically supported principles of change (ESPs) and not credential trademarked therapies or other treatment packages

Current systems for listing empirically supported therapies (ESTs) provide recognition to treatment packages, many of them proprietary and trademarked, without regard to the principles of change believed to account for their effectiveness.

Rosen, G. M., & Davison, G. C. (2003). Psychology should list empirically supported principles of change (ESPs) and not credential trademarked therapies or other treatment packages. Behavior modification27(3), 300-312.

Mental health: A report of the Surgeon General--Executive summary.

Two messages are conveyed in the report: Mental health is fundamental to health, and mental disorders are real health conditions. The surgeon general's report summarizes the Office's detailed review of more than 3,000 research articles, plus 1st-person accounts from individuals who have been afflicted with mental disorders. 

Satcher, D. (2000). Mental health: A report of the Surgeon General--Executive summary. Professional Psychology: Research and Practice, 31(1), 5-13.

Replication: A Design Principle for Field Research

This article suggests the routine use of replications in field studies. 

Schafer, W. D. (2001). Replication: A design principle for field research. Practical Assessment, Research & Evaluation7(15), 1-7.

Multisystemic Therapy: Monitoring Treatment Fidelity

The challenges of specifying a complex and individualized treatment model and measuring fidelity thereto are described, using multisystemic therapy (MST) as an example.

Schoenwald, S. K., Henggeler, S. W., Brondino, M. J., & Rowland, M. D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process39(1), 83-103.

Home-Based Behavioral Treatment of Young Children with Autism

This study evaluated the impact of intensive behavioral treatment on the development of young autistic children.

Sheinkopf, S. J., & Siegel, B. (1998). Home-based behavioral treatment of young children with autism. Journal of autism and developmental disorders28(1), 15-23.

Roles and responsibilities of researchers and practitioners for translating research to practice

This paper outlines the best practices for researchers and practitioners translating research to practice as well as recommendations for improving the process.

Shriver, M. D. (2007). Roles and responsibilities of researchers and practitioners for translating research to practice. Journal of Evidence-Based Practices for Schools, 8(1), 1-30.

Teaching and learning in the interactive classroom.

For many centuries, the professor was the primary source of information, the font of knowledge. Books were nonexistent or scarce, as they still are today in developing countries of the world, and information was passed orally from teacher to pupil. The didactic lecture is an effective method for conveying information from one person to a larger number of students, but, as most of us have experienced, simply telling information to someone does not ensure that learning takes place.

Silverthorn, D. U. (2006). Teaching and learning in the interactive classroom. Advances in Physiology Education30(4), 135-140.

Horticulture as Therapy: Principles and Practice

Did you know that plants and plant products can be used to improve people’s cognitive, physical, psychological, and social functioning? Well, they can, and Horticulture as Therapy is the book to show you how! 

Simson, S., & Straus, M. (1997). Horticulture as therapy: Principles and practice. CRC Press.

The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation

A recent large-scale evaluation of Reading Recovery, a supplemental reading program for young struggling readers, supports previous research that found it to be effective.  In a 4 year, federally funded project, almost 3,500 students in 685 schools found that generally students benefitted from the intervention. Students receiving Reading Recovery receive supplemental services in a 1:1 instructional setting for 30 minutes 5 days a week from an instructor trained in Reading Recovery.  In the study reported here, students who received Reading Recovery had effect sizes of .35-.37 relative to a control group across a number of measures of reading.  These represent moderate effect sizes and account for about a 1.5 month increase in skill relative to the control group.  Even though the research supports the efficacy of the intervention, it also raises questions about its efficiency.  The schools that participated in the study served about 5 students and the estimated cost per student has ranged from $2,000-$5,000.  These data raise questions about the wisdom of spending this much money per student for growth of about a month and a half.

Sirinides, P., Gray, A., & May, H. (2018). The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation. Educational Evaluation and Policy Analysis, 0162373718764828.

Outcome of Early Intervention for Children With Autism

Since 1980, 12 peer-reviewed outcome studies (nine on behavior analytic programs, one on Project TEACCH, and two on Colorado Health Sciences) have focused on early intervention for children with autism. Mean 10 gains of 7-28 points were reported in studies of behavior analytic programs, and 3-9 in studies on TEACCH and Colorado. 

Smith, T. (1999). Outcome of early intervention for children with autism. Clinical Psychology: Science and Practice6(1), 33-49.

The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half-Century of Research

A soon to be published meta-analysis of Direct Instruction (DI) curricula that reviews research on DI curricula between 1966-2016 reports that DI curricula produced moderate to large effect sizes across the curriculum areas reading, math, language, and spelling.  The review is notable because it reviews a much larger body of DI research than has occurred in the past and covers a wide range of experimental designs (from single subject to randomized trials).  328 studies were reviewed and almost 4,000 effects were considered.  Given the variability in research designs and the breadth of the effects considered, it suggests that DI curricula produce robust results.  There was very little decline during maintenance phases of the study and greater exposure to the curricula resulted in greater effects.

Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C. (2018). The effectiveness of direct instruction curricula: A meta-analysis of a half century of research. Review of Educational Research88(4), 479-507.

School-wide positive behavior support: Establishing a continuum of evidence-based practices

The purpose of this article is to describe how effective practices are incorporated into an approach termed schoolwide positive behavior supports (SWPBS)

Sugai, G., & Horner, R. H. (2010). School-wide positive behavior support: Establishing a continuum of evidence based practices. Journal of Evidence-Based Practices for Schools11(1), 62-83.

Training in and Dissemination of Empirically-Validated Psychological Treatments: Report and Recommendations

At the request of David Barlow, President of Division 12, and under the aegis of Section III, this task force was constituted to consider methods for educating clinical psychologists, third party payors, and the public about effective psychotherapies

Task Force on Promotion and Dissemination of Psychological Procedures, Division of Clinical Psychology, American Psychological Association. (1995). Training in and Dissemination of Empirically-Validated Psychological Treatments: Report and Recommendations. The Clinical Psychologist, 48, 3-23. 

Identifying research-based practices for response to intervention: Scientifically-based instruction

This paper examines the types of research to consider when evaluating programs, how to know what “evidence’ to use, and continuums of evidence (quantity of the evidence, quality of the evidence, and program development).

Twyman, J. S., & Sota, M. (2008). Identifying research-based practices for response to intervention: Scientifically based instruction. Journal of Evidence-Based Practices for Schools, 9(2), 86-101.

Linking implementation fidelity to impacts in an RCT

Simple experimental studies randomize study participants into two groups: a treatment group that includes participants who receive the offer to participate in a program or intervention, and a control group that includes participants who do not receive that offer. Such studies primarily address questions about the program impacts on the average outcomes of participants.

Unlu, F., Bozzi, L., Layzer, C., Smith, A., Price, C., & Hurtig, R. (2016). Linking implementation fidelity to impacts in an RCT (pp. 108-137). Routledge.

Twenty-Sixth Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act

The text and graphics contained in the 26th Annual Report to Congress were developed primarily from data from the Office of Special Education Programs (OSEP) Data Analysis System (DANS). DANS is a repository for all the data mandated by the Individuals with Disabilities Education Act (IDEA) to be collected from states annually. 

US Department of Education. (1998). Twentieth annual report to Congress on the implementation of the Individuals with Disabilities Education Act.

Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication

The unrealistically high rate of positive results within psychology has increased the attention to replication research. However, researchers who conduct a replication and want to statistically combine the results of their replication with a statistically significant original study encounter problems when using traditional meta-analysis techniques. The original study’s effect size is most probably overestimated because it is statistically significant, and this bias is not taken into consideration in traditional meta-analysis. We have developed a hybrid method that does take the statistical significance of an original study into account and enables (a) accurate effect size estimation, (b) estimation of a confidence interval, and (c) testing of the null hypothesis of no effect. We analytically approximate the performance of the hybrid method and describe its statistical properties. By applying the hybrid method to data from the Reproducibility Project: Psychology (Open Science Collaboration, 2015), we demonstrate that the conclusions based on the hybrid method are often in line with those of the replication, suggesting that many published psychological studies have smaller effect sizes than those reported in the original study, and that some effects may even be absent. We offer hands-on guidelines for how to statistically combine an original study and replication, and have developed a Web-based application (https://rvanaert.shinyapps.io/hybrid) for applying the hybrid method.

van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication. Behavior Research Methods, 50(4),1515–1539.

The Innovation Journey

The Innovation Journey presents the results of a major longitudinal study that examined the process of innovation from concept to implementation of new technologies, products, processes, and administrative arrangements.

Van de Ven, A. H., Polley, D. E., Garud, R., & Venkataraman, S. (1999). The Innovation Journey, New York: Oxford Univ.

Finding, evaluating, refining, and applying empirically supported treatments for children and adolescents

The Child Task Force report represents an important initial step in this direction. Here they offer both praise and critique, suggesting a number of ways the task force process and product may be improved. 

Weisz, J. R., & Hawley, K. M. (1998). Finding, evaluating, refining, and applying empirically supported treatments for children and adolescents. Journal of Clinical Child Psychology27(2), 206-216.

Bridging the Gap Between Laboratory and Clinic in Child and Adolescent Psychotherapy

This article addresses the gap between clinical practice and the research laboratory. We focus on the issue as it relates specifically to interventions for children and adolescents.

Weisz, J. R., Donenberg, G. R., Han, S. S., & Weiss, B. (1995). Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. Journal of consulting and clinical psychology63(5), 688.

More of what? Issues raised by the Fort Bragg study.

The study does suggest that "more is not always better" (L. Bickman, 1996), but more of what? Little is known about the specific interventions that were combined to form the Fort Bragg system of care, so the study does not really reveal what failed or what needs to be changed. 

Weisz, J. R., Han, S. S., & Valeri, S. M. (1997). More of what? Issues raised by the Fort Bragg study.

Stressing the (other) three Rs in the search for empirically supported treatments: Review procedures, research quality, relevance to practice and the public interest

The Society of Clinical Psychology's task forces on psychological intervention developed criteria for evaluating clinical trials, applied those criteria, and generated lists of empirically supported treatments. Building on this strong base, the task force successor, the Committee on Science and Practice, now pursues a three‐part agenda

Weisz, J. R., Hawley, K. M., Pilkonis, P. A., Woody, S. R., & Follette, W. C. (2000). Stressing the (other) three Rs in the search for empirically supported treatments: Review procedures, research quality, relevance to practice and the public interest. Clinical Psychology: Science and Practice7(3), 243-258.

Evidence-Based Education (EBE)

This slide show presents what is EBE and what are EBE goals in education. 

Whitehurst, G. J. (2002). Evidence-based education (EBE). Washington, DC. Retrieved Juanuary9(2), 6.

The Core Analytics of Randomized Experiments for Social Research
This paper examines the elements of randomized experiments for social research.
Bloom, H. S. (2006). The core analytics of randomized experiments for social research.
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
A study comparing the efficacy of randomized controlled trials to observational studies.
Concato, J., Shah, N., & Horwitz, R. I. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342(25), 1887-1892.
New Federal Policy Favors Randomized Trials in Education Research
Thisis an article from the Chronicle of Higher Education discussing the pros and cons of randomized controlled trials in education
Glenn, D. A. V. I. D. (2005). New federal policy favors randomized trials in education research. The Chronicle of Higher Education, Retrieved March, 25, 2005.
Single-Case Designs for Educational Research
This paper examines the benefits and challenges inherent in using of randomized clinical trials and quasi-experimental designs in the field of education research.
Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings . Oxford University Press.
Single-Case Designs for Educational Research
This book provides a thorough summary of information about the use of single-subject experimental designs in educational research
Kennedy, C. H. (2005). Single-case designs for educational research. Pearson/A & B.
Evaluating Teacher Preparation Programs Using the Performance of their Graduates
This commentary addresses concerns for the use of value-added outcome measures commonly used to evaluate teachers and implications for the use of these metrics to assess the effectiveness of preparation programs.
Koedel, C. and Parsons, E., (2014). Evaluating teacher preparation programs using the performance of their graduates. Teachers College Record. Retrieved November 18, 2014 from http://www.tcrecord.org/Content.asp?ContentID=17741
Single-case research design and analysis: New directions for psychology and education.
This book provides a thorough summary of information about the use of single-subject experimental designs.
Kratochwill, T. R., & Levin, J. R. (1992). Single-case research design and analysis: New directions for psychology and education. Lawrence Erlbaum Associates, Inc.
Single-Case Design Technical Documentation
Single case design has made important contributions to identifying effective educational practices. Until recently, there have been no standards for evaluating the quality and quality of studies across a topic area. These standards were developed by the Institute for Education Science.
Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M. & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
Comparing Results of Systematic Reviews: Parallel Reviews of Research on Repeated Reading.
This paper demonstrates that different well accepted methods for reviewing research on repeated readings produces different results.
O’Keeffe, B. V., Slocum, T. A., Burlingame, C., Snyder, K., & Bundock, K. (2012). Comparing Results of Systematic Reviews: Parallel Reviews of Research on Repeated Reading. Education & Treatment of Children (West Virginia University Press), 35(2), 333-366
Combining estimates of effect size
This book is an in depth examination of literature synthesis along with useful advice when one attempts to interpret the results of a meta-analysis.
Shadish, W. R., & Haddock, C. K. (2009). Combining estimates of effect size. The Handbook of Research Synthesis and Meta-analysis, 257-277.
The state of the science in the meta-analysis of single-case experimental designs
This is a review of the issues and methods for conducting a meta-analysis of single-case design research studies.
Shadish, W. R., Rindskopf, D. M. & Hedges, L. V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 2(3), 188-196. doi:10.1080/17489530802581603
Evaluating the validity of systematic reviews to identify empirically supported treatments
Systematic reviews are a process for assessing the quality of the literature to determine if a particular practice has met criteria for empirically supported. As with any assessment process there are issues of validity. The concepts and methodological tools of measurement validity can be applied to systematic reviews to identify their strengths and weaknesses.
Slocum, T. A., Detrich, R., & Spencer, T. D. (2012). Evaluating the validity of systematic reviews to identify empirically supported treatments. Education and Treatment of Children, 35(2), 201-233.
TITLE
SYNOPSIS
Journal of Contemporary Clinical Trials
Contemporary Clinical Trials is an international journal that publishes manuscripts pertaining to the design, methods and operational aspects of clinical trials.
Logical Positivism
An overview of Logical Positivism and it’s impact on science and the issue of verifiability.
Spurious Correlations
An important rule of research is; correlation does not equal causation. Just because two events track each other over time does not mean that one caused the other. This web site mines data and uses to humor to make the point that for such correlations are often “Spurious Correlations”.
What Works Clearinghouse (WWC)

The goal of the WWC is a resource for informed education decision-making. The WWC identifies evidence-based practice, program, or policy, and disseminates summary information on the WWC website.

Back to Top