Summit 2009
Fourth Annual Summit on Evidence-based Education
Data-Based Decision Making: The Achilles Heel of Evidence-based Education
April 23, 2009
| 8:30 | Introduction to The Wing Institute and Summit Outcomes |
| View presentation | |
| Randy Keyworth, The Wing Institute | |
|
Keyworth shared information about the Wing Institute and demographics of the Summit participants. He introduced the Summit topic in the context of the new President's initiatives and philosophy, and reviewed the results of the participant surveys on the current status of data-base decision making in schools. |
|
| 8:45 | In God We Trust; All Others Must Bring Data |
| View presentation | |
| Jack States, The Wing Institute | |
| States discussed the importance of systematizing data-based decision making. He identified the outcomes, including: better accountability, feedback on quality improvement efforts, and increased efficiency. He examined the different types of data such as outcome data, process data, and satisfaction data. He then reviewed critical components of a key indicator report, including strategies for implementation. | |
| 9:00 | The Four Assumptions of the Apocalypse |
| View presentation | |
| Ronnie Detrich, The Wing Institute | |
| Detrich examined four basic assumptions for effective data-based decision making: (1) reliable and valid data are available, (2) decision-makers have skills and supports to base decisions on data, (3) decision makers know what to do when data indicate change is necessary, and (4) staff can implement interventions with integrity. He then reviewed data that suggested each of these assumptions is shaky, and finished with a discussion of strategies for addressing problem areas. | |
| 9:30 | Data-Based Decision Making for Students Social Behavioral Difficulties |
| View presentation | |
| Frank Gresham, Ph.D, Louisiana State University | |
| Dr. Gresham discussed methods for making valid data-based decisions for student social behavior. Data-based decision making using curriculum-based measurement (CBM) strategies have enjoyed 25 years of research and are now considered to be the "gold standard" progress monitoring tool for academic behavior. Unfortunately, there is no CBM analogue making intervention decisions for students' social behavior. Conceptual, practical, and technical issues for data-based decision making were described and recommendations for future research and practice were discussed. | |
| 10:35 | Work Group: Data Analysis |
|
Using guidelines presented by Dr, Gresham, each work group was given eleven different student performance graphs. Each group answered questions about the trends in the graphs, and discussed issues of process and decision making. |
|
| 11:30 | Systems-Level Data-Based Decision Making for Evidence-Based Practice |
| View presentation | |
| W. David Tilly III, Ph.D, Heartland Area Education Agency 11 | |
| Dr. Tilley examined systems-level data-based decision making in the context of implementation of Response to Intervention practices, providing a framework for thinking about what decisions need to be made, which data to collect, and how to use these data for multiple decisions. Triumphs and mistakes from the school of hard knocks were presented, mourned and celebrated. He presented a likely scenario for the next steps in the evolution of systems-level applications of data-based decision making with evidence-based practices. | |
| 12:20 | Lunch |
| 1:05 | Work Group: Building a Dashboard |
| Each group was given the task to identify key indicators for monitoring the performance of a school system, including measurements for student, staff, and organizational outcomes. | |
| 2:00 | Working with Staff to Promote Data-Based Decision Making: Recommended Strategies and Common Pitfalls" |
| View presentation | |
| Dennis Reid, Ph.D, Carolina Behavior Analysis and Support Center | |
| Dr. Reid discussed evidence-based ways of working with staff to promote program intervention integrity and accurate data collection, including: what is known about effective staff training in these two areas, why staff training should not be considered sufficient for ensuring program integrity, and ongoing procedures that research has identified for promoting integrity of intervention and data collection. These will include use of informative yet practical monitoring and feedback systems. Directions for future research and application in this area will likewise be proposed, such as those relating to recently identified advantages and disadvantages of computer- or video-based staff training and monitoring. | |
| 3:00 | Speaker Panel |
| Frank Gresham, Ph.D, Louisiana State University | |
| W. David Tilly III, Ph.D, Heartland Area Education Agency 11 | |
| Dennis Reid, Ph.D, Carolina Behavior Analysis and Support Center | |
| 3:45 | Wrap-Up |