Wednesday, May 12, 2010

New Book Lays Groundwork for Better Assessment of Charter School Performance

Seattle, WA - Despite many assertions to the contrary, we are far from having unambiguous evidence on charter school performance. Study results have been subject to withering criticism, and for good reason. Too many researchers use blunt methods that fail to make appropriate 'apples to apples' comparisons of student test scores. Others use sophisticated methods in a small number of schools, but then overgeneralize the results.

A new book from the Center on Reinventing Public Education (CRPE), Taking Measure of Charter Schools: Better Assessments, Better Policymaking, Better Schools, is designed to help people understand what the current state of research can honestly tell us about charter school quality; to offer ways to improve charter school research; and to suggest ways to better link charter school policy with evidence. The book is based on the work of CRPE's National Charter School Achievement Consensus Panel.

Readers will learn that:

  • The preponderance of evidence on achievement suggests that charter schools are outperforming traditional public schools. However, rigorous studies that take into account a student's past academic progress are still limited to only a few states. Charter schools may also increase the probability of graduating from high school and the probability of attending college, while reducing the number of student disciplinary actions. But the results are not uniform; some charter schools are underperforming, especially in certain locations and in particular subject areas and grade levels.
  • The way charter school research is conducted truly matters. Different methods, even using the same data, can produce very different results - positive findings about charter performance in some cases, and negative findings in others. One chapter uses data from San Diego and shows that the weaker evaluation methods tend to indicate that charter schools are underperforming traditional public schools, but the stronger evaluation methods that take into account a student's academic history suggest that charter schools in San Diego are outperforming.
  • Evaluation of charter school performance need not be limited to test scores. Other measures can include students' course completion, graduation, admission and success at higher levels of education, ability to find productive work, and capacity to act as effective citizens.
  • Lottery-based studies have enormous potential to measure the value charter schools are adding to student achievement, but there are critical challenges to generalizing results to other students, schools, and contexts.
  • New charter schools likely have growing pains that depress early performance. Studies that ignore school maturation will likely depress achievement results.
  • Teacher factors, such as differences in job satisfaction and stability of employment, might prove important in explaining variations in charter school outcomes.
  • The weight of charter research can accumulate over time, especially when the quality of studies increases and their results converge on key points. Researchers should emphasize the need to frame the consequences of research modestly and avoid oversimplification of findings in search of headlines.

Editors Julian Betts and Paul Hill argue that new federal initiatives to intervene in failing schools will mean that all public schools will be assessed on the same student performance standards that now apply almost exclusively to charters - all the more reason to get these performance measurement issues right. Data and methods that once looked acceptable now prove inadequate; the wrong data, or the right data used wrongly, can lead to unwarranted conclusions about school success and failure.

About the National Charter School Achievement Consensus Panel

Improving the quality of research on charter school outcomes was the impetus for the Charter School Achievement Consensus Panel. In early 2005, CRPE's National Charter School Research Project convened nine outstanding researchers from different methodological traditions to identify the relevant facts that must be considered in charter school studies, show how better studies can be designed, influence the type of studies that receive funding, and educate the public on the complexities of charter school research and interpretation of results.

The Panel's earlier publications include the influential white paper, "Key Issues in Studying Charter Schools and Achievement: A Review and Suggestions for National Guidelines (2006)", and Making Sense of Charter School Studies: A Reporter's Guide (Rainey, 2007). Like this new book, the reporter's guide is targeted at an educated lay audience, with the aim of raising the standards of reporting and legislative use of evidence on charter school outcomes.

Taking Measure of Charter Schools: Better Assessments, Better Policymaking, Better Schools, edited by Julian Betts and Paul Hill, is available from Rowman & Littlefield Education. Contributors include Consensus Panel members Dominic Brewer, Laura Hamilton, Jeffrey Henig, Robin Lake, and Patrick McEwan, as well as June Ahn, Larry Angel, Robert Olsen, Lydia Rainey, Brian Stecher, Y. Emily Tang, and Andrew Zau.