Flawed Research by National Center for Education Statistics Should be Viewed with Great Skepticism
An education research study scheduled for release tomorrow relies on bad data about students attending charter schools, according to the Center for Education Reform (CER). Studies using raw data collected in 2004 by the National Center for Education Statistics (NCES) include seriously flawed poverty indicators among charter school students, a problem NCES acknowledges. The problem is compounded when researchers apply various assumptions about poverty to assess and compare academic achievement between charter school students and those attending conventional public schools.
“This problem is at the heart of the ongoing research wars,” said CER President Jeanne Allen. “The education establishment – teachers, unions, school boards associations, and more – tout these flawed studies in an attempt to discredit new school opportunities for parents. My message to any parent, especially those with children attending charter schools, is to look at state-level assessments and get a real picture of student achievement. It’s there that charter school students shine like the true stars they are.” Allen cited the following examples:
- In 2005, a higher percentage of charter school students in Massachusetts, compared with students in conventional public schools, scored proficient or advanced in the state’s assessment test – 9.2 percent, 8.7 percent, and 8.3 percent higher in English, math, and science, respectively.
- In a comparison of African-American students, those attending charter schools outperformed the students in conventional public schools in Michigan’s 2004 assessment test – 46 percent of eighth grade African-American charter students passed the math assessment compared with 21 percent among African-American eighth graders statewide.
- In 2005, charter schools in California showed stronger year-to-year improvement than conventional public schools, especially in Los Angeles. Statewide, charter schools scored an average gain of 28 points in the state assessment compared with a 20-point gain posted by conventional public schools.
- In the 2004-05 school year, fourth grade charter school students in New York were 7.1 percent more proficient on the state’s English test and 7.7 percent more proficient on the math test than conventional public school students.
- The percentage of charter school students in Florida who tested proficient in the state’s reading assessment has grown faster than the gains posted by conventional public school students – charter school students rose from 55 percent to 58 percent, compared with an increase of 54 percent to 56 percent among conventional public school students.
NCES data rely on surveys of schools drawn from a nationally representative sample self-reporting the number of students who participate in the free and reduced lunch program as a proxy for poverty. When CER raised concerns in 2004 with NCES about its poverty data, that government data center acknowledged, “free and reduced lunch is a rather poor proxy for poverty, but it is all that is available.” Using this definition of poverty, NCES finds slightly lower poverty rates among charter school students (42 percent), compared with students from conventional public schools (44 percent).
The problem is that many charter schools, for a number of reasons, either do not participate in the school lunch program or do not expend limited resources monitoring the number of participants.
Since 1997, CER has regularly surveyed charter schools and receives unprecedented, first-hand information that is not available from any other source. The 2005 survey found that 63 percent of the students in a typical charter school qualify for the free and reduced lunch program. New data from charter schools revealed that nearly half of all respondents to a CER survey qualify for the federal lunch program but choose not to participate for a variety of reasons.
Analysis of the NCES data will also reportedly claim that school district-based charter schools are more successful than independent charter schools that are authorized by public entities other than school boards. This is based on flawed survey questions that do not accurately address the variations from state to state in authorizing and operational independence, regardless of sponsors.
“This latest interpretation of government data adds nothing to the question of how well students perform in charters compared to other schools,” said CER’s Allen. “As more than a million children go off to charter schools over the next month, how they learn will best be answered by local and state measures that have more depth and validity, and not by sloppy analyses of complex statistics.”
Fast Facts:
Charter School Achievement in the States
Charter Schools Free Lunch Facts
In The News:
Study of Test Scores Finds Charter Schools Lagging, by Diana Jean Schemo, New York Times, August 23, 2006
Charter Schools Fail to Top Their Public Peers, by Greg Toppo, USA Today, August 23, 2006
Charter Schools Face New Setback Amid Test Review, By Robert Tomsho, Wall Street Journal, August 23, 2006
Charter Schools Lag, Study Finds; Modest Difference in Test Scores Unlikely to Alter Debate, By Jay Mathews, Washington Post, August 23, 2006
Charter School Scores Down, Study Shows, by Stephen Ohlemacher, Associated Press, August 23, 2006
Do Charter Schools Make the Grade?, by Elaine Korry, Morning Edition, National Public Radio News, August 23, 2006
Charter School 4th-Graders Lag Others, U.S. Says, Los Angeles Daily News, August 23, 2006
Smearing Schools That Work , Op-Ed by Peter Murphy, New York Post, August 24, 2006
Charter School Nonsense, Wall Street Journal Editorial, August 28, 2006
Exploding the Charter School Myth, New York Times Editorial, August 27, 2006
Don't Confuse Learning With Research Skirmish, By Jeanne Allen, Letter to the Editor, Albany Times Union, New York, August 26, 2006
Roadblocks Won't Deter Charter School, by Jim Wooten, Atlanta Journal Constitution, Georgia, August 28, 2006
Only A Snapshot On Charters, The Tennessean Editorial, Tennessee, August 27, 2006
The Charter Challenge, Fort Wayne Journal Gazette Editorial, Indiana, August 29, 2006
Good News In Public Education, Plattsburgh Press Republican Editorial, New York, August 28, 2006
Charter Schools, by Jeanne Allen, Letter to the Editor, Kansas City Star, Missouri, August 29, 2006
# # #
The Center for Education Reform (CER) creates opportunities for and challenges obstacles to better education for America’s communities. Founded in 1993, CER combines education policy with grassroots advocacy to foster positive and bold education reforms. For more information, contact CER at (202) 822-9000 or send us email.
