An average Michigan charter school student will learn more in a year than his or her public school peer, according to a new report by Stanford University’s Center for Research on Education Outcomes.

The study found that students from Michigan charter schools learn an average of two month’s more of math and reading per academic year.

Twenty-seven percent of the state’s charter school students are from Detroit, and Detroit charter school students gained up to three months’ worth of additional education, it said.

Charter schools are publicly funded but can be privately run. They were established in part so that individual schools could have more independence over curriculum and teaching staff.

Margaret Raymond, director of the center, praised Michigan’s charter school practices, especially given problems that districts like Detroit face.

“These findings show that Michigan has set policies for charter schools to produce consistent high quality across the state,” Raymond said. “The results are especially welcome for students in communities that face significant education challenges.”

It is the center’s first in-depth study of charter schools in the state. A total of 85,650 students attend 276 charters in the state. For the study, 61 schools were too small to be analyzed, resulting in a total study sample of 212 charters.

Not all of the findings were favorable to the alternative public schools, however.

For example, 14 percent of Michigan charter schools showed below average growth and achievement, and 25 percent of students perform below average in math.

Devora Davis, a co-author of the report, attributed those conflicting numbers to the use of averages — there are both struggling charters and high-performing charters that distort the data.

The poor performances are offset by the growing proportion of charters with high-level achievement, Davis said.

“Should these trends continue, the share of schools which currently lag the state averages would be expected to decline,” Davis said. “These absolute improvements are within sight in Michigan.”

Stanford’s earlier national study in 2009 was heavily criticized by the Center for Education Reform — based in Washington, D.C. — for its use of inaccurate state data.

According its president, Jeanne Allen, the new study done in Michigan and a similar one done in New Jersey use an improved methodology.

“In these state-level studies, it appears that the inclusion of a wider range of students and more school-level data were used to identify and compare individuals to their ‘traditional public school’ counterparts,” Allen said.

Doing so provided a more realistic view of students, and therefore, more credible results, she said.

Other experts are still critical of the study.

For instance, Amber Arellano, executive director of the Royal Oak-based Education Trust-Midwest, said that the use of averages in the study actually hides more accurate results, and she called for more government accountability for charter quality.

“The study’s focus on average charter student learning gains masks some great disparities in Michigan charter performance,” Arellano said. “Some charter schools are doing well and should be recognized for that.

Other criticisms included the study’s failure to account for the more than 30 new charter schools that opened this fall, or the more than 20 percent of previously established charters whose schools were too small for CREDO’s study standards, as well as the fact that most charter high schools weren’t studied at all.

Michael Van Beek, director of education policy for the Mackinac Center for Public Policy in Midland, said that it was important to remember that many students attending charter schools are among the poorest in the nation.

“Based on the well-established relationship between test scores and student poverty, one should expect most Michigan’s charter public schools to score below the state average since they serve a higher portion of poor students,” Van Beek said.

“The Stanford study says 70 percent of charter public school students qualify for a free or reduced-price lunch compared to 43 percent in conventional public schools,” he said.

In a follow-up to their initial 2009 report on the Bay State, CREDO has released its latest Charter School Performance Report on Massachusetts, a six year study that analyzes the effectiveness of Massachusetts’s charter schools and in particular, their performance in the Boston area.

The report was largely positive on both math and reading tests, notably when comparing Boston charter schools to their public school counterparts. When analyzing just Beantown charters, the report found that 83 percent had significantly positive learning gains in both reading and math and no city charters were performing lower than the local public schools. That’s a pretty good sales pitch for charter schools in Massachusetts.

For a little more background on CER’s long history with CREDO and our concerns with their methodology, which they use in this report, here’s a link to help you out: a little intro to CREDO.

The latest CREDO report looks at New York City charter school achievement and finds generally positive results.

CREDO research on other cities and states, like the Michigan report released in January, has generally yielded positive results. But perhaps more interesting is the fact that NYC CREDO findings are in line with work done by other researchers studying New York City charter schools. Check out the studies below for more on charter schools in the Big Apple:

• How NYC Charter Schools Affect Achievement:
This study done by Caroline M. Hoxby employs quality charter school research methodology and finds that NYC charter school students will learn more over time than those students who remain in conventional public schools.

• The State of the NYC Charter School Sector:
This report from the New York City Charter School Center gives an in-depth look into the city’s charter schools data, demographics and achievement, and indicates that charter schools continue to be a viable alternative for parents looking to better their children’s education in the Big Apple.

And for those of you scratching your head as to why you’ve heard the acronym CREDO before, it may have been because of a controversial and widely cited national report from 2009. Here’s some background to help you out: All About CREDO.

“It is hard to believe that year-after-year, smart, well-intentioned researchers believe they can make national conclusions about charter school performance using uneven data, flawed definitions of poverty and ignoring variations in state charter school laws,” said Jeanne Allen president of The Center for Education Reform (CER).

Yes, CREDO is at it again, using the same virtual twin methods that came under fire in their 2009 report.

In a Wall Street Journal Opinion video, David Feith breaks down — in less than 3 minutes — problems with the numbers in the latest CREDO report, as well as problems with how the report is being interpreted.

Uncorrelated Variables

CREDO’s analysis does not account for the great variances in charter laws from state to state or how those laws may differ from paper to practice.

• While the report suggests correlations exist between student achievement and charter law components, they admit to not fully understanding the impact of specific laws.
• The report suggests a negative correlation between student achievement and multiple authorizers. In fact, such charter authorizers vary greatly in law and practice, as CER’s 2012 study and scorecard demonstrate. There is clear evidence that charter students succeed in states witha number of meaningful, independent and highly accountable authorizers who compete for chartering. See our Multiple Authorizers Primer for more information on charter authorizers.
• The study was based on student population and not the overal strength of their charter system. Therefore CREDO missed most opportunities to see really strong charters in action.

Virtual Methodology

While there are virtual schools, there is no such thing as “virtual” student achievement.

• The CREDO report acknowledges the creation of new research tools to assess the unknowable. Instead of comparing real students who attend charter schools to real students who attend conventional public schools, CREDO merged demographic data to create “virtual twins.” Randomization, the gold standard of research, is not used.
• By virtually replicating the demographic profile of a charter school student — ethnicity, age, socio-economic status — and averaging that same profile in a conventional public school students, researchers ignore something every parent and teacher knows is an impossible comparison since no two students are alike. The study essentially treats group averages as if they were students themselves. (Respected researcher Caroline Hoxby explains this point in greater detail in Fact-Checking Charter School Achievement)
• The virtual twins were not created with state test data. Instead, the study uses National Assessment of Educational Progress (NAEP) scores, which are meant to provide snapshots of state performance and should not be used to gauge individual school performance. Meaningful comparisons cannot be made since relevant achievement data aren’t used.

Contradictory Demographics

The reports’ findings are inconsistent across demographic populations and geographic jurisdictions.

• While CREDO found that charter performance is lower for African-Americans and Hispanics, they find effects of charters greater for students in poverty. Well-documented and recognized survey work which represents more than a third of all charters, finds such demographics to be one and the same.
• The CREDO results are an anomaly in finding some states falling short on student achievement. Yet these same states – Florida, Minnesota, California, DC, and Georgia – have demonstrated enormous success on state and national assessments and have undergone intense scrutiny only to report superior student performance year after year.

The demonstrated success of, demand for, and attendance in charter schools escape broad stroke conclusions. Without student-by-student, school-by-school, and critical, specific data, we remain publicly ignorant of what makes schools work for children.

Once again we have good reporters getting snookered by “research” based on un-comparable data and lacking any value-added measurement of performance progress over time.

By any reasonable standard, reputable research needs to be based on an apples to apples comparison of subjects. The NEPC methodology makes no effort to compensate for the fact that the basic nature of virtual schools like K12 makes it difficult to compare their students to those in traditional public schools. Consequently, it ends up comparing apples to watermelons.

The NEPC report also cites a 2009 CREDO study that is one of the most egregious examples of bad research out there. CER has successfully debunked it time after time and yet the media continues to trot out that Trojan horse for some reason.

Where does madness end? When is the media going to learn to recognize good research from bad?

–Jeanne Allen, Founder and President of the Center for Education Reform

For K12 Inc.’s perspective, check out the Spotlight section on their website.

A small research unit at Stanford (not the university itself) piloted a methodology pairing virtual twins in charters with students in traditional public education, producing results at odds with most state and national assessments that show far better results. And the longer students are in charters, the better they do.

Obama’s Race to the Top would not be complete without such reforms, but Hunt errors in giving credit to states that have done little to create strong laws that allow for high numbers of high performing charter schools to flourish. The real test will be whether, when state legislators return to work, they will be willing to allow charters to start outside of school board control, free from union contracts and other constraints and funded equitably.