California parents have yet another way to assess how their local schools are doing.
The California Charter Schools Association has developed a new metric, called a Similar Student Measure [PDF], to assess how a school's students perform on state tests compared to other schools serving similar students. The measure is used to identify which schools overperform or underperform over a three-year period by comparing a school's Academic Performance Index to a predicted API that controls for the impact of students' background in those schools.
The measure was developed to assess how charter schools are doing compared to regular public schools, but could have wider utility for the majority of schools in California.
Using a complex regression analysis, the measure takes a number of characteristics of the school's student population into account. These include the socioeconomic background of the student body, the average education level of their parents, the number of students with disabilities, the percentage of English language learners, and the racial and ethnic makeup of the students.
By going to this mapping tool, parents can look up any school in the state.
The state currently ranks every school on a "similar school" scale, from one to 10, which assesses how a school does compared to 100 other schools with similar student populations on the state's annual Academic Performance Index.
But, the researchers on this report said, those similar school rankings can fluctuate from year to year. Many schools, especially smaller ones, don't receive a "similar schools" ranking at all.
At the same time, the analysis underscores the limitations of using complex methodological techniques. For those without statistical backgrounds, simply understanding the techniques used is a challenge.
Another limitation is that it places the majority of schools [PDF] – 78 percent – in a single category of performing within 5 percent of their predicted level on state tests. Just under 2 percent of schools performed far below their predicted level, and 1.6 percent far above. Another nearly 10 percent performed just below their predicted level, and another 10 percent just above.
Samantha Olivieri, the charter schools association's accountability manager, said that the purpose of the tool is "to identify schools at the polar ends, those that are far overperforming and those that are underperforming."
"We can say with a high degree of certainty those schools are overperforming or underperforming," she said.
But she acknowledged that the measure is not very helpful in explaining what is happening in the 78 percent of schools that fall into the middle category.
"When you have such a large portion of the school in that category, it does not tell you that much," Olivieri said. "We recognize that as a limitation and hope to be able to do further research, and identify more performance classifications, and identify with more precision additional levels of student performance."
While she acknowledged the important role of a student's background in influencing test scores, she said the report shows that it does not have to be the decisive factor in determining student achievement. "What we find is that many schools, and many charter schools in particular, are proving that background is not destiny," she said.
In particular, she noted, the new measure shows that charter schools are four times more likely to be among the top 5 percent of schools that exceed their predicted test scores – and twice as likely to be among the bottom 5 percent across the state.
At the release of the ratings report last week, the California Charter Schools Association rebutted questions about whether its findings were influenced by the fact that the research was carried out by the association's own staff. Staff members said the work had been reviewed by outside experts, and that the report had been endorsed by U.S. Education Secretary Arne Duncan, who said it "confronts the successes and shortfalls of California's charters with unblinking candor."
The report does in fact identify a number of low performing charter schools, in addition to higher performing ones. However, the report's greater contribution may be that it could also help identify regular public schools whose students are doing extremely well or poorly on state tests, while taking students' backgrounds into account more than any existing rating system in the state.