Higher Ed | Daily Report

Flickr photo by Dave Scelfo

A new paper co-authored by a University of California Berkeley education professor could add fuel to the long-simmering debate on whether the widely used SAT college admissions exam is unfair to African-American students.

The paper, published in Harvard Education Review and first reported in Jay Mathews' blog at the Washington Post, found that African-American students fared worse on certain questions from the verbal portion of the SAT than white students with equal cognitive ability.

"These findings are important because they show that the SAT, a high-stakes test with significant consequences for the educational opportunities available to young people in the United States, favors one ethnic group over another," wrote the study's authors, Maria Veronica Santelices of Pontificia Universidad Católica de Chile, and Mark Wilson of UC Berkeley.

The College Board has challenged the study, saying the data sample is too small and that the findings are inconsistent, according to coverage in Inside Higher Ed.

The new paper actually re-examines an older study by Roy Freedle, whose 2003 analysis was reported in Atlantic Monthly and was summarily rejected by the College Board, which argued that there is no bias in the test – only differences in students' education.

Despite the College Board's dismissal of Freedle's research, then-UC President Richard Atkinson was intrigued. He commissioned new research into Freedle's findings. This year's study by Santelices and Wilson is the end product of that work. It took the researchers two years to get the detailed data they wanted from the College Board.

In their analysis, Santelices and Wilson confirmed Freedle's findings of racial bias in SAT questions, even though they used newer SAT test data and addressed some critics' concerns about Freedle's metholodogy.

The newer paper looked at SAT responses for all California students from public schools who took two specific SAT forms in 1994 and 1999.

Both the current study and Freedle's 2003 report found that African-American students who were equally matched with white students did worse on the "easier" verbal questions, while they did better than matched white students on the most difficult questions. The researchers found this even though the College Board screened test items for fairness.

They also tested three different methodological approaches, focusing on a specific criticism waged against Freedle's work by Educational Testing Service researchers. Santelices and Wilson said the patterns still held up, although the math section did not show such differences in performance.

The researchers used data from before the SAT was revised in 2005, but the researchers said that does not affect their results because the changes to the test were minor and the main revision was the addition of a writing section.

Santelices and Wilson warned that their findings could present legal challenges for universities and other institutions that use the SAT to determine admission. The SAT is the most widely taken college admissions test, with more than 1.5 million students taking it every year.

"Neither the specifics of the method used to study differential item functioning nor the date of the test analyzed invalidate Freedle’s claims that the SAT treats African-American minorities unfairly," they wrote.

"The confirmation of unfair test results throws into question the validity of the test and, consequently, all decisions based on its results. All admissions decisions based exclusively or predominantly on SAT performance – and therefore access to higher-education institutions and subsequent job placement and professional success – appear to be biased against the African American minority group and could be exposed to legal challenge."

Notably, Santelices and Wilson did not find evidence of the Freedle phenomenon in the comparison between Hispanic and White students.

They also did not dig into what drives the performance differences. Freedle's 2003 report included some hypotheses about some of the potential reasons for the phenomenon – namely, he suggested that "easy” verbal items tap into a more culturally specific content.

College Board spokeswoman Kathleen Fineout Steinberg told Inside Higher Ed that Santelices and Wilson's analysis falls short:

She said every test question used on the SAT is subjected to rigorous analysis (before use) to weed out any that would not be fair to all test takers. 'We believe that our test is fair,' she said. 'It is rigorously researched, probably the most rigorously researched standardized test in the world.'

 

As to the persistence of score differences, Steinberg said that this is not because of the test. 'There certainly are subgroup differences in scores,' she said. 'We recognize that and acknowledge it. It's a reflection of educational inequity. It's something we are concerned with.' She also said that the College Board welcomes research on the SAT, but viewed the Freedle study as having been 'discredited,' and said that nothing in the new study changed that view.