This won't be on the test!

EDUCATION: To Spot Bias in SAT Questions, Test Maker Tests the Test
By Amy Dockser Marcus
The Wall Street Journal, August 4, 1999

PRINCETON, N.J. -- The question won't ever appear on an SAT, but the answer is critical to the college-entrance exam's future: Is the SAT fair?

Educational Testing Service, which makes up the SAT, has worked for years to get rid of bias. Yet on average, black and Hispanic students continue to score below whites on the test, and women still lag behind men.

Now there's more concern than ever. In June, the U.S. Education Department's Office of Civil Rights began circulating draft legal guidelines outlining what it considers bias. As a result, colleges using the SAT may face legal action because of the disparate scores.

"We're not saying throw out the SAT, but there are inherent biases in the test," says Dennis M. Walcott, president of the New York Urban League.

"Score disparities exist, but the bias isn't in the SAT," responds Paul A. Ramsey, vice president of ETS.

The test maker argues that a variety of factors explain why whites and men continue to perform better on the SAT than other groups. It notes, for example, that better schools and higher parental income have an impact on test scores and that women still don't take as much advanced math and science as men. Critics counter that the test itself is the culprit, containing language and cultural nuances to which minority kids are less exposed.

ETS says all of its tests are checked for fairness by company reviewers and outside committees appointed by ETS and the College Board, the SAT's sponsor. Reading-passages featuring women and minorities have been added to make the test more balanced, and certain topics -- sports and the military, for example -- are avoided.

The company also tests some questions each year on a section of the SAT that doesn't count toward the total score and evaluates how students of various races and sexes fare in answering them. If one group answers a question significantly better than other groups do, the question is banished from future tests.

______

Wiped Out

Below are questions that were tried out on students who took the SAT exams in 1998. All of the questions were shown to be biased and will not appear on future SAT exams.

Verbal

-- DUNE: SAND::
a. beach: ocean
b. drift: snow
c. wave: tide
d. rainbow: color
e. fault: earthquake
Correct Answer: B

Results: 23% more whites than African-Americans and 26% more whites than Hispanics answered the question correctly.
Hypothesis: Regional variations. High proportions of African-Americans and Hispanics live in the south and southwest areas of the country, where there is less familiarity with terms associated with extreme winter weather.

-- Because Barbara McClintock's identification of 'jumping' genes represented a turning point in genetics, it is considered a ____ event.
a. formulaic
b. tenuous
c. landmark
d. universal
e. surreal
Correct Answer: C

Results: 9% more men than women answered this question correctly.
Hypothesis: Although the question is about a woman scientist, the science terms may have made women more uncomfortable than men.

-- The actor's bearing on stage seemed ____; her movements were natural and her technique ____.
a. unremitting. . .blase
b. fluid. . .tentative
c. unstudied. . .uncontrived
d. eclectic. . .uniform
e. grandiose. . .controlled
Correct Answer: C

Results: 9% more women than men answered this question correctly. 8% more African-Americans than whites answered this question correctly.
Hypothesis: Women and African-Americans tend to do better on questions dealing with the humanities or the arts.

Math

-- For all positive numbers a and b, is defined by -- (a + b)(ab).
If -- (2a + 2b)- (a + b), what is the value of k?
Correct Answer: 8

Results: 4% more females answered this question correctly than men.
Hypothesis: Females tend to do relatively better than males on questions, like the one above, that are from the curriculum or textbooks.

-- If 80 percent of X is equal to 20 percent of Y, then Y is equal to what percent of X?
a. 16%
b. 25%
c. 40%
d. 250%
e. 400%
Correct Answer: E

Results: 12% more men than women answered this question correctly.
Hypothesis: Females tend to do worse than men on questions involving percentages, particularly nonrounded number percentages or percentages over 100.

-- If 2x Squared is an integer, which of the following must also be an
integer?
a. x squared
b. x
c. 4x
d. x squared
e. 2x squared
Correct Answer: C

Results: 7% more African-Americans than whites answered this question correctly.
Hypothesis: African-Americans tend to do better on problems that come from the standard math curriculum and don't involve applied math.

-- At North Industries, 1,200 employees are in the health plan and half of all company employees are in the savings plan. Of all the company savings plan members, 800 are in the health plan and 250 are not in the health plan. How many employees of North Industries are not in the health plan?
Correct Answer: 900

Results: 16% more whites than Asian-Americans answered this question correctly.
Hypothesis: Asian-Americans tend to do worse on word problems applied to real life situations than whites of the same ability.

Source: Educational Testing Service

For all the latest on the tweaking of the SAT, see THE SAT SCRAMBLE: A Compendium of Recent Events.

CER Home Page Monthly Letter E-Mail Us Publications List