What do SAT, ACT scores really mean?

The big news in the world of college entrance exams — that would be the SAT and the ACT — is that the scores from the high school class of 2012 were disappointing.

Newly released reading scores on the SAT hit a four-decade low, and
(bigstock) writing scores edged down too, while math scores were essentially unchanged from last year.  SAT average scores have declined by 20 points since 2006, when the test was revised to include a writing section.

ACT scores for the 2012 high school class were released in August and the national results were essentially the same as in 2011, meaning no real progress.

So what does it all mean? On one level, a lot. On another, pretty much nothing.

We all know that many college admissions offices imbue SAT and ACT scores with importance. At schools that are deluged with tens of thousands of applications, numbers matter, so these scores can play an outsized role in admission decisions. That means these scores can affect where individual students get to go to school.

Yet significant research shows that SAT and ACT scores don’t really tell us anything meaningful about a student’s future, either academically or in the work world.

For one thing, lots of things can affect how well a student does on a high-stakes test, but the strongest correlation to any single factor is family income. As my colleagues Lyndsey Layton and Emma Brown reported here, average SAT scores increase with every $20,000 in additional family income

Furthermore, these tests are highly coachable, even if the organizations that own and administer them say they aren’t, giving a boost to kids who have the money and/or wherewithal to get tutored.

The content of the ACT is closer to the material a student has covered in high school (in fact, it is based on a national curriculum survey) than is the content of the SAT — but it is still no more accurate than the SAT in predicting college grades, research has shown. Why? Because no standardized test in which students sit there and fill in bubbles and write an essay can capture all of the work habits, coping skills, motivation and other traits needed to be successful in college.

Thus at the individual level, ACT/SAT scores are not particularly meaningful, says Bob Schaeffer, public education director of FairTest, or the National Center for Fair & Open Testing, is a non-profit organization that is dedicated to ending what it says are misuses and flaws in standardized testing.

“High school grades — even with all the variety between schools and courses — are better predictors of a teenager's performance in higher education, particularly the likelihood of graduation,” he said.

That's why 875 accredited, bachelor-degree granting colleges and universities do not require all or many applicants to submit test scores before making admissions decisions. The list of test-optional and test-flexible schools includes nearly 150 ranked in the top tier of their respective categories; the full database is available online here.

But Schaeffer said long-term, aggregate SAT and ACT scores trends are one tool for evaluating overall education quality — and they point to the conclusion that U.S. K-12 education is headed in the wrong direction.

For Schaeffer, that’s an indictment of the high-stakes fixation of the last decade of No Child Left Behind . “At a minimum, we are failing to make the progress promised by high-stakes testing advocates either in terms of improving overall readiness for college/careers or in closing long-standing test score gaps between racial groups,” he said.

Proponents of NCLB and similar state-level standardized testing programs have been saying that test-based accountability systems would lead to increased achievement. It hasn’t happened, and in fact, the opposite has.

The SAT and the ACT are said by the organizations that own and administer them that the exams are not really “coachable” and that kids who take lessons on how to improve their score don’t really see much gain.

The 2009 book, "Crossing the Finish Line: Completing College at America’s Public Universities," co-authored by former Princeton President William Bowen, found that:

* High school grades are a far better incremental predictor of graduation rates than are standard SAT/ACT test scores.
* Overly heavy reliance on SAT/ACT scores in admitting students can have adverse effects on the diversity of the student bodies enrolled by universities.
* The strong predictive power of high school GPA holds even when we know little or nothing about the quality of the high school attended