Each
semester I spend a day in my Public Speaking class talking about critical
thinking through the lens of advertising claims. I point out that it is against the law for
advertisers to lie, but that they are allowed to make claims that lead us to
make incorrect assumptions.
What
advertisers don’t say may be far more meaningful and significant than what they
do say. Back in the 1970s the
now-defunct National Airlines was a leading carrier to Florida, and ran ads
touting the fact that no other airline flew to Miami/Fort Lauderdale for less
than National. What they didn’t say was
that all fares were the same, because the Federal government regulated airline
fares. When Folgers coffee claims that it’s “mountain grown,” what isn’t said
is that all coffee is in some sense mountain grown (Arabica beans are grown at
higher elevations than Robusta beans). When
products like shampoo and toothpaste highlight some special ingredient, what
they fail to mention is that their competitors have the same ingredient.
Evaluating
advertising claims requires knowing the right question to ask. When Trident chewing gum reports that 4 out
of 5 dentists surveyed recommend sugarless gum (for their patients who chew
gum), the logical question is “What would I expect the results to be?” I would expect 100% of dentists to prefer
sugarless, making the statistic not that impressive. When an actor washing his hair with ten times
as much shampoo as needed claims to know that Denorex is working because it
tingles, the relevant question is “Do you need your shampoo to tingle?” When products are “new and improved,” the question becomes, "Is that
really an acknowledgement that the old product was flawed?"
I
thought about that upon learning a couple of weeks ago that there will be a
“new and improved” SAT beginning in 2016.
Within minutes of the announcement, I was being asked what I thought by
parents and colleagues, and my answer is that I’m not sure. I like a lot of what I’ve heard. I like the idea that the test might be more
closely aligned with what students study in high school. I like abandoning the 2400 scale, because it
is still odd every time I hear someone talk about having gotten a 1950 on the
test. I like the fact that the SAT will become less a test of stamina now that
the Writing section will be optional, and being able to guess your way to success
without penalty seems to reflect some back corner of the American Dream. But I am also by nature a skeptic, especially
when it comes to the College Board, and there are still a number of questions
to be answered.
Foremost
among those questions is whether the changes are motivated by philosophy or
economics. I would like to believe that
the changes were in response to the thoughtful work done by the NACAC
Commission on Testing, but that seems unlikely given how quickly the College
Board was to dismiss and ignore the Commission report.
There
is an ongoing internal battle for the soul of the College Board—membership
organization or corporate entity, .org or .com?
In recent years the corporate forces seem to be winning the war.
College
Board meetings too often feel like infomercials for College Board products, and
when I attended the business meeting at a National Forum several years ago, the
treasurer reported a “non-profit” of $95 million, a figure that many businesses
would envy. That wealth, much of it
generated by SAT fees, allows the College Board to do a lot of good, but it
also leads to suspicion that most College Board decisions are based on how they
impact the bottom line. Is the new test an attempt to better measure
the skills and knowledge students need in the 21st century, a
reaction to the fact that the SAT has lost market share to the ACT, or a
strategic first step to position the College Board to become the primary
provider of assessments of the new Common Core standards (partly written and
developed by new College Board President David Coleman)?
Those
questions are particularly relevant given the fact that the “new and improved”
SAT reverses the miracle ingredient from the last iteration of the SAT, the
Writing section. At the time, the
addition of the Writing section seemed designed to keep the University of
California system as clients, and almost immediately critics such as MIT’s Les
Perelman argued that the 25-minute essay and prompts were lousy measures of a
student’s ability to write, especially when the scoring rubric did not penalize
a student for writing that the War of 1812 started in 1944.
The
other significant piece to the recent announcement is the College Board’s
collaboration with Khan Academy to produce free test prep materials. That seems like a good attempt to reestablish
the SAT as a tool for college access rather than a test that measures and
rewards privilege. I have always been a
skeptic when it comes to the test-prep industry, believing that it is one of
the marketing success stories of our time and that the benefits of test prep
are far more modest than generally assumed (but that may be the naïve dinosaur
within me speaking), and it bothered me that several media reports about the
new SAT took for granted that scoring well on the SAT is purely a function of
test prep rather than economic advantage.
Whatever the reality, it hurts the credibility of the College Board and
the college admissions profession if it is possible, or perceived to be
possible, to game the test and the admissions process.
That
leads to the final question. What does
the SAT measure, and is it measuring the right things? It was originally designed to predict
freshman year performance in conjunction with high school grades. Grade inflation makes such a tool necessary,
but studies such as that recently done by former Bates Dean of Admission Bill Hiss
on the long-term impact of test optional policies raise questions about how much
added predictive value SAT scores provide. More importantly, neither the SAT, ACT, or
any other test adequately measure personal qualities such as motivation, work
ethic, and “grit” that may be the best predictors of success. A test that could measure those would truly
qualify as “new and improved.”