The Great Ratings Debate

Office of the President

Macalester College
208 Weyerhaeuser Hall
62 Macalester Street
St. Paul, MN 55105-1899
651-696-6207
651-696-6500 fax

This "Household Words" column appeared in the Fall 2007 issue of Macalester Today.

By Brian Rosenberg

Absent a personal scandal or similar disaster, small-college presidents rarely make big-time news. This past summer was different, however, as presidents representing institutions in the Annapolis Group--an informal organization of 125 of the leading U.S. liberal arts colleges--took up arms against the annual U.S. News & World Report rankings of American colleges and universities.

Although it took no formal action, the group announced that "the majority of the Annapolis Group presidents attending the annual meeting in Annapolis, Maryland, expressed their intent not to participate in the annual U.S. News & World Report ranking exercise." (Macalester is a member of the group, but I was unable to attend this year's meeting.)

The meaning of this statement is less than wholly clear for two reasons: first, because to date fewer than half of the Annapolis Group colleges have taken a public stance on U.S. News rankings, and second, because the impact of a decision "not to participate" is difficult to gauge. Most of the data used to construct U.S. News rankings are publicly available, allowing the magazine to get much of the information it wants without institutional cooperation. When the information is not available, U.S. News simply estimates. Reed College in Oregon, for example, has refused for years to participate in the U.S. News exercise, yet it is ranked alongside the rest of the private colleges.

Probably what most rebellious presidents intend is that they will no longer participate in the reputational survey that determines about a quarter of each school's score. In this survey--sent to the president, chief academic officer, and chief admissions officer of participating institutions--we are asked to rate the academic quality of a long list of our peer schools.

It is a silly exercise--what do I know about the true quality of what happens at most other colleges?--yet it plays a critical role in the rankings. It also pretty much guarantees that those rankings will turn out as the public expects them to and that there will be little change from year to year: the best-known colleges with the strongest national reputations will naturally score best on a reputational survey. Of course, this is in the interest of U.S. News: how much credibility would the rankings retain if Harvard and Williams suddenly slipped to the middle of the pack?

As I ponder the right decision for Macalester with regard to U.S. News, I am caught between competing impulses. On the one hand, I believe the rankings are unreliable and misleading, constructed as they are from a combination of highly suspect reputational perceptions, financial indicators, and inputs such as SAT scores and selectivity. They speak not at all to the actual quality of the education or to the actual value-added at any college. They have led too many college applicants to care more about a college's ranking than about whether it is the right institution for them and, unhappily, too many colleges to manage toward a higher ranking. And they misleadingly apply a zero-sum-game business model to higher education. Whereas it is probably true that if Toyota improves, Ford suffers, it is not equally true that if Macalester improves, Grinnell suffers. Yet the rankings suggest that if one college rises, another must fall.

On the other hand, I recognize that U.S. News will continue to publish its college rankings issue as long as it continues to be profitable, regardless of the criticism of college presidents. Indeed, such criticism may actually increase the magazine's impact, since controversy tends to spark interest (as those who have attempted to ban books have never quite learned).

If college presidents refuse to fill out the reputational survey, U.S. News will likely begin to survey high school counselors, corporate CEOs, or some other group whose knowledge is equally or more anecdotal.

In the end, I am inclined to think that more information is better than less and that consumers should determine which forms are most useful and reliable. Attempts to suppress even the most baseless and scurrilous publications have rarely succeeded. Rather, it is incumbent upon those of us who dislike the efforts and influence of U.S. News to come up with alternative information sources for college applicants that will prove more helpful (a project upon which the Annapolis Group has now embarked). Bad practice is defeated not by broadsides and boycotts but by better practice. Our goal should be not to suppress the flawed rankings of U.S. News, but to make them irrelevant.