Our good friends at IvyGate, through what I assume must be great cleverness and sneakery, posted the top 25 overall and top 25 Liberal Arts colleges in the U.S. News’ 2008 rankings earlier today. At first had ethical reservations about saying really anything on the topic since I felt I could be indirectly promoting the rankings which I criticize frequently for their negative impact on the college search and application process (as Thacker would say, for their commercializing of it).

Then I saw that some more of our good friends, this time at EphBlog, had reposted some of it (for the LACs) and so had some other blogs, so I said why not cover it myself! Those of you following logically should realize that that should do nothing to clear my conscience, but all the same I’m going to write about the rankings a little : )

I’ll relate the shocking news right now: The top 3 slots are the same as last year! Gasp! In the same order, no less–Princeton, Harvard, Yale. (For complete list, see the end of this post) While we’ve all become accustomed to the top 10 or so’s relative lack of volatility over the years, it’s worth remembering the way the methodology has been changed based more on editorial discretion than statistical or scientific merit. The methodology is explained on USNews.com; I will look through it and discuss the changes they made this year in another coming post.

Steve Hsu, who writes a totally awesome blog called Information Processing (he’s a physics professor at the U of Oregon), brought a Slate article to my attention back in July. It’s a fun read, centered around an explanation of the various ‘fudge factors’ that U.S. News uses to make sure the rankings maintain a certain… standard, shall we say.

The story of how the rankings were cooked goes back to 1987, when the magazine’s first attempt at a formula put a school in first that longtime editor Mel Elfin says he can’t even remember, except that it wasn’t HYP. So Elfin threw away that formula and brought in a statistician named Robert Morse who produced a new one. This one puts HYP on top, and Elfin frankly defends his use of this result to vindicate the process. He told me, “When you’re picking the most valuable player in baseball and a utility player hitting .220 comes up as the MVP, it’s not right.”

The article is from 2000, and I know there have been changes since then, but the points it makes are still entirely valid as they touch on the whole history of the rankings. In 1999 Caltech was #1 but the next year dropped to #4; the reason for this was the application of special ‘logarithmic adjusters,’ applied only in categories where Caltech had an edge on HYP. These ‘adjusters’ in place, Caltech dropped back down, HYP went to the top… problem solved, from U.S. News’ perspective.

…the credibility of rankings like these depends on two semiconflicting rules. First, the system must be complicated enough to seem scientific. And second, the results must match, more or less, people’s nonscientific prejudices. Last year’s rankings failed the second test. There aren’t many Techie graduates in the top ranks of U.S. News, and I’d be surprised if The New Yorker has published a story written by a Caltech grad, or even by someone married to one, in the last five years. Go out on the streets of Georgetown by the U.S. News offices and ask someone about the best college in the country. She probably won’t start to talk about those hallowed labs in Pasadena.

The fact that the formulas had to be rearranged to get HYP back on top doesn’t mean that those three aren’t the best schools in the country, whatever that means. After all, who knows whether last year’s methodology was better than this year’s? Is a school’s quality more accurately measured by multiplying its spending per student by 0.15 or by taking a logarithmic adjuster to that value? A case could also be made for taking the square root.

But the logical flaw in U.S. News’ methodology should be obvious—at least to any Caltech graduate. If the test of a mathematical formula’s validity is how closely the results it produces accord with pre-existing prejudices, then the formula adds nothing to the validity of the prejudice. It’s just for show. And if you fiddle constantly with the formula to produce the result you want, it’s not even good for that.

Caltech is #5 this year. Happy rankings everyone…

Here’s the Top 25, after the break:

Best National Universities
1. Princeton University (NJ)
2. Harvard University (MA)
3. Yale University (CT)
4. Stanford University (CA)
5. California Institute of Technology
University of Pennsylvania
7.  Massachusetts Inst. Of Technology
8.  Duke University (NC)
9.  Columbia University (NY)
University of Chicago
11. Dartmouth College (NH)
12. Cornell University (NY)
Washington University in St. Louis
14. Brown University (RI)
Johns Hopkins University (MD)
Northwestern University (IL)
17. Emory University (GA)
Rice University (TX)
19. University of Notre Dame (IN)
Vanderbilt University (TN)
21. University of California – Berkeley
22. Carnegie Mellon University (PA)
23. Georgetown University (DC)
University of Virginia
25. University of California – Los Angeles
University of Michigan – Ann Arbor

Best Liberal Arts Colleges
1.Williams College (MA)
2.Amherst College (MA)
3.Swarthmore College (PA)
4.Wellesley College (MA)
5.Carleton College (MN)
6.Middlebury College (VT)
7.Bowdoin College (ME)
Pomona College (CA)
9. Davidson College (NC)
10. Haverford College (PA)
11. Claremont McKenna College (CA)
Grinnell College (IA)
Vassar College (NY)
Wesleyan College (CT)
15. Harvey Mudd College (CA)
Washington and Lee University (VA)
17.Colgate University (NY)
Hamilton College (NY)
Smith College (MA)
20. Oberlin College (OH)
United States Naval Academy (MD)
22. Colby College (ME)
United States Military Academy (NY)
24. Bates College (ME)
Bryn Mawr (PA)