Tuesday, April 03, 2007

Most numbers don't speak for themselves

We're now in phase 2 of the annual USNWR rankings reaction: the PR machine's touting of the numbers for those schools that are highly ranked or that have moved up in the rankings, and the explanations for those schools that have stayed where they were ranked last year or that have moved down in the rankings. Bravo to Gordon Smith for this post: Law School Deans & the US News Rankings. He gets it exactly right.

At MoneyLaw, Paul Caron has posted Biggest Moves in the 2008 US News Law School Rankings, and I've posted a comment there, suggesting that--until we know where the natural "groupings" of schools are--the "numbers" don't mean much.

USNWR's rankings are, at best, useful for isolating trends, but they're not absolute measures of anything. In a way, they're exactly like curved grades for a course--they demonstrate a student's relative ranking against his peers on the test given that day, but they don't demonstrate absolute mastery and, at best, indicate trends that the student might want to investigate.

For each school that uses this year's rankings to say that the school itself is demonstrably better than before, perhaps that school could also list the changes that it has made--changes that the voters might know about through PR, for example--that have altered its inherent quality for the better. Houston has been doing great things for several years now (some say in spite of the "numbers") and perhaps the cumulative effect has had a change in its overall reputation. Peter Hoffman's work there--along with Jim Lawrence's work and the work of all of the folks in the Blakely Advocacy Institute--has certainly been paying off. But Peter and Jim have been doing their work without playing to the rankings. Their recognition in USNWR is a happy side effect, not their main aim.

In a way, phase 2 of the annual USNWR reaction reminds me of George Carlin's old bit about partial scores and sportscasters: "This just in.... [School X], 7."

No comments: