Michael Thaddeus, who specialises in algebraic geometry at Columbia University, US, has challenged the university’s No. 2 ranking this year with a statistical analysis that found key supporting data was “inaccurate, dubious or highly misleading”.

In a blistering critique on his website, Thaddeus is not only challenging the rating but redoubling the debate over whether college rankings — used by millions of prospective students and their parents — are even accurate.

Columbia said it stood by its data. Officials said there was no accepted industry standard for the data that goes into college rankings — every rankings project does it differently — and they strived to meet the technical requirements as set by U.S. News & World Report rankings of best universities. But, they said, the university was not necessarily defending the process.

“I think the majority of institutions would be happy if the rankings went away,” said Colin Diver, former president of Reed College, US, who has a book coming out about college rankings. “But as long as they are taken seriously by applicants, they’re going to be taken seriously by educators.”

This year, Columbia rose to No. 2 from No. 3, surpassed only by Princeton in the No. 1 spot and tied with Harvard and MIT. Thaddeus notes that Columbia was ranked 18th in 1988. “Why have Columbia’s fortunes improved so dramatically?” he asks in his analysis.

He does not question that in some ways Columbia has gotten stronger over the years, but some of the statistics immediately aroused his suspicion because they did not conform to his own observations as a professor in the classroom.

Searching further, he said he found discrepancies with other sources of data that he believes made undergraduate class sizes look smaller than they are, instructional spending look bigger than it is and professors look more highly educated than they are.

Columbia officials said that the numbers could be sliced in different ways, including in ways that would be even more favourable to the university. Asked about Thaddeus’ analysis, U.S. News & World Report did not address the details, but said that it relied on schools to accurately report their data.

Columbia officials said its rise had hardly been as precipitous as Thaddeus suggested. In 1989, the university vaulted 10 places, to eighth, in large part because the rankings relied more on data, and less on a survey of reputation among university presidents. It has been in the top five for a decade.

Thaddeus found discrepancies in important criteria that go into the ratings: class size, proportion of faculty with the highest degree in their field, percentage of faculty who are full-time, ratio of students to faculty and the amount of spending on instruction. Those categories make up about onefifth of the ranking formula used by U.S. News.

Columbia claimed that 100 per cent of its faculty had “terminal degrees”, the highest in their field; Harvard, for instance, claimed 91 per cent. By poring through the 958 full-time faculty members of Columbia College listed on its website, Thaddeus came up with 69 people (since corrected to 66) whose highest degree was a bachelor’s or master’s or a degree that was not in the field that they were teaching.

They include distinguished scholars like writer Orhan Pamuk who won the Nobel Prize but received a bachelor of arts from Istanbul University.

“Columbia would be a lesser place without them,” Thaddeus wrote. Columbia officials said that Thaddeus was fixated on the PhD but in many fields — like writing — that might not be the relevant degree.

Of the top-ranked universities, Columbia scored the best in the percentage of classes with under 20 students — 82.5 per cent. But by looking at the directory of classes, he calculated that the correct figure was between 62.7 per cent and 66.9 per cent. Columbia officials said the directory of classes Thaddeus relied on was not an official record of enrolment.

Thaddeus also disputed Columbia’s reported 6-to-1 ratio of students to faculty, calculating that using the U.S. News methodology, it should be between 8-to-1 and 11-to-1. Columbia officials said that if they included all part-time faculty the ratio would be even lower than 6-to-1, but they believed they were complying with the spirit of what U.S. News wanted.

On the financial front, Thaddeus found that Columbia’s claim that it spent $3.1 billion annually on instruction was “implausibly large” — more than Harvard, Yale and Princeton combined. He said that Columbia appeared to be including patient care in the spending, something he noted that New York University, for instance, does not do, to its detriment in the rankings.

Columbia officials essentially confirmed that point of his analysis, saying that the university educates close to 4,000 full-time equivalent students in medical fields, and that instruction and care often happen at the same time.

NYTNS