The Centre’s annual exercise to rank educational institutions overlooks teaching quality, lacks adequate transparency on methodology and data integrity and incorporates subjective perceptions with potential for bias, two educational administrators have said, flagging concerns about its reliability.

V. Ramgopal Rao, vice-chancellor of the Birla Institute of Technology and Science (BITS), Pilani, and Abhishek Singh, chief manager for strategic initiatives, BITS Pilani, have called for greater transparency and refinements in the methodology underlying the National Institutional Ranking Framework (NIRF).

The Union education ministry has released eight annual rankings — the latest in 2023 — since it introduced the NIRF in 2015. The rankings cover colleges, universities, engineering, dental and medical colleges, among other academic institutions. They take into account multiple parameters such as teaching, learning, resources, research practices, graduation outcomes, outreach and inclusivity, and perception. They are intended to help members of the student and the academic community select institutions where they want to study or teach.

Rao and Singh have asserted that the NIRF has proven to be a valuable tool for assessing and comparing educational institutions. But, in a paper accepted for publication in Current Science, a journal of the Indian Academy of Sciences, they have said certain issues, “if left unaddressed, may impact the credibility and relevance of the rankings”.

They have expressed concern that the NIRF rankings lack specific mechanisms to directly assess teaching quality, overlooking crucial aspects of classroom observations, student evaluations, and feedback from alumni.

“The omission of these evaluation methods hinders a comprehensive assessment of teaching effectiveness, leading to incomplete depiction(s) of an institution’s educational prowess,” they wrote.

The ranking’s limited focus on practical training elements such as hands-on projects and internships
also leads to an undervaluation of institutions that prioritise experiential learning,
they said. “This flaw disproportionately affects (institutions) that excel in providing real-world learning opportunities, potentially resulting in lower ranks despite their effectiveness in preparing students for practical applications in diverse fields.”

They have said the inclusion of perception into the ranking exercise adds subjectivity into the assessment with perceptions obtained from members of the academic community and from potential employers. “Students are the primary potential beneficiaries of the ranking exercise but their perceptions are not taken into account at all,” Singh said.

Rao and Singh have cited the 2023 rankings in which certain legacy institutions have earned lower perception scores compared to several relatively new institutions. For instance, BITS, a prestigious 60-year-old institution received a perception score of 25.3 compared to Graphic Era University’s 33.4.

Such perception scores may be influenced by the pool of respondents — academicians and employers — in any given year. Some may not be interested in participating in the survey for well-established institutions which could result in skewed perception scores influenced by who responds.