University home


From the Vice-Chancellor

vc-column

The Tertiary Education Commission (TEC) recently unveiled a new website which purports to provide the public with comparative data on the performance of students at tertiary institutions. In one sense, The University of Auckland can be well pleased with this – we rank first or second among the universities in all four performance indicators. Unfortunately, the institutions also know – and have pointed out repeatedly to the TEC and the Minister – that the data are not statistically robust and are therefore likely to mislead rather than inform the public.

Visit the TEC website

The fundamental problem is that the TEC uses unadjusted institutional average performance measures in its presentation. In reality, the performance measures are influenced by a great many factors, including the socio-economic backgrounds of the institution’s students, student ethnicity, part-time versus full-time status, subject area and whether the students are internal or extramural. It is highly likely that the comparison of institutions based on unadjusted averages simply reflects these underlying differences, rather than true differences in the teaching and learning environment, which is what is implied by the inter-institutional ranking of “performance indicators”. While the TEC website includes pie charts showing some aspects of the make-up of each institution’s student body, it is impossible for readers to use this information to “adjust” for these underlying parameters.

A second issue is that, even if they were robust, the data would be difficult to interpret. For example, is a high course completion rate a good thing because it reflects an institution that has excellent teaching and a high level of student support, or a bad thing because it reflects an institution that has low standards and makes it easy for students to pass?

Third, the website gives no indication of what are statistically significant differences between institutions, many of which are within a few percentage points of each other. Nor is there any international context for the data. International students are even less likely than domestic students to understand how the make-up of an institution’s student body is likely to affect its performance, yet they rely heavily on league tables of this kind to make decisions about where to study. Thus, despite the laudable aims of this exercise, it is likely to be misleading to both domestic and international students.