Enhancement and Evaluation of Teaching and Courses Policy and Procedures


Application


This document applies to all students and staff members of the University

Purpose


To support the University’s goal to develop and maintain a high-quality learning environment by providing a framework for the evaluation and quality assurance of learning and teaching at the level of the course. 

This policy is supported by the Enhancement and Evaluation of Teaching and Courses Guidelines

Policy


Principles

1.    Students will have the opportunity to provide feedback about their experiences in their courses and in turn receive feedback from teaching staff members

2.    As part of academic staff development and for quality assurance purposes staff members will use multiple methods of evaluation to reflect on and enhance their teaching and provide evidence of teaching capability and excellence

3.    Course and teaching evaluation evidence collected by the University will form one part of the evidence used to demonstrate course quality or teaching capability

4.    Regular end-of-course reviews will contribute to the ongoing management and quality assurance of courses offered

5.    Course pass rates will be monitored regularly

Summative Evaluation Tool (SET) course and teaching evaluation

6.     Student participation in SET course and teaching evaluations is to be voluntary

7.     All eligible undergraduate and postgraduate taught courses are to be evaluated each time a course is offered

8.     All eligible teachers associated with a course are to be evaluated each time a course is offered

9.     Eligible teachers are those who:

(a)  have one of the following permanent or fixed-term roles: professor;  associate professor; senior lecturer; lecturer; senior research fellow; research fellow; professional teaching fellow; senior tutor

OR

(b)  are employed on a contract basis with a significant teaching role 

10.     The final decision on a contract teacher’s eligibility rests with the associate dean with responsibility for learning and teaching, in consultation with the academic head

11.     Tutors, graduate teaching assistants, and honorary and adjunct roles are not eligible for SET

12.     SET course and teaching evaluations are conducted at the level of the course

13.     Course and teaching evaluation questionnaires are to be administered centrally in one evaluation instance in the final two teaching weeks of a semester or quarter

14.     Teaching staff are to provide the current and next cohort of students with a summary of their response and the actions they will take as a result of the feedback

15.     Student responses in summative evaluations are confidential.  The identity of individual students must not be revealed to teaching staff

16.     SET teaching and course reports are to be provided to individual staff members and academic managers according to role

Formative evaluation

17.     Formative evaluation, including where appropriate peer observation and review, will be conducted by individual teaching staff members or teaching teams during a course to assess how well teaching is contributing to student learning

Course Review

18.     Course Reviews must be co-ordinated by Course Directors after every offering of a course. Course review reports are to be included in the documentation for the Academic Development and Performance Reviews of academic staff members who have contributed to the delivery of the course

Pass rates

19.     Courses not meeting a nominated threshold pass rate must be subject to additional review within the faculty

20.     Courses under the threshold rate are to be discussed with the faculty dean and senior staff members during annual meetings with the Deputy Vice-Chancellor (Academic)

Roles and responsibilities

21.      Students are responsible for providing honest, responsible, timely and constructive feedback on their experiences of the learning and teaching process  Note: feedback should be provided in terms that are consistent with the expectation that members of the University treat one another with respect

22.     Academic staff members are responsible for: using course and teaching evaluation information, together with formative evaluation methods, to reflect on the learning and teaching experiences of students within a course; providing feedback to students; and using properly contextualised evaluation information in applications for promotion, continuation or teaching awards

23.     Course directors are responsible for supporting the quality enhancement and assurance of a course by using course evaluation information in course review

24.     Academic heads are responsible for: using evaluation information to monitor teaching quality and the learning and teaching experiences of students within an academic unit; and reviewing course and teaching evaluation information as one source of evidence for ADPR and promotion and continuation applications

25.     Deans (or nominated delegates) are responsible for ensuring follow-up formative and summative evaluations are undertaken where courses do not meet the overall satisfaction and pass rates thresholds as determined by this policy and associated procedures

26.     The Deputy Vice-Chancellor (Academic) is responsible for: maintaining the policy framework governing course and teaching evaluations; ensuring faculties are informed of courses falling below pass rate and overall satisfaction thresholds and that they address contributing factors

27.     The Academic Quality Office (AQO) is responsible for: management of the evaluation system and process, including commissioning of evaluations; configuration of individual course and teaching evaluation reports; and aggregated reporting from SET for academic managers.  The AQO also prepares, together with the Planning and Information Office, annual reports on evaluation results for the faculties and Teaching and Learning Quality Committee

28.     The Planning and Information Office (PIO) is responsible for evaluation reporting through dashboards in Strategic Management Reporting, and for supporting the load of data for the configuration of SET evaluations  

29.     ITS Academic Services is responsible for the management of system issues, as required, including upgrades and maintenance and SET data integration 

30.     Teaching and Learning Quality Committee is responsible for advising and recommending on policies and procedures for the evaluation of learning and teaching and for monitoring institutional-level reports on evaluation information 

31.     The Senior Leadership Team (SLT) is responsible for monitoring institutional-level reports on evaluation information

32.     VCDD is responsible for approving the annually determined threshold for pass-rates

 

Procedures


SET course and teaching evaluations

Evaluation schedule

33.    A schedule of evaluation periods is to be published annually on the evaluations website

Evaluation questionnaires

34.    SET course and teaching evaluation questionnaires are to be made available on the evaluations website

35.    Supplementary questions may not be added to the course and teaching questionnaires

Evaluation process

36.    Eligible courses and/or teachers may not be opted out of evaluation

37.    Faculties are to be advised of the list of courses and teachers scheduled to be evaluated in each evaluation period

38.    Eligible staff members with a ‘Teacher’ role in Canvas and a University identity (username) will be evaluated in SET

39.    Faculties are responsible for checking and updating teacher roles in Canvas 

40.    If completed before the deadline advised by the Academic Quality Office, corrections will be reflected in SET 

41.    Only eligible academic staff (as outlined in the Policy) may be given a ‘Teacher’ role in Canvas 

42.    Just before each evaluation period the Academic Quality Office is to provide faculties with a SET configuration summary, including the final list of the courses and teachers to be evaluated

43.    Students are to be notified about evaluations due for completion through email and notifications in Canvas.  Students will receive targeted reminders

44.    Students may complete evaluations online at any time during the evaluation period on a computer or other device, and may be asked to complete evaluations in class   

45.    Students are to answer questions on the course, followed by questions for each selected staff member.  Students may select teachers using teaching names and photographs, where a photograph has been published on the University Directory or is available

46.    Information is to be available to students about providing constructive and appropriate feedback

47.    Faculty associate deans with responsibility for learning and teaching may monitor response rates during the evaluation period

Feedback to students

48.    Teaching staff are to provide feedback on SET results to the current and next cohort of students.  This may be done through Canvas

Reporting

49.    Course and teaching evaluation reports are to be available to teaching staff, Course Directors and academic managers after the close of the evaluation period

Note - SET evaluation reports are generated at the level of the course

50.    SET course reports are to be distributed to academic staff, course directors, and their academic managers (including academic heads and deans)

51.    SET teaching reports are to be distributed to individual teachers and their academic managers (including academic heads and deans)

Note - Teaching staff may access course reports, and their own individual teaching reports, through the link within Canvas courses.  Academic managers may access summary reports and individual course and teaching reports through the link within Canvas courses.  Further information on SET results is available through Strategic Management Reporting (SMR) dashboards.  Deans and associate deans with responsibility for learning and teaching have access to secure dashboards for detailed evaluation results

52.    To protect student anonymity individual SET teaching reports must not be distributed to teachers for courses with fewer than five responses.  These data will be included in aggregated reporting

53.    SET course and teaching reports are not to be automatically available to professional staff members

Note: professional staff may use the open-access dashboards in SMR for SET information At the discretion of a dean (or delegate) or academic head, copies of evaluation reports may be made available to professional staff members

54.    The Planning and Information Office may consider requests for amendment to SET data on a case-by-case basis.       

Use of SET data  

55.     SET evaluation results will inform Course Review

56.    Faculty SET results are to be reviewed each semester and quarter by associate deans with responsibility for learning and teaching and academic heads

57.    Summary annual Faculty reports and details of courses with an overall student satisfaction result of less than 70%, and pass rates under the threshold rate, are to be notified to and reviewed with faculty deans

58.    A summary annual report on evaluation results for the University is to be reviewed by TLQC

59.    Staff members may add contextual statements to quantitative evaluation results used for career purposes

60.    Academic managers and staffing committees may take into account contextual information when using summative evaluation information for performance review or career progression purposes

Pre-2016 course and teaching evaluation data

61.     Staff may request copies of pre-2016 (Cecil) evaluations through the Academic Quality Office by contacting quality@auckland.ac.nz

Confidentiality of SET data

62.    Student identification will be used by the system to send targeted reminders to students who have not completed evaluations, and to link to student attributes as recorded on the student management system for the purposes of further demographic analysis of results

63.    Student identification numbers and/or names must not be identified in any reporting

SET open-ended comments

64.    SET open-ended comments are to be provided to staff unedited

Note -  A staff member who considers that a student comment has breached the University’s Prevention of Bullying, Harassment and Discrimination Policy and Procedures may report concerns to their Academic Head.  Any subsequent investigation must be conducted in accordance with the Prevention of Bullying, Harassment and Discrimination Policy and Procedures and the Statute for Student Discipline

65.    Inappropriate comments are to be removed from evaluation reports at the recommendation of the Academic Head following a finding of misconduct

66.    The identity of the student is not to be disclosed to the complainant unless approved by the Deputy Vice-Chancellor (Academic) 

Evaluation process

67.    Formative evaluation will normally be conducted at an appropriate point between the third and tenth week of semester (or at corresponding points for other teaching periods)

68.    Formative evaluation will enrich and supplement information provided by formal SET course and teaching evaluations in career review and development processes

Course Reviews

69.    Course reviews must include evaluative reflection on course structure, content and delivery; assessment; learning outcomes; student feedback; staffing, resources and facilities

70.    A completed course review form is to be submitted to the Head of Department within one month of the submission of final grades

Note: a template for Course Reviews is available on the Learning and Teaching website

Pass rates

71.    Pass rates are to be monitored at the level of the academic unit (through course review and examiner’s meetings); at the faculty level and at the University level

72.     In accordance with an Education Committee requirement, faculties must provide a report and commentary on pass rates every two years

73.    When a course has a pass rate under the defined threshold rate:

  • the academic head must review the outcomes of the course on its next offering

  • a reflective statement reviewing issues that contributed to the low pass rate and discussing progress in resolving them is to be submitted to the Dean (or his or her delegate)

  • deans (or delegates) are responsible for ensuring appropriate response actions are taken, and for monitoring subsequent pass rate information

74.     When an evaluation returns an overall SET course quality satisfaction result of less than 70%, AND has a pass rate under the threshold rate:

  •  a reflective statement reviewing issues that contributed to the low student satisfaction and low pass rate and discussing progress in resolving them is to be submitted to the dean or nominated delegate
  • reflective statements and the results of subsequent SET evaluations may be reviewed by the DVC(A) with faculty deans or their nominated delegate

75.    Where a course does not meet the threshold pass rate in consecutive offerings the DVC(A) is to advise the Vice-Chancellor

Resources and assistance


 

Note: further assistance with the evaluations process, including advice on the interpretation of evaluation results, is available from the Academic Quality Office, quality@auckland.ac.nz

Further guidance on evaluation practice is available from:

Enhancement and Evaluation of Courses and Teaching Guidelines

The Evaluation website - www.auckland.ac.nz/evaluate

Definitions


The following definitions apply to this document:

Course Review is a review, involving all academic staff contributing to a course, conducted after the end of each delivery of the course

Formative evaluation is in-course feedback gathered by a variety of means during course delivery to understand how students are responding to the course and the teaching methods

Pass rates are the percentage of the enrolled students in a course who gain a pass grade. Enrolments are measured in EFTS. All students who remain enrolled after the deadline for deletion of enrolments are included in the calculation of pass rates

Peer observation and review is normally a collaborative process in which academic partners work together to understand and improve their teaching practices

Staff member refers to an individual employed by the University on a full or part-time basis

Summative evaluation is formal evaluation using the standardised University survey instrument

University means the University of Auckland and includes all subsidiaries

Document management and control


Content manager: Manager, Academic Quality Office

Owned by: Deputy Vice-Chancellor (Academic)

Approved by: Council

Approval date: 30 April 18

Review date: 30 April 21