Online Learning Evaluations 2020

Guidance for staff on evaluation of courses and teaching in Semester 1 and Quarter 2, 2020

Online Learning Evaluations

SET course and teaching evaluations will be replaced with Online Learning Evaluations (OLEs) for Semester One and Quarter Two, 2020.  The OLEs will evaluate courses in the online environment with a modified questionnaire.

Will there be both course and teacher evaluations?

Teacher evaluation questions will NOT be included in the OLEs.  This is because faculties have expressed a desire to minimise pressure on academic staff at this time, while still providing students with an opportunity to give feedback on their overall learning and teaching experience.  In addition, it could prove challenging for students to identify the contributions of different staff in the online environment.

Staff who DO wish to collect feedback on their teaching can see the section below on 'Other ways to collect feedback'.

How will OLEs be set up and distributed?

OLEs will be set up and distributed in the usual way using Blue, the platform that hosts SET.  Staff will be notified by email that their course/s will be evaluated, and students will receive automated invitation and reminder emails.

How is course eligibility for OLEs determined?

Eligibility for an OLE will be determined in the same way as SET, ie eligible courses are those with an in-scope Primary Graded Component (PGC) and at least 10 enrolled students.  Faculties may request courses to be excluded; the Academic Quality office will discuss requests with faculties on a case-by-case basis and can be contacted on quality@auckland.ac.nz

What are the objectives for the OLEs?

  • To provide students with an opportunity to give feedback about their learning experiences in the online environment
  • To obtain student perspectives on the effectiveness of learning and teaching in the online semester/quarter
  • To enable central collection of evaluation data which can be used by the centre and by academic leaders, as well as by staff teaching on courses, to reflect on course delivery and make improvements

When is the evaluation period?

The OLEs will open in the final week of the 'final assessment' (exam) period, and close after two weeks.  Students will be asked to complete the OLEs after they have completed their final assessments (if this is relevant for their course).  This timing has been agreed for this semester/quarter to allow students to provide feedback on this part of their remote learning experience.

Dates for OLEs

Semester One 2020 (1203): Monday 29 June (00:00) - Monday 13 July (23:59)
Quarter Two 2020 (1204): Monday 22 June (00:00) - Sunday 5 July (23:59)  
 

OLE questionnaire

The OLE questionnaire was discussed with Learning Design Service Staff, Associate Deans (Academic) and (Learning and Teaching), and students. Click here to view an image of the questionnaire.

The questions are:

  1. It was easy to find the information and resources I needed on the Canvas course website.
  2. The online learning environment provided me with opportunities to communicate and/or collaborate with my peers.
  3. The online learning environment allowed effective communication between teaching staff and students.
  4. I was clearly informed how my learning would be assessed.
  5. I received helpful feedback on my learning progress.
  6. Did you complete a time-restricted final-assessment for this course in the exam period? YES/NO___If NO: continue to Question 7___If YES:continue to 6a) My final, take-homeassessment allowed me to demonstrate my learning in this course. (5-point scale)  AND 6b) Did your final, take-home assessment allow you to demonstrate your learning in this course?  Please comment on this, or any other aspect of your assessment. (Open-ended comment)
  7. I felt part of a community of learners in this course.
  8. I felt I could stay motivated and engaged with my learning.
  9. Overall, I was satisfied with the learning experience in this course.
  10. Overall, what aspects of the course were most helpful for your learning? (Open-ended comment)
  11. Overall, what did you find the most challenging about learning online in this course?      (Open-ended comment)
  12. What changes could be made by the University that could address the challenges you have mentioned above?

What if my course usually uses Qualtrics and not SET to be evaluated?

For any course to be excluded from OLEs, you must have discussed this with your Academic Head and received approval from the Academic Quality Office.

For courses excluded from OLEs, staff can use an OLE template for setting up a Qualtrics survey to include the same questions, with the option to add their own questions.  Within Qualtrics, staff should select the  “Online Learning Evaluation (OLE) - course questionnaire” template. University of Auckland Qualtrics Library

Other ways to collect feedback

Formative feedback

Formative, in-course feedback is particularly important in the transition to the online environment.  Quick surveys can be used to gather this feedback allowing teachers to identify and correct any issues students have with the course or teaching in a timely manner.

Questions should be simple and may be open-ended.  Quick surveys or questionnaires can be done in a variety of ways:

  • A short online survey with your own questions (could be done as a Canvas Quiz, in Qualtrics, or using another online survey or feedback platform)
  • A minute paper (for teaching), eg what helped you to learn in this class and what didn't?
  • Traffic lights, eg I don't understand; I need help; I get it/do less of this; this is working OK; do more of this

How do I create a survey to get feedback on my teaching?

The Qualtrics survey tool is available to all staff who wish to collect formative and/or summative feedback on their teaching this semester or quarter. If you do not have an account, you can create one at https://auckland.au1.qualtrics.com

A number of teaching questionnaires have been added as exemplars that can be modified.  These can be found in the University of Auckland Qualtrics Library, under 'Academic Surveys'. (note: there is no requirement this semester to add your Associate Dean as a collaborator on the survey.) The University has created a Guide to Setting up a Qualtrics Survey.

Examples of formative feedback mechanisms

The Faculty of Business and Economics uses a '5-point fast forward feedback' and a '12-point fast forward feedback' questionnaire for teachers to gather feedback on their teaching.  These can be found in the Qualtrics library.

    Five-item fast feedback:

  • Do more of this...
  • Do less of this...
  • Start this...
  • Stop this....
  • If you have any other suggestions, please write them here...

    Twelve-item fast feedback:

Use the grading scale below to evaluate the course

1 - strongly disagree; 2 - disagree; 3 - neutral; 4 - agree; 5 - strongly agree

    1. The course teaches you what you need to know

    2. You know what you have to do on the course

    3. This course is well-organised

    4. The course is too hard/fast

    5. The course is too easy/slow

    6. This course is engaging and interesting

    7, 8, 9. If you could change three things about the course they would be:

Use the grading scale below to evaluate your performance.

0-20%; 21-40%; 41-60%; 61-80%; 81-100%

    10. Approximately how many of the taught sessions offered have you attended?

    11. Approximately how much of the set reading and out-of-class work have you done?

    12. How could you improve your performance on the course?    

Templates for these fast feedback questionnaires are available in the University of Auckland Qualtrics Library.

    Peer review

This is another mechanism that you can use to get feedback on your teaching and some faculties have established processes for organising peer review of teaching.  The University of Auckland has created resources on peer review of teaching, including templates, observation forms, and guidance on procedure.  These resources are on the CLeaR website.