Re-designing data discussions to improve teaching and learning

A research project hopes to improve the way data is discussed in classrooms by creating a new framework that builds on teacher preferences, rather than rejects them.

The project, a joint investigation between the University of Auckland and the Norwegian University of Science and Technology, aims to support school leaders and teachers in analysing data about student learning in a way that supports and motivates teachers to use these data to improve their teaching practices.

Associate Professor Mei Lai, University of Auckland researcher, says that it’s common to hear of data discussions taking place that don’t lead to improvements in teaching practices, or improvements in student outcomes.

“The question then became, why is it that some discussions of data appear to lead more to improving outcomes and some don't? The only way to understand that is to really understand the discussions themselves in-depth.”

In order to focus on depth rather than breadth, the project trial took place in only three schools in order to gather data from the widest range of perspectives possible.

“In my previous work, when we used data discussions as part of interventions to improve outcomes for students, theywere with entire schools typically across three years with anything between seven to 33 schools. You can’t get a depth of understanding about what is happening in these discussions and why people think they’re working. So, I wanted to do a study that would go deep, not wide.”

Researchers worked with school leaders and teachers to co-design, test and evaluate the new approach which focused on a small group of students in a class. A broad range of specific, personal, and relevant student data (e.g., test data, teacher observations) were used to determine students’ strengths and needs, which in turn was used to develop more effective and targeted teaching practices.

Samples of student work such as books they were reading and struggling to understand were incorporated into the discussion to allow for greater depth of diagnosis and offer a holistic view of student learning in context.

The project allowed teachers to discuss strategies for improvement during the data discussions and leave with user-friendly strategies and concrete practices that they could implement immediately.

By the end of the initial trial period, all teachers strongly recommended that their schools implemented something similar (9 of out 10 on the rating scale with 10 being the highest rating) and rated the approach as very useful for themselves.

Almost all rated the approach as being useful for all teachers, regardless of experience, knowledge in teaching reading and preference for using qualitative or quantitative data – a successful trial, Mei says.

“We also had a professor of literacy who examined all the transcripts of the meetings to check whether teachers had actually understood the strengths and weaknesses of their students, and whether they were recommending good practices. He reported that their understanding of the data was entirely appropriate and, in some instances, really sophisticated.”

In future, Mei would like to focus on testing and extending the findings with a larger sample and different types of schools. This would also involve testing the impact of their approach on student learning and achievement.

The study’s co-investigator, Henning Fjørtoft, plans to trial the approach in Norway, with tweaks to fit the local context.

“There are similarities between Norway and New Zealand in that schools have local autonomy to develop their own curriculum and that teachers are trusted to make informed assessments about student learning.

"The model developed in the NZ context is therefore likely to be useful in a Norwegian context. However, this would require some adaptation to the Norwegian curriculum and assessment system.

"For example, Norwegian teachers do not have access to a broad repertoire of assessment resources, so they rely more on close observation in the classroom. Understanding how teachers outside New Zealand discuss strategies for improvement would enrich the research results.”