One of the first steps to collectively trying to find new ways of constructing our school systems, I think, really is about changing the questions we think we are answering. Instead of using the type of questions needed to drive research, e.g. anything of the form ‘what works?’, we need to start asking, ‘how do we build systems that increase the likelihood that teachers will make intelligent and wise decision in their work?’
While medical-style guidelines may seem to have come from God, such guidelines, even in medicine are often multiple and contradictory. The “cookbook” teacher will always be chasing the latest guideline, disempowered by top-down interference in the classroom.
In medicine, over five years, fifty percent of guideline recommendations are overturned by new evidence. A comparable situation in education would create unimaginable turmoil for teachers.
We believe teachers should be heard more clearly in the conversations about evidence; policy makers and other decision-makers need to listen to teachers. The type of evidence that teachers want and can use should be basic to any plan around ‘evidence-based’ or ‘evidence-informed’ teaching in Australian schools.
Mathematical models are used everywhere in science and can even be turned inward to study mathematics itself. They are incredibly powerful tools that allow us to trade a problem we don’t fully understand for one we have a better handle on.
But using models is inherently tricky. We can never be certain that our model behaves enough like the thing we are actually trying to understand to draw conclusions about it. Nor can we be sure that our model is similar enough in the ways that really matter. So it can be hard to know that the evidence we collect from the model is truly evidence about the thing we want to know about.
So on a general level, the case for evidence-based practice has a definite value. But let’s not over-extend this general appeal, because we also have plenty of experience of seeing good research turn into zealous advocacy with dubious intent and consequence. The current over-extensions of the empirical appeal have led paradigmatic warriors to push the authority of their work well beyond its actual capacity to inform educational practice. Here, let me name two forms of this over-extension.
Simply ask ‘effect on what?’ and you have a clear idea of just how limited such meta-analyses actually are.
While in regards to RCT’s, he states:
By definition, RCTs cannot tell us what the effect of an innovation will be simply because that innovation has to already be in place to do an RCT at all. And to be firm on the methodology, we don’t need just one RCT per innovation, but several – so that meta-analyses can be conducted based on replication studies.
Another issue is that Research shows what has happened, not what will happen. This is not to say no to evidence, but a call to be sensible about what we think that we can learn from it.
What it can do is provide a solid basis of knowledge for teachers to know and use in their own professional judgements about what is the best thing to do with their students on any given day. It might help convince schools and teachers to give up on historical practices and debates we are pretty confident won’t work. But what will work depends entirely on the innovation, professional judgement and, as Paul Brock once put it, nous of all educators.
A multiplicity of research approaches provides diverse ways of understanding education, but we need to interrogate the approaches and arrive at conclusions with caution. Teachers’ wisdom of practice and immersion in their own contexts needs to be honoured.
The ALP’s pledge to fund an ‘Evidence Institute for Schools’ lacks attention to what is needed most—funding for schools and classrooms. Further, the effectiveness of this large sum of funding spent on an institute is premised on the notion that it will produce significantly more effective research than is already available.
Here’s what could be done
- investigating more efficient ways to encourage the uptake of educational research in our schools and universities
- improve overall accessibility of education research to the public