So, what kind of ‘evidence’ is being referred to by the former Minister when he rightly insists we need to ensure that pedagogy is evidence-based’. Is he referring to evidence derived from primary research, such as randomized control trials (RCTs) and observational studies; or secondary research, including systematic reviews of the research literature? The fact is there is no single type of evidence. It is generally recognised that different evidence types have different methodological strengths. At the pinnacle of the ‘hierarchy of evidence’, are systematic reviews, followed by RCTs, cohort studies and then case-controlled studies, case reports and finally expert opinion. Without identifying the type of evidence to which he refers, the former Minister, appears to resort to lay-opinion disguised as evidence.
Tag: Evidence
Too often “evidence-based” policy has involved limiting rather than broadening alternatives, privileging particular forms of evidence over others, and narrowing consultative processes.
It is more about whose evidence is valued, and for what underlying purpose, than employing an “evidence-based” approach to policy making.
Don’t get me wrong: it’s essential that policy and practice are based on solid evidence. But too often only half the picture is revealed. Either evidence is sought to justify an existing preferred position, or the complexities of teaching and learning are glossed over in favour of an “evidence-based” silver bullet.
In my own workshops, I’ve certainly been guilty of focusing on communication skills at the expense of strategy and not fully addressing the flawed deficit model. But I’m learning to better challenge scientists’ assumptions about how communication works. The deficit model, I’ve found, is difficult to unlearn. It’s very logical, and my hunch is that it comes naturally to scientists because most have largely spent their lives in school—whether as students, professors, or mentors—and the deficit model perfectly explains how a scientist learns science. But the obstacles faced by science communicators are not epistemological but cultural. The skills required are not those of a university lecturer but a rhetorician.
One of the first steps to collectively trying to find new ways of constructing our school systems, I think, really is about changing the questions we think we are answering. Instead of using the type of questions needed to drive research, e.g. anything of the form ‘what works?’, we need to start asking, ‘how do we build systems that increase the likelihood that teachers will make intelligent and wise decision in their work?’
While medical-style guidelines may seem to have come from God, such guidelines, even in medicine are often multiple and contradictory. The “cookbook” teacher will always be chasing the latest guideline, disempowered by top-down interference in the classroom.
In medicine, over five years, fifty percent of guideline recommendations are overturned by new evidence. A comparable situation in education would create unimaginable turmoil for teachers.
A summary of a paper ‘A broken paradigm? What education needs to learn from evidence-based medicine’ by Lucinda McKnight and Andy Morgan
We believe teachers should be heard more clearly in the conversations about evidence; policy makers and other decision-makers need to listen to teachers. The type of evidence that teachers want and can use should be basic to any plan around ‘evidence-based’ or ‘evidence-informed’ teaching in Australian schools.
Meghan Stacey and Nicole Mockler share some of their finding associated with what evidence teachers value in the classroom. This is in contrast to external meta research.
Mathematical models are used everywhere in science and can even be turned inward to study mathematics itself. They are incredibly powerful tools that allow us to trade a problem we don’t fully understand for one we have a better handle on.
But using models is inherently tricky. We can never be certain that our model behaves enough like the thing we are actually trying to understand to draw conclusions about it. Nor can we be sure that our model is similar enough in the ways that really matter. So it can be hard to know that the evidence we collect from the model is truly evidence about the thing we want to know about.
So on a general level, the case for evidence-based practice has a definite value. But let’s not over-extend this general appeal, because we also have plenty of experience of seeing good research turn into zealous advocacy with dubious intent and consequence. The current over-extensions of the empirical appeal have led paradigmatic warriors to push the authority of their work well beyond its actual capacity to inform educational practice. Here, let me name two forms of this over-extension.
James Ladwig unpacks evidence-based approaches. In response to ‘synthetic reviews’, he suggests:
Simply ask ‘effect on what?’ and you have a clear idea of just how limited such meta-analyses actually are.
While in regards to RCT’s, he states:
By definition, RCTs cannot tell us what the effect of an innovation will be simply because that innovation has to already be in place to do an RCT at all. And to be firm on the methodology, we don’t need just one RCT per innovation, but several – so that meta-analyses can be conducted based on replication studies.
Another issue is that Research shows what has happened, not what will happen. This is not to say no to evidence, but a call to be sensible about what we think that we can learn from it.
What it can do is provide a solid basis of knowledge for teachers to know and use in their own professional judgements about what is the best thing to do with their students on any given day. It might help convince schools and teachers to give up on historical practices and debates we are pretty confident won’t work. But what will work depends entirely on the innovation, professional judgement and, as Paul Brock once put it, nous of all educators.
This touches on Mark Esner’s argument that great teacher will make anything work to a degree. Also, Deborah Netolicky’s observations about evidence.
The reasons why Australian students underperform has little, if anything, to do with funding. Compared to stronger performing education systems, Australian classrooms have been forced to adopt a dumbed down, overcrowded curriculum that lacks academic rigour.
I find this all problematic Kevin. I feel that deciding that the supposed ‘child-led’ approach is at fault is no different to me questioning the ‘effect on what’ of the evidence-based approach. I wonder if what is really at fault is teacher agency. We need ‘evidence’, but surely this needs to be informed by context as well. I think that Dr. Deborah Netolicky captures this best when she says:
A multiplicity of research approaches provides diverse ways of understanding education, but we need to interrogate the approaches and arrive at conclusions with caution. Teachers’ wisdom of practice and immersion in their own contexts needs to be honoured.
The ALP’s pledge to fund an ‘Evidence Institute for Schools’ lacks attention to what is needed most—funding for schools and classrooms. Further, the effectiveness of this large sum of funding spent on an institute is premised on the notion that it will produce significantly more effective research than is already available.
Here’s what could be done
Emma Rowe and Trevor Gale suggest that rather than spending money on a new institute, the government should instead:
- investigating more efficient ways to encourage the uptake of educational research in our schools and universities
- improve overall accessibility of education research to the public