Acknowledgements
Collection deals with NAPLAN in Australia, but our introductory and concluding chapters seek to situate the research reported here in a broader global context, aware of the circulation today of globalised education policy discourses and the significance of international testing as a complement to national testing such as NAPLAN.
Page 23
1 National testing from an Australian perspective
Unlike other national testing regimes such as the National Assessment of Educational Progress (NAEP) in the US or the Pan-Canadian Assessment Program (PCAP), NAPLAN is a census test, not a sample test.
Page 29
NAPLAN data are thus used for a variety of purposes, including governing school systems, accountability purposes, managing staff within systems and schools, and making educational decisions regarding curriculum and pedagogy in systems, schools and classrooms.
Page 30
Together, NAPLAN, MySchool and the raft of programs and contractual arrangements between governments and schools that reference testing data illustrate the pervasiveness of technocratic rationality in Australian schooling
Page 31
NAPLAN was established to improve teaching and learning outcomes, but one significant effect has been that much teaching is now aimed at improving NAPLAN scores.
Page 32
NAPLAN data were useful in providing a common language for communication between principals, teachers and parents about student progress and achievement.
Page 39
2 What national testing data can tell us
In summary, we would say that a NAPLAN test only provides an indicative level of the performance of a student: whether the student is struggling, on track, or performing above average. The NAPLAN tests do not provide fine grading of students by their performance levels because of the large uncertainties associated with the ability measures.
Page 58
If teachers do not change the way they teach, the school mean scores for a year level can vary within a range of 32 NAPLAN points for 90% of the time if we have the opportunity to repeatedly allocate random samples of potential students to this school. Compare this margin of error with the expected annual growth rates of 44 points at Year 3, 28 points at Year 5, and 21 points at Year 7; the fluctuation in school mean scores due to a particular cohort of students has a magnitude close to one year of growth. This means that for many schools with a year level size of 50 or fewer, the average school performance could change significantly from one calendar year to another.
Page 60
We need to always remember that using student assessment data to evaluate teachers is making an inference, since we have not directly measured teacher performance. The validity of making this inference needs to be checked in every case.
Page 62
One should never jump to conclusions of ineffective schools whenever NAPLAN results are low. NAPLAN results indicate where further investigations are warranted.
Page 64
As teacher effect accounts for only a small portion of the student achievement variance, individual teacher effect is likely to be swamped by the large variations in student abilities in a class. This is a reliability issue.
Page 65
In conclusion, national testing data can inform us about performances of large groups of students, but not tell us a great deal about individual students or schools. National testing data cannot provide teacher performance measures, so there should not be any link between student test results and teacher appraisal or pay. National testing data have the potential to inform teaching and learning, and to frame education policies. However, we need to ensure that evidence-based decision making is backed by sound data and valid inferences.
Page 66
3 The performative politics of NAPLAN and MySchool
Focusing on NAPLAN and MySchool as interesting objects – as actors in their own right, rather than as effects or products of neoliberal governance strategies – provides the opportunity to explore the technologies and mechanisms through which such objects serve to delegate trust, create new intimacies and reorganise relations.
Page 70
By providing access to much more detail about each school, it brought parents closer to knowing their child’s school. It also revealed to schools themselves information that they previously did not have about themselves and about other schools.
Page 71
Here I take NAPLAN and MySchool to be calculative objects – objects that resulted from policy decisions, to be sure, but which also became participants in the policy arena, actively rearranging the goals of schools, parents, teachers and policy makers and bringing to the forefront new issues and problems. I present four specific features or functions of interesting objects: creating new intimates, translating interests, displacing trust and creating informed publics.
Page 72
Not only did MySchool become a technology through which the government entered intimate spaces of schools, schools themselves entered intimate spaces of living rooms and kitchens through discussions between parents
Page 72
By involving parents in the job of keeping schools accountable and in continually improving their performance, parents and the government were cast as intimates – partners in the shared enterprise of school improvement.
Page 74
By inserting itself between parents and their child’s school, MySchool attempted to enrol parents as canny stakeholders, casting the schools as secretive actors who were reluctantly being forced to reveal information they would rather have kept to themselves
Page 74
NAPLAN and MySchool thus changed the original goals, motivations and plans of various actors
Page 74
NAPLAN and MySchool thus created relations of distrust and suspicion between schools and the government, as well as schools and the public. They displaced trust from local actors with immediate knowledge and delegated trust instead to distant and impersonal actors.
Page 75
NAPLAN and MySchool produce an abstract, impoverished and interested version of the very complex phenomenon of schooling in Australia. However, these interested observations of NAPLAN and MySchool are not merely providing useful, detailed accounts of Australian schooling; rather, they are actually changing the very nature of Australian schooling, so that it is beginning to more closely resemble the abstract version presented on the MySchool website. Rather than NAPLAN and MySchool reflecting an abstract version of Australian schooling, they are perhaps remaking Australian schooling in their image.
Page 78
4 Questioning the validity of the multiple uses of NAPLAN data
As Strathern (1997) states: ‘When a measure becomes a target it ceases to be a good measure’ (308).
Page 83
In the context of NAPLAN, while the tests may measure attainment in numeracy or literacy, it is questionable whether the information from these tests can be used validly for explaining how well the school has performed. Yet the aggregation of test scores across students to provide composite measures of educational effectiveness for teachers, schools, states or even the nation are commonly used in education for accountability purposes.
Page 83
5 Local experiences, global similarities: teacher perceptions of the impacts of national testing
What policymakers intend is always mediated by how policy ‘hits the ground’, or is enacted, by individuals in diverse, complex community and institutional settings.
Page 91
It must be stressed that NAPLAN is designed to change practice and behaviour through the emphasis on test-based accountabilities. However, not all change is desirable
Page 98
The most dangerous possibility of testing data is that it distorts and corrupts the very processes it intends to measure. As education policy makers seem intent on continuing to use test data to steer practice from a distance, it remains to be seen how this distortion can be prevented.
Page 99
6 NAPLAN and student wellbeing: teacher perceptions of the impact of NAPLAN on students
In the case of schools, the use of NAPLAN results as a blunt accountability instrument through their publication on the MySchool website has significantly increased the pressure on schools to treat NAPLAN results as more than just a snapshot of student achievement at a particular point in time
Page 107
First, rather than NAPLAN itself being the central issue of concern in this instance, it is the use of NAPLAN results in largely inappropriate ways that is likely to be generating serious negative consequences
Page 108
Second, these types of findings, and the likely reasons behind them, suggest a serious lack of knowledge amongst some policy makers, bureaucrats, principals, teachers and parents about the limitations of NAPLAN results (and indeed, any single test score)
Page 108
Overall, it seems evident that the NAPLAN program is generating stress-related responses amongst substantial numbers of students across Australia. While there is a need for further research to elucidate the reasons behind this, it is highly likely that the use of NAPLAN results in inappropriate ways is contributing to student stress through the messages sent to students in the words and actions of principals, teachers and parents. Blaming these groups is not the way forward – rather, the time has come to discuss the relevance of NAPLAN, whether the benefits are worth the substantial costs (including psychological), and if NAPLAN is to continue, what the appropriate, statistically defensible and reasonable use of student test results might look like.
Page 108
7 Literacy leadership and accountability practices: holding onto ethics in ways that count
The common agreement for literacy is a school-based policy, collaboratively developed between teachers and leaders, that prescribes what should be included in the daily uninterrupted literacy block. The block includes: guided reading, Jolly Phonics (Reception – Year 2), explicit teaching of comprehension strategies, daily reading practice (Choosing to Read), shared reading, handwriting, writing, spelling program, grammar and punctuation, as well as the locally mandated assessments to be undertaken over the year and the SMARTA (Specific/Student focused; Measurable; Achievable; Relevant; Time-lined; Agreed) targets for reading endorsed by the region. All teachers are given copies of the literacy agreement in their induction folders at the beginning of the school year and they were posted prominently on the notice board in the staff room.A locally generated text, the literacy agreement has come into existence as a result of very low NAPLAN results. It not only reflects the programs that teachers considered to be valuable, but the shaping force of NAPLAN. In this way, NAPLAN regulates the school’s common literacy agreement, constitutes the literacy problem and coordinates everyday classroom work in more or less obvious ways. For instance, the literacy component of NAPLAN includes a reading comprehension test, a writing test (genre writing), a spelling test and a grammar and punctuation test
Page 118
As we have seen, Sandford has engaged with the unavoidable accountability requirements associated with NAPLAN. We have shown the extent to which NAPLAN has evoked a narrow view of literacy as the practice of content-free skills and how this view is reproduced in the active and occurring text of the literacy agreement that shapes what happens in classrooms. Nevertheless, NAPLAN does not always dominate what can be said. The potential sedimentation of NAPLAN is unraveled and reworked, at least to some degree, in the literacy chats, a product of the school’s recognition of the teachers’ needs for professional mentoring conversations that take account of actual students and their learning trajectories. In these educative and dialogical spaces, the senior leader works with teachers to design pedagogical interventions for students whose progress in school literacy learning is cause for concern. However, it is not only a question of looking at data as an artefact of the student, as the excerpt of Carrie’s literacy chat indicates. In mediating translocal policies that might otherwise close down possibilities for engaging ethically with students, the senior leader offers teachers the possibility of creative and critical literacy pedagogies. Despite their value in turning teachers around to students’ knowledge and practices as resources for school literacy learning, such pedagogies are less and less visible in schools since the advent of NAPLAN.
Page 125
8 Contesting and capitalising on NAPLAN
… a warm-up session to ensure students were ready to learn;an ‘I do’ session in which the teacher demonstrated the specific task which was the focus of the lesson;a ‘we do’ session in which teachers worked with students as a whole class to co-construct a model response;a ‘you do’ activity involving students working independently;and a ‘ploughing back’ session in which students revised the lesson objectives and outcomes
Page 137
9 Understanding the politics of categories in reporting national test results
Strong average performance in numeracy by some LBOTE students is not simply ascribed to a cultural fixation on academic attainment but may be a reflection of numeracy skills attained through comprehensive educational backgrounds;this strong average performance clouds the heterogeneity of the LBOTE category;LBOTE classification encompasses a broad heterogeneous group of students, which in the absence of a measure of English language proficiency, is most evident when NAPLAN results are disaggregated according to visa status of LBOTE students. Visa, in turn, is informative about disadvantage related to prior educational opportunities because students of refugee background are performing far below those of other migration categories, particularly the skilled visa category;language proficiency levels and years of schooling are associated with NAPLAN outcomes; andstudents who are of refugee background, with reduced years of schooling, and in the early stages of acquiring English are most disadvantaged in NAPLAN test results, but are completely hidden in the LBOTE category.
Page 165
NAPLAN data need to be interpreted and understood within the context of language learning, whereas, in its current form, the breadth of LBOTE can only render a shallow interpretation, which dangerously ignores understandings about academic second language development.
Page 167
10 Students at risk and NAPLAN: the collateral damage
Evident in the above is how, over the years of NAPLAN administration, support for students with different needs – social and emotional, language background, learning difficulties – to participate in NAPLAN has narrowed to serve the priority of administrative consistency.
Page 173
NAPLAN data were reported to have little utility compared to information already obtained: [NAPLAN] does not provide us with any information about students that we don’t already know ourselves. We profile our students. And it just gives us another piece of information that we would otherwise have anyway.(Principal, independent PY–12 school)
Page 174
…teachers reported positive value from NAPLAN as confirming their own professional judgements
Page 175
16 The life of data: evolving national testing
Following Simons (2014), international and national tests can be seen to function as global/national positioning devices, evidence of a new spatial disposition and, in Australia, evidence of the emergence of a numbers-based national system of schooling. While these developments provide some evidence of a world polity approach that talks about the global diffusion of modernity and also the global dissemination of a particular version of science and social science, they also reflect the global impacts of an Anglo-American model of school reform based very much on test-based, top-down modes of educational accountability.
Page 284
There is a common perception that testing data are inert, lifeless objects that provide an unbiased and objective measure of educational process, practices and outcomes.
Page 286
However, we must be careful in making this claim that there is a life of data. In its most extreme form, this can lead to positing data as an agentive actor that makes decisions and behaves in certain ways. This is clearly not the case – data are expressions of human subjectivity, an expression of the values, sensibilities, processes that lead to their creation, and then the paths that the data lay down for individuals in terms of their choices, actions and acts of enunciation. Data are thus part of new spaces of subjectivity that are not contained within human bodies, but instead extend into information systems such as testing regimes, but also other data-driven applications such as social media or mobile phone usage. To understand the life of data, then, is to recognise that data produce possibilities and are invoked through the behaviours and values that result from the production of data. We cannot see data as external to the production of subjectivity, rather as Guattari (1992) argues, there is a little piece of human subjectivity in each data point: the technologies that we use to engage with data ‘are nothing more than hyperdeveloped and hyperconcentrated forms of certain aspects of human subjectivity’ (18).
Page 286
Data have a life, they are always and everywhere put to work, they are always and everywhere in motion. One demonstration of this principle was highlighted by Nichols and Berliner (2007). Their argument was that the higher the stakes attached to any single measure that is used to make important decisions about students, teachers and schools, the more liable it is that the initial measure becomes corrupted because the processes are distorted by the emphasis. This is called ‘Campbell’s Law’, which stipulates: …the more any quantitative social indicator is used in social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to measure.(Nichols & Berliner 2007: 27)For example, tests like NAPLAN, which are designed to measure student achievement in the constructs of basic literacy and numeracy skills, become corrupted when teachers devote excessive class time to preparing for the tests. In other words, the tests no longer measure constructs regarding literacy and numeracy, rather they begin to measure the construct of how well a teacher can prepare a class. Obviously this is a problem, if important decisions are being made about literacy and numeracy on data that do not measure what they purport to measure, such decisions may not drive the improvements that were intended.
Page 288
If data have lives, they are enacted through the space and time of data, and notions like consequential validity advanced by test developers themselves speak to this life
Page 288
The critical question then is ‘what ought to be the future orientation to data at all levels of schooling’? This is primarily a political question and it needs to trouble the thinking and work of politicians, policy makers, system leaders, principals, teachers, students, the broader community and also educational researchers. I
Page 291
Given this, we are not opposed to national testing, but we do believe that our assessments of national testing clearly point to areas where action must be taken to reduce its negative effects in Australia and elsewhere.
Page 291
Mentions
Likes