[Skip to content]

NAHT - For Leaders, For Learners
Search our Site
Opinion icon

Warwick Mansell

The former TES journalist writes for NAHT on current education issues. The views expressed do not necessarily reflect those of NAHT

 



A great start to school life, and my reflections on test-based reductionism

Our daughter has had a lovely start to her time at school. In fact, “fantastic” pretty much describes our family’s experience so far, with her half-way through her time in reception.

I’ve been thinking – as I am prone to do – about the implications of this from a system/policy perspective. But probably the most important thing to do with this blog is simply to tell the happy story of what has happened so far.

Our positive experiences of the school kicked in more or less from the start. There was an introductory evening for parents last summer when we noticed a couple of things: how well-oiled the school’s administrative machine seemed to be in terms of getting the preparation right; and the look of quiet amusement on the face of our daughter’s future teacher as she took us through the little things that needed consideration in the management of reception children: from what they needed to wear with getting to the toilet quickly in mind to keeping them enthused about reading and storytelling.

That impression of the school as an impressive place has just solidified as the months have gone by. The settling-in process in September, carried out over a week or so, was very well-organised and smooth. The school had a well-established drill for pick-up in the playground that seemed, again, very well-practised. Our daughter has enjoyed the various enriching parts of school life that seem so much fun: the Christmas production, the regular “forest school” trips to the local park, the teaching topic on dinosaurs which included a search for pterodactyl eggs…She tells us she loves her teacher and all the teaching assistants, too.

In fact, particularly memorably during last year’s Indian summer of September and October, taking daughter to the playground in the sunshine after school and chatting to the parents of her fast-accumulating number of friends, I felt very lucky indeed to be part of a new community which offers her and we expect her younger brother the hope of those precious qualities of early life: a feeling of happiness, stability and care.

Daughter’s teacher seems universally respected as excellent. I am just an outside observer on this – I haven’t seen any lessons – but there appear to be two aspects to it, which I guess may tally with everyone’s experience of a good teacher. She seems to have that sparkle in the eye and sense of fun, communicating an understanding that four- and five-year-olds have their inherent comic side and I think going along with their playfulness and encouraging their imaginative play as much as possible. But she’s also - it seems to me and others - in control at all times. As a parent with now nearly five years’ experience of trying to strike the right balance, I think that’s quite a combination.

Other parents speak of how she and the teaching assistants work very effectively as a team, and I’ve never seen anyone flustered.

What strikes me, too, in comparing our experience with that of my own primary schooling in the 1970s, aside from the welcome availability of extra adults in addition to the teacher in the classroom, is the level of communication with parents. This is on a completely different scale from 40 years ago as I remember it. We are kept constantly informed of what is going on, and invited in for events keeping us in the loop about how the children are being taught to read and write; I even attended one on the government’s new assessment system last term and resisted the urge to make a speech…

At the last of these events, last month, I was left reflecting again just how much thought goes into the business of helping the children make progress with their learning, as the school discussed details such as when or if to correct children when they make mistakes, the merits of teaching joined-up or not-joined-up writing and how to introduce “sight words” which are phonetically irregular. What came across was a sense of detailed knowledge and professionalism, with the school sometimes having adapted practice in light of recent experience, tweaking policy to improve it where necessary. If our one worry is slightly that, in recent weeks, our four-year-old has talked occasionally about finding it difficult to sit still for “learning”, which she says she finds tiring, the school is reassuring and seems to take seriously not reducing the emphasis on play too quickly.

I wanted to write this piece in large part to place on the record that positive experience. And indeed, if you’d rather read about a positive experience of our education system than about more criticism from me about the far more problematic policy frameworks against which professionals have to operate, stop here.

The policy bit – less positive – starts here

Our experience also makes me reflect, when I hear and write about schools with very high turnovers of staff, on what may be being lost in terms of the benefits of experience. (Note 1)  And the trauma of schools having an outside provider imposed on them by remote ministers or civil servants, about which I’ve also written an awful amount in recent years, seems frightening: schools such as this serve and are central to communities. They are surely strongest when, as here, those communities feel they have a stake in the institution.

But, given the themes of this blog and my obsessions as a writer, it will come as no surprise that observing these details close-up has also left me thinking again about our education system’s use of assessment results to try to reach a judgement on “the truth” of school quality.

Happily, this is still largely a theoretical debate in terms of our own experience so far of the school. But, of course, I still think it’s important.

For the job of pronouncing on school quality is a task which assessment results alone cannot, in reality, perform. And seeking to use them in this way has, I think, sold all our schools short.

“Reductionism”, I think, is the key word here: did anyone really ever think that complex organisations such as schools could be reduced to a few numbers without some costs along the way?

Assessment results are, of course, key to how schools get judged: they are central to Ofsted inspection judgements, and can inform Ofsted’s calculations as to when or if schools get visited. They are now central to questions as to whether schools should be subject to takeovers by outside organisations. And their place in other measures such as league tables and often as elements of teachers’ performance pay is well-known.

But what can a result really tell me about this school’s quality? Really, beyond perhaps extremes of “performance” one way or the other, very little. Expecting a few results for each child in a couple of subjects at age 11 to provide as much information about what kind of school this is as we get as parents from the hundreds, if not thousands, of interactions we will have with it over a seven-year period, is ludicrous. Yet that seems to be what our system embraces, with test results the key to the evaluation of school quality.

Some people argue that we need league table data to help parents in their choice of school. But I would respond that data really pales behind, say, testimony from parents who already have children at a school, because they have day-to-day experience of interacting with the school which can never be captured by a simple exam result spreadsheet. (Note 2).

And I speak as someone who spends a lot of time manipulating education data spreadsheets: I didn’t even look at league table information on this school before opting for it as I thought it would have very little to tell me. At best, the data might be there in the background as part of understanding what’s going on in a school. But that’s about it.

Data has its uses…

As an education journalist, I know that data are powerful for many reasons. Among them is that statistics allow anyone trying to sum up the quality of institutions – or of the system as a whole – very succinctly. Data spreadsheets are, then, basically a simplification system, reducing what is going on in many diverse institutions into a few numbers. Instead of, for example, us trying to have qualitative discussions as to what is going on in thousands of institutions, we can capture some “outcome” numbers for each of them, on dimensions of education which, while not being seen to sum up the entire educational experience of pupils, nevertheless seem important.

We can then work out averages for the system as a whole on these indicators, and work out how these compare to previous years’ averages, and whether individual institutions are going up or down. The ability of this system to simplify what would otherwise be a very complex exercise – to allow the summary of information related to thousands of institutions, in potentially, a single number - is profoundly useful to anyone trying to, for example, write about it. We can then think we can have a reasonably-informed discussion about notions such as overall educational quality, without ever having to visit the schools or talk to their users about their experiences. (Note 3)

Data can serve a variety of other functions. As I survey the accounts of academy chains, I’m not the first to wonder if the test and exam results listed within them operate as something akin to factory production statistics, to be audited every year to see if production has improved or slowed down, even though education is a far more complex process than industrial production and the securing of good grades something over which any institution has only – or should have only, unless it is cheating - limited control. I suspect that England’s giddyingly-fast development of a quasi-private model of school takeovers, written about a fair amount here, depends on the reduction of institutions to a few statistics, with those not doing well on the chosen metrics ripe for absorption by an incoming organisation. 

Data also make, for example, Ofsted inspection judgements easier to defend in the face of challenge: a metric – any metric – is probably easier to back in the face of external scrutiny than is qualitative judgement.

And, let us not forget this in the age of austerity, judging schools by statistics is cheap. I had an interesting discussion with a former senior inspector at a conference a few weeks ago. He said he left Ofsted 10 years ago as the inspectorate sought to reduce the length of school inspections, in the face of large budget reductions since the late 1990s, which I think have continued. Ofsted does not now spend enough time in schools to find out about the detailed interactions with pupils and parents such as the ones I’ve described, so that its verdicts are based on a few snapshot observations, with a consequent reliance on data. (Note 4)  I continue to think that reducing spending on inspection is a false economy, in an environment in which so much weight is placed on inspectors’ verdicts: if we want to make school accountability so important to what goes on in the classroom, we need to invest in doing it properly, which for me means qualitative investigation, more detailed reporting and engagement with the users of a school to a level far beyond what currently happens. I think that this exercise would emphasise teachers’ professionalism, though I may be in the minority of readers of this blog in advocating longer inspections.

The data drive can and is defended, of course, on the grounds that what is assessed is important and that we cannot lose sight of what matters in the end about what goes on in education: the understanding of important subjects with which children leave their schools, as measured by assessment. This argument would say that the experiences I’ve described above in relation to our daughter’s school are fine as far as they go, but they are just background to the production of good results. In other words, what matters in the long run, to the children themselves, are their “outcomes”.

But the data drive has limitations…

Yet there are several arguments against our continuing drive to sum up what happens in education according to a few indicators:

-I think it is one means by which the national conversation about what goes on in schools has been reduced. The professionalism I’ve observed, and the qualitative changes in education which have taken place in recent decades, tend not to get discussed as debate centres around those more-easily-obtained assessment production figures. And, because such figures are always reducible to a headline which says “x per cent of pupils can’t do this” (or, if that’s a push, perhaps “y (rather high) percentage can do this, so the measure must be wrong”), then questions of whether parents are getting good experiences along the way tend to get sidelined. Schools can always be compared to others/the average so many can be made to feel as if they are “underperforming”, even if in reality they are part of a national body of schools which in general is providing good, life-enhancing experiences for pupils, day in, day out.

It is true that in recent years Ofsted has been stressing the largely positive experience that children are receiving in state primary schools. But many of the details get missed in a conversation which is reduced to “outcome” numbers. I also found myself wondering, on reading this piece (http://bit.ly/1WGfHrs), if the focus on numbers might itself in some cases be continuing to undermine the ability of individuals to act as professionals, in extreme cases leaving them just to act as if following a script.

- While I understand the importance of the concept of an “end product” for schools, there is something tragic about what may be getting lost along the way.

There are a couple of things to unpick, here. First, let’s assume that what are seen as the publicly-recorded “outcomes” of, in this case, primary education, were indeed all that mattered in terms of eventual goals for young people. That is, we could sum up all that was important in terms of their future life experiences in a few numbers centring on how they had progressed in English and maths.

Even accepting what is clearly a contestable statement on those terms, there would still be a case that education is not just about the end product, but about what happens along the way. The Cambridge Primary Review, for example, I think, argued that primary education should not just be seen as a preparation for the next phase, but of importance of itself.

That seems right, from my new perspective as a parent. Of course I want our children to have gained a good understanding of what is taught by the time they leave the school. But this time is also a very special one in its own right: the experiences they have will stay with them for the rest of their lives, and they are important. In other words, our jobs as parents is to prepare our children for hopefully long lives as adults; but we also need to value this time in its own right.

Sometimes it feels as if the rush to sum up everything about education only through inevitably quite superficial, and inevitably error-prone (note 5) “outcome” measures as to what has happened in tests at age 11 is akin to someone racing through the pages of a book, only to judge it on the contents of the final page. It seems to miss out a lot of what has gone on before.

If I get to look back with my children on their primary education, and ask them what was important, I expect we will agree that the notion that it could be summed up in a few results indicators in English and maths would be shown up for what it is: ridiculous. It will hopefully have been about far more than that.

-For the reality, of course, is that the word “outcome” is probably misleading. At the same conference where I met the inspector, one of the speakers said we shouldn’t talk about assessment results as “outcomes” but as “outputs”: statistics produced by a system, but which are not actually reflective of what are the final achievements of pupils.

The speaker, I think, argued that “outcomes” should be seen as something even longer-term: more in terms of, say, eventual work or higher education destinations for young people. Under this argument, then, assessment results would not be seen as final “outcomes”, but more as staging posts.

I would go further: the desired “outcomes” from this phase of education should not be seen as a set of test results but as primary education laying the foundations for a life lived well, which includes the development of a love of, and engagement with, learning; persistence; an ability to work with others; and so on. If we must, then, see education as important only in terms of foundations which are laid, let it be those foundations, rather than the by-product, which is short-term assessment results. And if we are to obsess about test results, at the very least let’s call them what they are, rather than refer to them more vaguely as “outcomes”.

To put it another way, if a child was to achieve good test results – perhaps through excessive test preparation - but at the expense of being turned off learning in the longer term, it would be counter-productive.

Where does this leave us? I realise that England has gone a long way down the road of summing up institutional quality in a few results, and that it would be difficult to unpick, even if policymakers chose to. I realise, too, that a school which concentrated exclusively on what could be seen as the “inputs” of good teaching without thinking about whether such good intentions were translating into children actually making progress with their understanding – you know: knowing stuff – would be getting it wrong.

But our experience of the first six months of education has underscored a sense of what the system as a whole may be losing in the clearly reductionist drive to judge everything by a few measures of “outcomes”, which aren’t really “outcomes”. We are, of course, a long way from changing that.

A more hopeful view is that what our experience may underscore is schools’ continuing ability to offer a great experience for pupils, in spite of the policy trends and seemingly unending dysfunction of England’s policy regime. In other words, professionals can still thrive within an overall policy framework which seems not to value professionalism as it could. While there are blots on the horizon for us in terms, as ever, of the possible impact in-school of government policy– notably in terms of the Department for Education’s largely untested (note 6)“tougher” new curriculum, the chaos around assessment policy and how all this will impact on pupils as well as teachers – the experience so far has been great.

If readers take nothing away from this blog, the last seven words in the paragraph above should do it.

Note 1: A friend of a friend talks of how a well-known academy chain is getting through large numbers of teaching staff, and how seeing teachers leave was upsetting for her daughter before she moved school as a result. I can identify with this, as a parent: my hunch is that four-year-olds need to feel secure to learn well, though I cannot think of any direct academic evidence on the subject. Certainly, my wife makes the point that our daughter would be really thrown if her beloved class teacher were just to disappear overnight, as we have heard has been known to happen elsewhere. This recent article raises concerns about happiness among children here, with the academic arguing there has been too much of a focus on attainment and not enough on social relationships: http://bit.ly/1PChETY.

Note 2: I think my fellow journalist Laura McInerney is brilliant and I also think Schools Week, which she edits, has been an amazing addition to our education news scene in the last couple of years. I Unlike Laura, I’ve not been a teacher. But when she wrote (http://bit.ly/1ViT8sJ) last week that there were concerns – among teachers, no less - that the new “Progress 8” league table measure for secondary schools might leave parents “struggl[ing] to know which local school is performing best”, I found myself asking: why would any sensible parent use one exam indicator as even a major criterion in choosing a school? I’d say school visits and, above all, talking to other parents who have used it to find out what actually goes on there would be much more useful. And would not inspection reports which actually went into detail about the subjects on offer and how they were taught be more useful than an indicator which, inevitably, will be subject to change over the five years a pupil is at secondary school? Seriously. As a prolific user of the DfE’s educational spreadsheets, I would say don’t ever take what a school’s results data seem to say without a great deal of caution, and rely on as much qualitative information about what is happening in schools as you can get if you are ever choosing them for your children.

Note 3: This can happen at a very superficial level: we can talk about what data seem to say without ever, for example, really investigating what might seem like basic questions to be curious about, such as the implications of pupil year groups shrinking over time. (See my story here: http://bit.ly/1P8RysS) So official data don’t tell the “truth” about any organisation; only providing one version of a truth, which may change dramatically on further examination.)

Note 4:  Anyone doubting just how data-focused the inspectorate remains should have a look at the recent verdict on the country’s largest academy chain, the Academies Enterprise Trust. The first four bullet points of Ofsted’s verdict as communicated in its “outcome letter”, viewable here http://bit.ly/1L8OOvu, are either assessment results indicators themselves or statistics related to Ofsted inspections, which themselves are, of course, likely to be highly assessment data-orientated).

Note 5: pupils can under- and over-perform on the day in tests.

Note 6: If it works, great. My issue is that the likely detailed effects on pupils haven’t been checked by the government which nevertheless has pushed it through.

Page published: 25 February 2016