The TES journalist http://www.naht.org.uk/warwick-mansell.rss If I were a parent there, I would be absolutely aghast at what appears to be happening to a group of primary schools in Haringey, north London. Parents I have spoken to this week, whose children attend a Haringey primary which in November was placed in special measures by inspectors and which now seems to be being compelled into academy status, are enraged What leaves those I spoke to almost speechless is the extent to which they say they have been bystanders in the whole process, with the DfE not bothering to explain to them why a change in the school's constitution is needed, the detail of how it will improve matters or even what an academy is. The whole process, from parents first finding out that the school was being lined up to become a sponsored academy whether they liked it or not to the Government finalising that decision, would take only two months including the Christmas holiday, an absurdly short time for a decision which would change the school seemingly permanently. This, at least, is the view of parents fighting the forced move to academy status at Noel Park primary, a 580-pupil serving an economically disadvantaged, racially mixed community in Wood Green. I would guess there have been similar goings-on in the nine other English local authorities which the Department for Education has targeted for its first wave of “forced academy” conversions and which may be the model for many more across the country. I am continuing to write about this situation at length because I believe that, whatever you think about the academies policy, this process is shockingly undemocratic and disrespectful to views other than those within the Department for Education. It seems in these cases, that parents are only allowed a choice if it coincides with that of the Secretary of State. Although the campaign at Noel Park has been less high-profile than that at nearby Downhills school (which I wrote about here http://bit.ly/voANSD and here http://bit.ly/uUvwcN), feelings seem to be running just as high. In results collated this week, despite that seemingly damning inspection judgement, 208 parents – in an electorate of 400 – voted against Noel Park becoming an academy, with only 14 in favour and 16 undecided. It seems, though, that these views will count for nothing, with governors being given until the end of this week by the DfE to agree to academy status, or face the governing body being sacked and replaced by one appointed by the department itself to implement the academy move. The move towards academy status at Noel Park started, for parents, barely seven weeks ago, on December 16th, as they learned how ministers had reacted to Ofsted's verdict on the school. Following an inspection visit in November, the school had been placed in special measures. Yet this was greeted with shock – and disbelief in some quarters, of which more below – as only three years earlier, under the same head and with seemingly lower English and maths test results, Noel Park had been found to be good with outstanding features. Nevertheless, on December 16th parents were informed that the school had received a letter from Lord Hill, the schools minister, saying it would have to become an academy, and giving governors until only January 20th to agree. Governors have since won a stay of execution until today (Friday February 3rd), but the position remains the same. As at Downhills, what seems particularly to be enraging parents is the top-down nature of the whole process, with their views seemingly counting for nothing with the DfE, and no-one from the department even taking the time to explain the policy, or the detail of the school's likely future under a sponsor, to them. Noel Park's future is, then, simply being decided centrally from Whitehall. Edel Brosnan, an office IT manager with a daughter in year two at the school told me: “No-one at the DfE has had any contact with the parents, apart from when we have contacted them ourselves for information. The parents' body have just not been taken into the process at all. It's very, very hard to understand why that has been happening. “People are bewildered: they are feeling like: 'where did this come from'? Everyone is trying to keep an open mind about it, but there is no information, and there's no explanation. “It's just: 'this decision has been taken, and we'll keep you posted'. She added: “It's bewildering to me that DfE officials have not met parents, and that a minister has not been to the school, talked to parents and seen the children to learn about how it develops them, and the progress that they make.” Francois Joubert, another Noel Park parent, who also works in IT, who grew up in South Africa and has children in reception and year three at the school, said: “There is so little information…parents are wandering around saying: we are going to become an academy, but what's an academy?” The business of who might become a sponsor at the school seems, to this observer, particularly bizarre. Parents say two academy sponsors have visited Noel Park in recent weeks: the Kemnal Academies Trust – a chain of eight secondary and four primary academies – and the Academies Enterprise Trust, which is one of the largest academy chains, with 19 schools. Contact with neither seems to have been instigated by Noel Park, with Kemnal reportedly simply asked to sponsor it by the DfE. Parents are concerned that all of Kemnal's four primary schools are in rural or suburban settings with small numbers of pupils speaking English as a second language – compared to multi-ethnic, multi-lingual Noel Park, with its 50 per cent free school meals stats – and that Kemnal had not instigated interest in Noel Park itself. Opinions of the AET were more positive, but again, both Ms Brosnan and Mr Joubert were highly unhappy with the process. Kemnal took over the first of its primary academies in April last year, while the AET did so only last September, so both are still in their first year of running primary schools. Ms Brosnan said: “No-one has said to us: these are a range of possible sponsors, so would you like to pick the sponsor that you think would be best for the school, before the governing body votes. “I used to run a small business. This is not how you run a small business. It seems completely lacking in transparency and fairness: we have no information and the DfE obviously has loads of information.” It is fair to say that Tuesday's appearance by Michael Gove in front of MPs on the Education Select Committee, when he reportedly referred to opponents of the plan to force academy status on Downhills school as “Trots”, has not further endeared him to parents at Noel Park. Ms Brosnan added: “The people who have the information [in the DfE] seem under no obligation to share it with us, yet the minister in the Select Committee can crack jokes about the integrity and motivation of people who disagree with him. “That's really unprofessional. I would be up before human resources if that was the way I behaved at work.” Mr Joubert said, of the “Trots” claim: “It's just name-calling. I think he has got an absolute cheek. He's not bothered to even contact or speak to us or even send a representative to explain the situation to us. He's a coward. He is not going to come and face the music here in Haringey about what he is doing.” Then there is the timetable. Ms Brosnan said the December 16th starting point for the process meant that there was no time to carry out a detailed consultation with parents necessitated by such a big change at the school, with Christmas holidays virtually started. In the end, the snap poll of parents was conducted by the governing body after many families reportedly said one had to be conducted. The whole process of converting the school would have to be in place by the start of the coming academic year. “Which leaving aside the arguments about whether academy status is a good thing or not, September just seems an absurdly bad timetable. We are in special measures at the moment. My feeling is that we should be focusing on getting out of special measures, and not on a change of governance,” said Ms Brosnan. Finally, there is the issue of choice. Mr Joubert said he and his wife wanted a school which worked with the local community, not one which was imposed against its wishes. He said: “We chose to live here because we wanted to be part of a community, and we chose the school as part of that: to contribute to our community and feel part of it. Then someone comes along and wants to rip the heart out of that. We are not going to go quietly.” He added that while an emphasis on maths and English was important – improving the proportion of children achieving Government expectations in both subjects at key stage 2 is the reason given by ministers for the policy – he worried they would be over-emphasised in an academy. A new organisation might feel it had to focus too much on these two subjects, to get it through Ofsted's “narrow” inspection process, he said. “I want my children growing up with a broad education, to expose them to a wide range of possibilities. If I go back to my own upbringing, in South Africa, I did not have half the confidence that my children have, I did not have half the emotional and social skills that my children have. They're getting it largely from the school community and the confidence of the teachers. “I cannot state how angry I am about what is going on.” This chimes very much with the experience I had talking to parents at Downhills school, in Tottenham, before Christmas. The feeling then was that this was not a democratic process. Parents who had chosen to send their child to the school because of its ethos were particularly angry at the prospect of this potentially being taken away from them without them even being asked. I have heard that parents at another high profile “forced academy” primary – Montgomery, in Birmingham, where teachers have been on strike over the move – have also been feeling disenfranchised. The level of explicit compulsion in the coalition's forced academies policy is certainly greater than what went before. But it strikes me there was always been a large element of arm-twisting and centralisation in the opening of academies under Labour. The idea of sponsors not being arranged for a school by parents, the head or the governors but being put forward by officials within the education department in Whitehall certainly seemed to happen under the last administration. And it was often the case that central government bullied councils into academies, telling them they would not get their schools rebuilt under Building Schools for the Future if they did not let some of them close and re-open as sponsored academies. Although a different case can be made for academies whose governing bodies have chosen to convert under the coalition, in general the history of the policy has been that it has been a centralised initiative driven by a perceived need by politicians to be bringing about dramatic structural changes to English schools, for which they can then claim credit. The latest experiences in Haringey and Birmingham strike me as different from decisions on schools' futures, including school closures, over which local authorities have presided. As a local newspaper reporter in the late 1990s, I sat in on council meetings at which school re-organisations were discussed and argued at length by local politicians. Councillors pushing a closure plan would appear at a school to defend their decision, in contrast to the stance of ministers and their officials now. (Although DfE officials have visited schools to talk to governing bodies, this has been along the lines of “this is what the Secretary of State wants”, with no public explanation or defence of the policy on evidence-based grounds to the local community). In local government, the whole process would be far more drawn-out than two months, I think. Of course, school closures or re-organisations under local government were far from perfect. There were always complaints, for example, about council consultations being skewed to come to the outcome the council wanted. But at least there was a process. It is easy to spot the Government's response to all of this: process and consultation are for wimps. We just need to improve schools, and to improve them fast. This was the justification for the Government pushing through its Academies Act 2010, which opened the way for this element of compulsion, in double-quick time under emergency legislation. The astonishing thing about this latest development is the contempt for any sense of localism, or context: if a school's results are not good enough, that would seem to be justification enough for the Government's chosen reform route, and ministers' favourite policy - academy status -and there seems little that anyone can do about it. That last bit may not be completely accurate. Some 1,000 people reportedly took to the streets of Haringey on Saturday to protest against the forced academies policy in relation to primaries there, where another two schools seem also to be facing compulsory constitution changes. Mr Joubert said he would investigate legal challenges, possibly even under human rights legislation. At the time of writing 17 Labour MPs had signed a Commons Early Day Motion against the forced academies policy. So we will see; suffice to say these campaigns may not be universally welcomed at the DfE, despite the confidence of Mr Gove's appearance before MPs. Just finally, there is the question of Noel Park's Ofsted inspections. I should say here that I do not have enough detail to reach any kind of judgement on the reality of the school's quality, only having conducted interviews on the phone with parents and looked at the last two inspection reports, including the most recent, which said it required special measures. It is fair to say, also, that there are dark mutterings circulating in Haringey about the large number of primary schools which recently have been subject to negative Ofsted inspection judgements in the run-up to the Government's forced academies move in the borough. Any claims of political influence on the inspection process would be damning for Ofsted, although there is no hard evidence as yet, as far as I can see. What I do know is that it is a school with key stage 2 results which are below the Government's floor targets but which serves a highly disadvantaged community, with half its pupils eligible for free school meals, three quarters speaking English as a second language and with high pupil turnover, only half of its children having been with the school since year one. In 2011, “only” 56 per cent of its year six pupils achieved level four in English and maths Sats, which is below the Government's floor target of 60 per cent. (I say “only” in quotes because it is quite a thing to say that a school with this pupil clientele is performing poorly when one considers, of course, that level 4 was originally set as the performance of the average pupil.) But the interesting thing is how the latest Ofsted report on Noel Park contrasts not just with parents' views of the school – as measured by Ofsted's own statistics – but with the previous inspection judgement. So, of 105 parents who answered the question at Noel Park's last Ofsted inspection in November 2011, 101 said they agreed with the statement “I am happy with my child's experience at this school”. Four parents disagreed, two of them strongly. Now, arguments can be made about how parents might not have the “right” picture of the school: they may have low expectations of their child's teachers, that they do not see the detail of what is going on in lessons that inspectors gain and that they lack inspectors' professional expertise. Nevertheless, I find it strange that inspections can reach a verdict – the worst possible – which seems so at odds with parental opinion. And no substantive reasons seem to be given for this contradiction in the latest Ofsted report. And, for what it is worth – which I suspect is very little – all of the admittedly huge number of 12 people claiming to be Noel Park parents on Ofsted's “Parent View” website strongly agreed that they would recommend the school to another parent. But perhaps the most interesting contrast is with the previous inspection report on Noel Park, in September 2008. The lead inspector then, Ruth McFarlane, offered a rhapsodic view of Noel Park, saying in the report: “In the words of one pupil, 'this is a great school, and it's getting even better.” The school was then adjudged good for leadership, under its head, Tunay Hussein. The report said: “The headteacher has created a strong caring ethos. She is ably supported by senior leaders who share her vision and set very clear direction to improve pupils' progress whilst providing high levels of nurturing care. Achievement was also rated good by the inspectors in 2008, a few months after 65 per cent of pupils had achieved level four or above in English, and 54 per cent in maths. (In 2007, the corresponding figures were 57 per cent and 62 per cent respectively). Fast forward to November 2011, and the school's results – or “outcomes for individuals and groups” in Ofsted language- were adjudged inadequate. The report of the latest inspection visit, led by Sheena MacDonald, found: “Most groups, including those with special educational needs and/or disabilities, achieve less well than similar groups nationally, particularly in mathematics.” It added: “Pupils' attainment is low and their progress is inadequate. This has been the case for some years.” Yet the results in 2011, before Noel Park's latest Ofsted, show that 74 per cent of pupils achieved level four in English, and 66 per cent in maths, which is an improvement on 2007 and 2008. With Ms Hussein still in post, the latest Ofsted rated the school's leadership this time as “inadequate”, with the first comment in this section of the report being that the school's self-evaluation was too generous. It added: “The capacity for further improvement is inadequate since leadership and management, including the governing body, have not been effective in addressing long-standing weaknesses.” Edel Brosnan, who has called Ofsted to complain about the latest inspection verdict, told me: “It's very very difficult to look at the Ofsted report and not feel they are judging the school on intake rather than outcomes, because the school got good with outstanding features in its last Ofsted, in September 2008. “The school has not materially changed since then. It's Ofsted's framework which has changed.” Now, I'm sure that schools which have been good can slide, even when the leadership does not change. I don't know the detail of what has gone on at Noel Park since the last inspection. But what does amaze me is that there is not any indication in the 2011 Ofsted report as to why this latest judgement contrasts so dramatically with the previous one. If I were a parent with a child at Noel Park, I would like an explanation as to what has happened in the school such that the standards of education on offer to pupils have really gone backwards, as the two judgements would imply. As a reporter, I would like some explanation as to exactly what has changed in the running of the school. Instead, it is as if Ofsted starts from a clean slate every time, with no sense in the latest report of how the school was seen the last time it visited, and the need for inspectors to explain to parents why the two judgements are different: to provide a narrative link between the two. Ofsted's measurement system does indeed change very frequently, but the idea of acknowledging this as a factor in reports on which parents will rely, if indeed it is a factor, is a move of modesty which I am afraid I cannot really imagine from the modern inspectorate. It strikes me, again, as a flawed reporting system, and not one that is really parent-focused. Maybe Ofsted needs a notice to improve. -Stop press: I understand that Noel Park's governing body has now voted to accept academy status, under the sponsorship of the Academies Enterprise Trust, despite the parental vote against any forced academy move. Mr Joubert said parental campaigning and scrutiny of the academy arrangements would continue, however. If you have experience of the forced academy process, or observations on Ofsted, I'd be keen to hear from you at warwickmansell@gmail.com http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=526 Fri, 03 Feb 2012 16:41:24 GMT 20120203164124 Fri, 03 Feb 2012 16:41:24 GMT 20120203164124 Fri, 03 Feb 2012 16:41:33 GMT 201202031644133 I was expecting to write a blog today about further evidence on the effects of exam-driven teaching on children's educational experiences, as revealed in written submissions to a current Parliamentary inquiry into secondary assessment. I find myself, however, drawn to focus again on the intricacies of how exam boards operate the system itself, because of some extraordinary and seemingly very revealing testimonies to that investigation. Since September, the House of Commons Education select committee has been investigating the exams regime for 15- to 19-year-olds. I gave evidence to the committee last week on the interaction between the exam system and accountability. Another theme of this inquiry, however, has been the debate around commercialisation in the operations of exam boards: the extent to which their modern development as multi-million-pound organisations – and particularly the competition among them for teachers' custom - has been beneficial to schools, colleges, pupils and the taxpayer. It is a very interesting line of inquiry, and one I can't recall being pursued in great detail elsewhere. Some of the submissions, as I say, are extraordinary. Graham George, a retired physics teacher who served for 25 years as a principal or chief examiner for A-level physics with the Edexcel board, offers one. “During this time I have seen awarding bodies change from being academic institutions to large business corporations, with attendant advantages and disadvantages,” he writes. While Mr George, himself a textbook author, backs the commercial activities of boards in areas such as textbook design – a very contentious subject which I will discuss below – he adds: “The commercial nature of the awarding bodies to become profitable organisations at the taxpayer's expense is to be deplored. The Government must find some way to regulate examination entry fees, which are crippling schools and colleges who are struggling to cope in the current economic down-turn.” In particular, Mr George highlights dividend payments he says have been made by Edexcel, England's biggest board by turnover, to its shareholders in recent years. Edexcel was bought by Pearson, the world's biggest educational publisher which also publishes the Financial Times, in a deal originating in 2003 and which made it the only one of England's three exam boards to be run for shareholder profit. (AQA is a not-for-profit organisation while OCR is owned by an arm of Cambridge University). Mr George's evidence says: "I personally find it highly objectionable that Edexcel has ploughed some £75 million of British taxpayers' money into the pockets of Pearson shareholders over the last couple of years. I suspect that I am not the only one!" This, of course, is potentially explosive. Expenditure on qualifications in secondary schools and colleges has rocketed in recent years. Not-for-profit organisations, such as Edexcel's rival AQA, can at least argue that money taken from schools and colleges is re-invested in the qualifications system which they use. By contrast, money given to shareholders would appear simply to be lost to state education as a whole. Supporters of the Edexcel approach would have to argue that the benefits of the involvement of a corporation such as Pearson in UK qualifications, including the ability of a large global corporation to invest in innovation, outweigh the money taken out of the system in this way. This, indeed, is Edexcel's line. (See statement below). I wanted to check that £75 million figure. Edexcel's accounts say £40 million has been paid out in “dividends” in each of the past four years, with this money I think going to Pearson itself. How much of that cash was then paid out to shareholders outside the company is more difficult to track. Edexcel, in its statement, does not dispute the £75 million figure. Pearson's annual report for 2010 says that £2.3 billion has been paid out to shareholders from the global business in the past 10 years. In any case, it is also worth bearing in mind how Edexcel's profits have soared over recent years, according to these recent accounts. (Note one). After-tax profits at Edexcel were 37.6 million, on turnover of £225 million, in 2007. In the year to February 2011, they had risen to £66.9 million, on turnover of £320 million. The profit rate – after-tax profit as a proportion of turnover - over that period also rose, as these figures imply, from 17 to 21 per cent. The great majority of turnover came from sales within the UK – in the year to February 2011, 90 per cent of Edexcel turnover came from sales to the UK, its latest accounts show - so money from British schools and colleges appears to be driving this. Shareholders must be delighted. Edexcel says early investment in technology shortly after the takeover, to the tune of £35 million, outstripped profits at the time and has brought about great benefits in professionalising the marking industry, mainly by having much examining now take place at the computer screen. Edexcel's accounts make clear that it is a rise in sales of vocational qualifications to schools and colleges, which rose 26 per cent in 2009 after a 15 per cent gain in 2008, which have driven these sharply increasing figures, with GCSE and A-level business rising much more slowly. I wonder about the political implications of all this. I wrote in my last blog here about Michael Gove's recent suggestion that the current structure of competing exam boards could be reformed. My guess is that concerns about these profits may put more pressure on Ofqual, to regulator, to try to act to hold down exam costs to schools and colleges. Mr George also writes about Edexcel moving away from being university-based: 25 years ago it was the University of London Examination Board and most A-level physics papers were set by university academics, while today it is a much more commercial organisation, with universities scarcely involved. But there is far more in the submissions than this. A sub-theme on one particular type of allegedly commercially-minded decision made by the boards is present in several pieces of evidence, including that of Mr George. This revolves around at least some of the boards, in some subjects, moving in recent years away from paying for face-to-face “standardisation” meetings between examiners, at which their marking work is checked and discussed with their supervisors in an attempt to ensure consistency, towards conducting this online and over the telephone. Cost-cutting is the reason I have heard or read given for this change by several senior examiners now, and not just from Edexcel. It is fair to say they have worries that it might be affecting the accuracy of some people's marking, while some examiners are also said to have been put off marking by the move: many like to discuss their questions about how to interpret a mark scheme at length and in person with a more senior examining colleague. Evidence from Neil McNaughton, a principal examiner at AS level in Government and Politics for Edexcel, says: “The 'standardisation' of examiners used always to take place in intensive, face-to-face meetings. With Edexcel (and, I think, other boards), these are being replaced by online meetings or other forms of 'remote' standardisation. “Naturally face-to-face meetings are far more expensive than 'remote' meetings and practices. That is undoubtedly why the boards are attracted by them. “All my close colleagues agree that this reduces the quality and accuracy of the standardisation process. However, it seems unlikely that we will return to face-to-face meetings.” Another piece of evidence, submitted by a Richard Nixon, a coursework moderator for GCSE applied science and an examiner for both GCSE and A-level chemistry and GCSE citizenship, also raises concerns about standardisation, having experienced both online and face-to-face versions through Edexcel. He writes: “[Having standardisation done through online or on the telephone] saves Edexcel lots of money in teacher release fees, travel costs and hotel bookings but not sure that it is the best way to prepare examiners for marking papers.” Mr Nixon writes that he raised the issue with Ofqual but was told it was up to Edexcel to decide how to organise standardisation. A final major theme of evidence on “commercialisation” concerns exam boards' association with the GCSE and A-level textbook market, including their endorsement of books badged as tied to each exam. Pearson itself now operates a publishing brand called Edexcel, tied to the exams it provides through the board of the same name. In my book on results-driven teaching, I devote a chapter to what I have come to believe are the narrowing educational effects of teaching through exam-orientated textbooks and other resources. This practice comes in for scathing criticism in several submissions here. The Society of Authors, whose membership includes more than 700 education writers (not including me, at the moment), includes in its evidence a host of quotes from individual members about this issue. One says: “We are all so driven as teachers to ensure students pass examinations that we have lost sight of the fact that we are supposed to be educating children. As a teacher, I am finding students to be ever more unwilling to think independently because all they want is the answer.” Another says: “I am.disturbed by the commercialisation of examinations and the links between publishers, examinations and textbooks. The specification is written, the textbook written by the examiners meets only the specification, and the teaching is restricted to the textbook. It is a circle that is difficult to break and destroys innovation, creativity and good teaching and learning.” And another: "Publishers have stifled initiative in their pursuit of materials more or less guaranteed to 'get pupils through the tests', which is what the schools want... until you get rid of this mentality or change the tests they will continue to do so. The exams are the tail which wags the dog." Other criticisms of board-endorsed textbooks come from organisations including the Advisory Committee on Mathematics Education, Mathematics in Education and Industry and SCORE, an umbrella body for five leading science organisations, while the British Academy is concerned about Edexcel being owned by Pearson, a publishing company. The submission from Oxford University Press gives an insight into commercial and professional rivalries in this field. It writes: “OUP has serious concerns about the commercial activities of awarding bodies and their links with publishers. The boundaries between awarding bodies and publishers are increasingly blurred, giving rise to very real conflicts of interest. These clearly affect the perceived worth of the exams and, in OUP's view, are also likely to undermine their actual worth, by encouraging narrow focussed teaching with no scope for differentiation of approach, undermining the scope of a pupil's educational experience and genuine attainment. Public confidence in the administration of exams is also affected by concerns about these issues, and must be restored.” The submission includes a claim from OUP that its representative was banned by Edexcel from an unnamed exam event in 2009 at which textbooks published by Pearson under its “Edexcel” publishing brand, written by an Edexcel examiner, were promoted. There is much, much more, among 71 written submissions at time of writing. Among concerns that England's current accountability regime may not be helping produce a good education for children, I would highlight evidence from Janine Clatsworthy, an English examiner, teacher and former local authority adviser. She appears to approve of some form of statistics-based accountability as helping to raise some schools' expectations of pupils. But she adds: “In schools where the 5 A*-C [grades] are an over-riding concern, almost all the school's energy will be focussed on the C/D border, with insufficient attention being given to those pupils who should be achieving B, A or A*: ironically, disadvantaging the most able and undermining their clear potential to become university undergraduates and – in the long term – raising their life expectations. Yes, the old chestnuts of the level of family support, discipline and expectation in impoverished households does impact pupil progress, but focussing overwhelmingly on the C/D borderline – and 'teaching to the test' has an equally limiting effect.” The Advisory Committee on Mathematics Education writes: “At times, the assessment system, and the regulations which govern it, undermine the broader aims of the national curriculum and [at post-16 level] the A-level subject criteria.” Returning to the issue of commercialisation, I have always thought that the move, under Charles Clarke, education secretary at the time, to allow part-privatisation of our exams system through Pearson's acquisition of Edexcel, has needed close scrutiny. In the past, I have written about positives as well as negatives stemming from boards becoming more “commercial”: undoubtedly greater business sense within the awarding bodies has helped both to move their finances on from what a decade ago seemed in some cases to be quite parlous positions – with resultant risks for schools and colleges using their products - and to bolster investment, especially in technology. I told the inquiry last week that, in the end, I was not persuaded by the benefits of a single exam board over today's system of competing boards, with the ability of teachers to take their “business” elsewhere should they feel let down on marking by a particular board particularly high up in the list of reasons, with reservations, to retain the current structure. (See my last blog). However, concerns raised through this inquiry – and through the Daily Telegraph's coverage in December of some examiners' conduct at training events – are bound to prompt further reconsideration of the overall effect of the competitive profit motive on the exams business. The statement I received from Edexcel in relation to profits having put Mr George's comments to the board follows: "When Pearson first took a controlling stake in Edexcel in 2003, the UK examination system was creaking under the pressure of a dramatic increase in volumes and complexity of qualifications. Many papers were lost as scripts were shipped around the country. Outdated practices and marking scandals had badly shaken public confidence in the system as a whole. This was the system that Ken Boston [former Qualifications and Curriculum Authority chief executive] described as a “19th century cottage industry”. "The problems that the Government faced with the national curriculum tests in 2008 demonstrate that these risks still need to be managed very carefully today. “The £35m investment Pearson made in 2003 in the development of innovative new mechanisms for online marking and infrastructure to deliver more reliable and professional systems has underpinned the stability of our examinations ever since. This investment was, at the time, some five times greater than Edexcel's annual profits, and we have continued to invest in developing and upgrading our systems. Other boards have spoken publicly of these changes as a catalyst for their own modernisation programmes. "Within the right regulatory environment commercial organisations can encourage innovation and improvement. For example, the development of national and international qualifications enables the UK to learn from new developments and best practises [sic] around the world, especially from those countries where high quality education is proving to be a major source of economic growth and global competitiveness. "As a large commercial organisation that is in education for the long term, we are acutely aware of our responsibilities to young people and teachers. These people choose our products because they represent high quality and bring with them a reputation which in itself enables progression. Commercial pressure pushes us to raise standards, not to lower them.” All submissions to the committee's inquiry can be accessed from this page: http://bit.ly/wcE3xf Last week's oral evidence session can be watched here: http://bit.ly/z5stU0 (Note one: In December, the Daily Telegraph reported that Edexcel's profits were £10 million in 2004. In 2003, accounts for London Qualifications, the organisation running Edexcel which Pearson bought, were £3.6 million, its accounts show.) http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=522 Tue, 24 Jan 2012 15:23:18 GMT 20120124152318 Tue, 24 Jan 2012 15:23:18 GMT 20120124152318 Tue, 24 Jan 2012 15:24:15 GMT 201201241532415 Radical change seems to be on the cards for England's GCSE and A-level exam system in the wake of last month's revelations in the Daily Telegraph of examiners seeming to give too much help to teachers, attending advice seminars, about the content of forthcoming exams. Undercover journalists also filmed a GCSE geography chief examiner for the Edexcel board, Steph Warren, saying that schools had moved to her board because its course was easy, while a maths examiner from rival board OCR also claimed Edexcel was seeking to attract schools to its version of exams by making its questions “straightforward”. I witnessed similar behaviour myself, back in 2005, in research for my book on the pressures on schools to raise their pupils' results. I sat in on two seminars, one in which a senior French examiner advised teachers how to get pupils through their GCSE by scripting oral answers, copying phrases for coursework assignments and not bothering with grammar; and a second in which an influential history examiner said that a “realpolitik” or “ends justify the means” approach to raising results meant that good students did not need to be stretched to get the highest marks. The results-are-everything culture imposed on schools can lead to this kind of thing being normalised, but I still found it shocking and depressing. A widely-made complaint about the exams system, heard much more forcefully in the light of the Telegraph's revelations, has been that competition between the boards lowers education standards, as awarding bodies seek to make their exams easier as they chase the business of schools who are now very motivated to improve results however they can. So, in reaction, Michael Gove has suggested that direct competition between exam boards could be scrapped. In its place, he has picked up on an idea which has been gaining currency over the last year or so. This would see direct competition between the boards for schools' “business” replaced with a structure in which each board would bid (presumably to the government) for a contract to run an exam in an individual subject. This has been likened to the system of rail franchises, whereby companies compete to provide a service on particular routes for a set period of time. There would, then, not be a range of boards offering exams for each subject, but just the one. It's likely we would still have the current range of GCSE and A-level boards, but each would only be active in the particular subjects whose contract they had won, rather than across the full range. All this is tentative at the moment, with Mr Gove saying before Christmas that he would await a report by Ofqual into the Telegraph's revelations before making any final decisions. But, as someone who has covered the ins-and-outs of the exams and testing system for nine years now, I thought I'd offer some thoughts. Although the idea of a single exam board – or at least a single board in each subject – would have positives, I think it would also have serious downsides. I'd be cautious about making this move, then, and think Mr Gove would be well advised to consider other reforms to the system at least alongside it, if not as alternatives altogether. So, again, removing competition between the boards for schools' business would have one apparent upside, in that it would seem there would be no danger of boards seeking to attract business by making their exams easier to pass, or at least giving this impression to schools. Now, I've never been convinced, on this, that boards go in for anything as crude as deliberately lowering grade thresholds to try to bump up their numbers. There is a case, I think, that boards in general might worry so much about their market share that, while not seeking deliberately to make each exam easier, they at least would not want their assessments to be seen as more difficult than those of their peers. For an example of the pressures the boards are under - with many if not most schools under great strain through the government's accountability system for their results to rise steadily, year on year - see this article I wrote for the Guardian in 2009. ( http://bit.ly/2usTF ) It's not always that simple, though. I have come across evidence of boards resisting such pressures, as this 2008 ( http://bit.ly/AtsSvi )report on AQA seemingly wanting to set a higher standard for a new GCSE science exam than its rivals would suggest. However, the boards don't always help themselves, if they want to be viewed by the wider public as standing above the business of seeking schools' business by lowering standards. I reported in 2006 how Edexcel was at that time marketing a series of multiple-choice tests for science GCSE on the basis that it gave pupils “more chances to succeed” and “allowing them to be tested on material when it's fresh, and can take multiple tests before submitting their best performance”. Edexcel also apologised in 2004 for a maths exam which had proved difficult, promising to make it “more accessible” (Note 1) the following year. I have also had it put to me, by teachers and educationists of – I think – a wide range of political perspectives, that competition between the boards for business does indeed lead to that “race to the bottom”. But the disadvantages to be weighed, in moving away from the current arrangements, would be quite considerable. First, the most glaring, for me, is that teachers would no longer have a way of venting their frustration with a particular board by taking their business elsewhere, should they be unhappy with the service it provided. Often, when I've spoken to teachers who have been on the end of what they think is poor performance by a board – usually over complaints of bad marking, and an alleged unwillingness of a board to sort it out properly – the idea of opting for a competitor is one of the only ways of taking some control over a situation in which they feel that they and their pupils have been treated unfairly. Changing boards is often not an easy step to take, of course, with often new textbooks to buy, and the conventions of a new syllabus and assessment to master afresh. But having this power is worth something. So I would suggest that, if ministers do take this decision, it is not going to be uniformly popular in staffrooms. Second, having a single board – either one in each subject, or one board for everything – might risk making England's already highly centralised, ultra-political education system even more so. Boards which make their income by competing for the business of schools and colleges – especially ones with an establishment such as Cambridge University behind them, as in the case of the OCR board – have a degree of independence from ministers which might not be possible were they under more direct oversight by one politician, or even competing for business not from schools and colleges, but from the Government. Third, on a more practical level, there is the question of the inevitable upheaval which arises from major structural change. One of the last major organisational changes in the assessment world in England came, of course, in 2008, when the decision to bring in an outside firm – ETS – from America to run our Sats marking system ended in spectacular failure. The GCSE and A-level exam boards were themselves not in a particularly healthy state as of around 10 years ago, after the last big changes in the boards' structures, when amalgamations of smaller organisations into three awarding bodies in England were still bedding down. Things are much more stable now, in terms of the way the boards operate. Although a string of mistakes in last summer's exams did not show the boards in good light, these were not on the scale of, for example, the ETS debacle. So there will be major risks with any big structural change. Finally, and perhaps related to this, the economics of exam boards would need serious consideration before any change was made. As I have always understood it, subjects for which hundreds of thousands of pupils are entered through each board every year, such as English and maths GCSE, generate large proportions of each board's income. This, the boards say, then helps to cross-subsidise exams in less popular subjects and qualifications. A board competing for a franchise in a major subject and losing might, then, face a serious impact on its overall finances, which might create knock-on problems for schools and colleges. Would boards want to bid for minority subjects if they lost out for franchises for more popular ones? And if a board were running only minority subjects, would it feel the need to hike exam fees to compensate? All these issues suggest to me potential headaches for a new system, bearing in mind our current starting point. What alternatives could there be to scrapping the current system of competing exam boards? Well, I'd look at the ways in which examiners earn money. There is a clear moral case, I think, simply for banning any examiner from taking money to offer advice on how to do well in exams over which he or she has influence. This would rule out examiners being paid to speak at conferences on their exams, or to earn money by visiting schools directly to give advice. Reading the above paragraph back to myself, it does seem strange that this has ever been allowed to happen. I know the argument which is used to justify it – along the lines of there being no harm in examiners giving schools clarity on what is, in general terms, to be expected – but really: paying examiners for advice on how to do well in exams they set? No potential problems there then… If advice on how to succeed in an exam is valuable and thought to be legitimate, and institutions and pupils are supposed to be competing on equal terms, it should be made available for free to all. Examiners, boards and companies should not be trading on “inside information” about exam success, even when the defence is offered that this is sufficiently non-specific about the content of particular papers to be acceptable. I heard Andrew Hall, chief executive of AQA, argue at a conference last year that restricting examiners' freedom of trade in this way might be difficult, given that most of them operate on a freelance basis. If this is so, should serious consideration not then be given to employing those holding senior positions in the system in-house? Finally, though, anyone seeking to reform this system effectively has to recognise that the business of using exam grades both for their original purpose – assessing how well a pupil has mastered a course of study – and for holding their teacher and school to account inevitably carries a risk of corruption. Teachers now have incentives to look for short-cuts which will enable them and their institutions to look good, really however this can be achieved. This can vary from the unethical – providing too much help to pupils with coursework or controlled assessment – to the not unethical but arguably anti-educational, such as the widespread use of exam-specific textbooks written by examiners and focused very closely on the likely content of each paper. I think that has to be taken on board at a fundamental level for any change to succeed. If you think, as I do, that the corrupting and anti-educational effects of judging teachers on their pupils' results are pervasive and damaging, you need to do accountability in a different way, because holding teachers to account through their pupils' exam grades will always carry the risk of lowering, rather than raising, standards. This is the case no matter how much anyone argues that teachers and schools should resist any bad incentives that exist. If, on the other hand, you view “high-stakes” results-based accountability as unavoidable and necessary, you need to police the system much better. You need to recognise at every stage that the chase for results comes with it serious potential downsides as well as – to its supporters – benefits and seek to minimise those side-effects where you can. It is not enough simply to blame teachers who follow the systems incentives and to say they should not. Although this government, like the last, has sought to change indicators and reform assessment systems in piecemeal fashion to tackle some of the side-effects, I'm not sure Mr Gove has recognised the depth and challenge of the kind of reform which would be needed really to root out these problems properly. (Note 1: In fact, that word “accessible” is interesting. It has a technical meaning in the exams world: papers should not be “inaccessible” for pupils if, for example, questions are written in a way that is not clear or confuses pupils, so that their underlying understanding of the subject is not assessed properly. So boards could be said to be acting reasonably by making their exams “accessible”. However, over the years I think the word has also been used to suggest to schools and colleges that an exam will not present unexpected, or too many, difficulties to their students.) http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=518 Tue, 10 Jan 2012 09:27:06 GMT 20120110092706 Tue, 10 Jan 2012 09:27:06 GMT 20120110092706 Tue, 10 Jan 2012 09:27:06 GMT 201201100992706 As a statement which is the polar opposite of reality – at least as is being observed in schools in one north London borough now – it is hard to beat. “Teachers, not politicians, know how best to run schools.” (Note one) This was the headline offered by the Department for Education above the press release of one of Michael Gove's first interventions as Education Secretary, in May 2010. In a letter to all schools, Mr Gove urged them to consider the benefits of academy status. The press release quotes him as follows: “The Government is genuinely committed to giving schools greater freedoms. We trust teachers and head teachers to run their schools. We think head teachers know how to run their schools better than politicians and bureaucrats.” This line – that heads and teachers, not politicians or the dreaded “bureaucrats” (note two), know best how to run schools – has featured prominently since then, in DfE speeches and press notices in relation to academies in particular. I have spotted the claim, for example, in a speech by Mr Gove from last January highlighting the fact that, at that time, one in 10 secondary schools had become academies, and again in comments in October by Nick Gibb, the schools minister, in relation to academies' GCSE results. Now 10 days ago, I visited what seemed to me – from, admittedly, one short visit - to be a lovely primary school in Haringey, north London. The head and staff see no reason to convert to academy status. Indeed, to do so would be a huge distraction from the improvement journey they have been on for the past 12 months, while also creating upheaval for the pupils, they argue. The governors have passed a unanimous motion against academy status. There is a vigorous parental campaign against it, and some parents I spoke to were outraged that, having chosen the school for their children because of its ethos, its character might change as it became an academy under the joint control of an external sponsor and the DfE. Despite all of this, it seems that the governance of this school will indeed change, in a process brought about in a matter of weeks and seemingly without any meaningful consultation, because a politician, Mr Gove, supported by “bureaucrats” from the DfE, is advocating it as the right thing to do. This much was set out in a meeting held at the school between two DfE representatives and Downhills governors, at which it was made clear that the Secretary of State expected to see sponsored academy status for schools in Downhills' position. (It is on a notice to improve from Ofsted, though the inspectorate said it was improving following the last visit, in September, and has been below ministers' Sats floor targets in recent years although again the latest set of data see it above the government's centrally-set thresholds.) Jacky Griffin, a consultant working for the DfE as part of a team focusing on promoting academy status for schools in nine local authority areas, told the meeting: “What the Secretary of State said in June and subsequently is that, in his view, the best option for schools where performance has been low for some time is that they become a sponsored academy… “Our preference would be, and it is with every local authority, and every school that we have worked with, is that the school is going to be a sponsored academy. “We would much rather work with the school, with the local authority and it gives us a better chance of finding the right sponsor, the right partner.” This school is one of hundreds across England seemingly soon to move to academy status not because either the staff or the community have opted for it, but because a “politician”, the Secretary of State for Education, is requiring it of them. As I say, the irony – if that is the right word in this case – is probably hitting staff and parents at Downhills, this school in Tottenham, with a sickening force at the moment. Certainly, it will have hit the head teacher and governors, for whom tears at what is happening to the school have been in evidence. I've written about what is happening at Downhills and other schools for the education section of today's Guardian. (http://bit.ly/voANSD) The first thing that struck me about this story was the extreme level of centralisation of policy-making and power now evident in the English education system. Despite, in this case, little evidence that anyone connected to this school wanted it to happen, conversion to academy status now seems on the cards because Mr Gove would appear to have the power to force it through. There seem very few checks and balances, which would allow for anyone to question whether Mr Gove's interpretation of the performance of the school is correct, or whether the case for intervention has been made. (Note three). This has been made possible through a steadily ratcheting-up of the Secretary of State's power, in a process which began under Labour. The Education and Inspections Act 2006 gave local authorities the power to intervene, by issuing an improvement notice, in schools which were identified as “underperforming”, on a variety of measures. When that law was passed, ministers wrote to local authorities urging them not to be afraid to use this method. Further legislation under Labour then centralised this system further, with the Secretary of State granted the power to force a local authority to issue a warning notice. But the most recent education act appears to take this even further. If I've read this right, it says: if a local authority decides to rebuff the Secretary of State's attempt to get it to give a warning notice to a particular school's governing body, it must write to the Secretary of State to let him know. In which case, he can order the local authority to follow his instructions anyway. And the local authority must comply within five days. Oh, and they must send him a copy of this notice as soon as they send it. (See http://bit.ly/sTsWRT ) Now, last year's Academies Act then further cemented this centralising trend, by allowing the Secretary of State effectively to intervene in the management of a school himself, by issuing an Academy Order to require it to leave the local authority. Schools which were identified as eligible for such intervention, in the Academies Act 2010, were those in an Ofsted intervention “category” or “eligible for intervention”, as defined in some other way, through Labour's original 2006 Act. And, it turns out, or so NAHT advice informs me (see this link:http://bit.ly/rrKHBv ), that “eligible for intervention” also embraces schools where the local authority has issued a “warning notice”, because of results which are felt not to be good enough. Coming across this stuff, I must say I wonder if this country is becoming the sort of place I always thought it wasn't: one where central power can be allowed to go more or less unchecked, in pursuit of some goal which also tends to be defined by those wielding the power, in this case that education standards – as defined, again, by those with the power – have to rise. There is an authoritarian, almost Orwellian, strain to all of this which is becoming very worrying. Underlying this, of course, is a belief that the Secretary of State is the ultimate guarantor of high standards in our schools. In his letter to Downhills, Lord Hill, a schools minister, writes that not only was it in an Ofsted “category”, but that it had “been below the floor standard for the last 5 years”. Well, let's leave aside the fact that the school was said by Ofsted to be improving satisfactorily at its last inspection, and therefore on a trajectory it believed it should be allowed to complete. And let's gloss over the fact that, actually, results published last week, just three days after Lord Hill's letter was received, show it above floor target. The deeper point is that, if you were being charitable to Mr Gove, this is coming down to a dispute over who has the best interests of the school's pupils at heart. It is right that the centre wields that power, it is contended, since this ensures that schools are always put under pressure to improve. One argument against that would be that the centralisation now apparent in this system, at least in relation to schools where results are not felt by ministers to be good enough, is dangerous because it allows only one person's interpretation of those results to carry the day. Mr Gove and his civil servants are looking at results data and Ofsted's judgement that the school needs intervention, the latter of which will have been heavily influenced by test statistics. They are then making two judgements: first that results are not good enough, and second that a structural solution - academy status with responsibility for the school to rest with an external sponsor - will improve those results. Those contesting these judgements, by implication and effectively also by assertion - if the recording of the meeting between DfE representatives and Downhills governors at which the sponsored academy route was laid out is anything to go by – simply need to have higher standards for the children they educate, or so the argument would go. It is extraordinary, though, that this viewpoint can be reached by Mr Gove seemingly without detailed local knowledge, and without any meaningful consideration of the views of parents in particular, as well as of staff. Then there is the data itself. A bit of context is needed here. Downhills educates large numbers of pupils eligible for free school meals (43 per cent), plus, the school says, more pupils which, because their parents do not qualify for state benefits, are not eligible for free school meals despite being very poor. They thus do not show up in the DfE's disadvantaged pupil statistics, it is claimed. As far as I can see, the school also goes out of its way to be inclusive. In 2009, the last year for which test data was available when inspectors conducted their last full inspection in January this year, Downhills had a bad year results-wise, which appears to have triggered that notice to improve. Unfortunately, teacher assessment data from 2010, when the school boycotted the Sats tests, is not being taken into account by the DfE in its judgement on whether this school, and others, have been above or below the KS2 floor targets in recent years, meaning the 2009 figures assume extra weight. The school also tells me that a small change in the number of pupils who are counted as not having been with the school sufficiently long to be included in its published KS2 data is likely to improve its published results from 61 per cent achieving the “expected” level four in English and maths to 64 per cent. This may not sound like a lot, but it could make a difference to the way the school is perceived when the DfE's (unilaterally imposed, and backdated) floor target now sits at 60 per cent. In other words, each set of figures is subject to interpretation. A different person, looking at these figures, might view them differently. But, crucially, ultimately only one person gets to interpret the statistics here. Again, this is authoritarianism, which stands in stark contrast both to allegedly decentralising coalition notions such as the “Big Society” and to the idea that millions of local decisions – parents choosing the schools they like –rather than top-down moves from Whitehall – should be given priority. But remember that this is not just a dispute about whether results are good enough; but whether a particular method which it is claimed will improve schools actually will do so. And, here, staff and governors at Downhills appeared to have a point when they argued, in their meeting with the DfE representatives, that there was very little evidence for the success of sponsored academies in the primary sector. That's not surprising, of course, given that so far there have been very few sponsored academies in the primary sector. As Ms Griffin told the meeting: “There is not a great deal of evidence specifically from primary academies [but there is] a wide range of evidence to support the academies programme more generally. That is focused in secondary, and there is no reason to believe that the elements of the programme, of the structure, will not work equally well in primary.” To repeat, the level of centralisation is extraordinary. Not only is one man's interpretation of results data seemingly unchallengeable by a school, its elected governing body or its community, but a policy is being pushed forward as the solution for which people, essentially, are having to take on trust from that one individual that it is a better option for their school. I could stop this blog here, with those worries about centralisation. But many people would argue I had not gone far enough, because, they would contend, Mr Gove also has an ideological aim for our schools system: he wants privately-managed providers, rather than local authorities, to be dominant players. Many feel that the long-term agenda is to allow profit-making institutions to run schools. It would be quite simple, it is contended, to allow the non- profit-making organisations now operating academy chains to be replaced by – or turn themselves into – conventional private companies. If that analysis is true, then, in concentrating power so heavily in one man's hands, we will effectively have allowed that individual to have forced through his own private (note four) agenda on at least a large part of the schools system, with the alleged low performance of hundreds of schools simply the vehicle for the pursuit of that agenda. I suspect, also, that the case of Downhills, other schools in an Ofsted “category” and those under the floor targets are only the tip of an iceberg in terms of schools which could be affected by structural governance change; certainly stories are emerging of local authorities all over the country feeling under pressure from the DfE to push other schools towards academy status. It's scary stuff, but in contrasting the lovely atmosphere of that school in Downhills with the “brutality” of the academy conversion process, I wonder if the scary stuff isn't winning out at the moment. Just almost as a postscript, having visited Downhills I think it's more necessary than ever to keep in mind the positive aspects of what goes on in schools up and down the country at the moment. For example, Downhills reported that it gives free music tuition to every year four pupil, in this disadvantaged area of Haringey; a fact seemingly not picked up by Ofsted inspectors whose report focused mainly on test results data. We should not forget, either, that Ofsted itself has consistently found huge numbers of parents supportive of their local school, with in excess of 90 per cent agreeing with the statement “I am happy with my child's experience at this school”, although this fact has tended to be buried within the annual report of the Chief Inspector of Schools. I also conduct interviews regularly with heads of international schools, as part of a series I am writing for the Daily Telegraph's international edition, and am struck by how regularly they comment on both the benefits of England's national curriculum approach to teaching, and on areas such as special needs provision; it seems that, in many other countries, this is not a strength. Of course, the system is not perfect. We do want more people not to have left school with a less-than-positive view of the whole experience. I'm hardly one to argue that assessment and accountability structures in particular are not producing some serious problems in schools. But still…all those arguing that the system needs radical structural change need to be treated with caution. It may just be that it's true: other people than national politicians are best-placed to make key decisions on schools' futures. (Note one: following a brief twitter discussion with Sam Freedman, Mr Gove's policy adviser, last night, my reading of it is that this statement only applies to “successful” schools, with success defined by the DfE). (Note two: DfE press releases containing this phrase have probably gone through a civil servant “bureaucrat”, just to add to the Alice in Wonderland feel to all of this…) (Note three: In this, I think it even contrasts to school closures conducted in the past through local authorities, which would seem to have left the option of an appeal to central Government by parents fighting school closures, if I recall correctly having covered a school closure story in detail more than a decade ago. Under Labour, government “hit squads” were also sent in to schools including Hackney Downs, the comprehensive in east London which eventually became the lauded Mossbourne Academy. But these changes happened on a much smaller scale than is envisaged now.) (Note four: I've just checked the Conservative party's manifesto and, though it did mention bringing in academy providers to run schools which had been in special measures for more than a year, there was no mention of doing the same for other schools, such as Downhills. This has not, then, been flagged up for voting on by the electorate.) http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=515 Wed, 21 Dec 2011 11:17:18 GMT 20111221111718 Wed, 21 Dec 2011 11:17:18 GMT 20111221111718 Wed, 21 Dec 2011 14:34:08 GMT 201112211423408 I have been writing about the Government's academies policy, on and off, since about 2002, but this was an angle I had never come across before. Speaking at an event organised by the Campaign for State Education and five other organisations which are critical of the policy, the barrister David Wolfe set out the legal basis on which academies stand. And what he had to say was in many ways very strange. I think that the public would also find aspects of what he said surprising, if they were ever faced with trying to get to grips with the intricacies of the academies policy. And it may be that, legally or even philosophically, problems are being stored up for the future which will become more apparent as more schools convert. Mr Wolfe, who regularly works for parents and pupils challenging decisions by academies and other schools, was speaking at a conference entitled “Caught in the Act”, which offered attendees the chance to hear detailed scrutiny of issues around the passing of the Education Act 2011. This became law only last month and includes clauses on academies. And the nub of the two presentations he made was that academies, essentially, are not covered by the main body of education law, as it has developed since the 1944 education act. Thus, he said, non-academy schools - or “maintained schools” as the law calls them – are covered by legislation embracing aspects of their provision including their governance, their admissions, the curriculum, the provision they must make for children with special educational needs, and so on. In academies, by contrast, the “rules of the game” in these areas tend not to be set out through acts of Parliament but rather through a funding agreement: the lengthy document which each academy signs with the Secretary of State for Education to allow the academy to qualify for state funds and thus to be established. Essentially, it is a contract between the trust running the academy and the Secretary of State. Why does this matter? Well, Mr Wolfe's central argument, I think, is that the rules as to what is to count as legal depend on the type of school it is and, in academies, on the contents of the funding agreement with the Secretary of State. With the academies sector growing, this is starting to have implications for large proportions of state-funded provision, mainly in the secondary sector. So, where once there was one body of law for the entire state sector, now what is legal depends on what is set out in each funding agreement, that contract between academy trust and Secretary of State. As Mr Wolfe put it to me: “I have spent the past 20 years doing education law. Before the advent of academies, if I was presented with a kid from Exeter, or from Birmingham, or from Newcastle, saying they had been excluded, for example, it would be the same response: is this in accordance with education law? “Now, my first question would be 'which academy?' and then I go straight to download the funding agreement. The idea that a child in Exeter should be subject to the same law as one in Newcastle does not apply now; it depends on the funding agreements which apply to individual academies.” This sounds like a lawyer's field day to me, and it seems Mr Wolfe would not disagree. He added that the fact that academies, unlike maintained schools, could no longer automatically rely on in-house legal advice from their local authorities could also make matters more complicated for them, as they might have to buy in legal support from solicitors charging by the hour. Aside from the practicalities for schools, this has serious implications. First, Mr Wolfe said it was impossible to generalise at all what the legal position on particular issues was with regard to academies as an entire sector. This was the case even with the Government's recent move to change the school admissions code which – in an aspect welcomed by critics of the academies policy –ministers said would ensure academies were now subject to the code. Mr Wolfe told the meeting: “The [school] admissions code applies to some academies. For the Department to say that it applies to all academies is simply…wrong. “Anything school-specific about exclusions, curriculum, admissions and so on…is covered in an Act of Parliament for maintained schools but that Act does not apply to Academies directly. It only applies if the academy's funding agreement says so. “The funding agreement might say: 'this academy will comply with the admissions code'. But it does not have to say that. And not all of them do.” In other words, the legal position will vary by individual academy. There are other serious implications, I think. First, if Mr Wolfe's view is correct, a principle of equity – that children should be treated equally whichever school they attend –may be being sacrificed in the claimed drive to provide greater freedoms to academies. In other words, the rules of the game – how pupils should be treated –may vary according to what is set out in the funding agreement. Mr Wolfe added that academies created recently under Michael Gove were – “to [Mr Gove's] credit” - tending to be established according to more of a standard funding agreement – which was actually closer to the regulations governing maintained schools – than was the case under Labour. But there were still some strange anomalies, he said. For example, in the latest Education Act, the government moved to change the rules on exclusions such that, in relation to maintained schools, appeals panels cannot now overrule the judgement of a head teacher to exclude a child. Mr Wolfe said that some academies said, in their funding agreements, that they would behave like maintained schools on exclusions, and thus that they would legally have to follow and abide by this change. But others said, in their funding agreements, they would have “old fashioned” appeals panels, as existed when the funding agreement was signed, where heads' judgements could be over-ruled. Thus, they would have to abide by these rules, which had now been changed in maintained schools. In this way, he said, funding agreements could essentially preserve the law in academies in line with what was agreed at the time, rather than being subject to change, as happened in maintained schools. Perhaps the most fundamental point he made, however, was in relation to the legal safeguards for pupils and parents in academies. Strictly according to the law, he said, these were non-existent. He said: “Academies are set up by a legal contract between two parties. [The academy trust and the Secretary of State]. They both have rights. No-one else does.” Parents and pupils' rights, in maintained schools, were subject to Acts of Parliament which, Mr Wolfe said, said “the governing body of a school shall do this, the head teacher shall do this”. Parents finding that a school had broken these rules could therefore bring legal proceedings. By contrast, academies could take a different course. Mr Wolfe said: “The strict legal answer from the academy is to say [to a parent who challenged them]: 'what is it to you? We have a contract with the Secretary of State, and you have no rights under that contract, so you can go and whistle.” (Note one). In practice, he said, he was regularly challenging academies having been approached by parents concerned about, for example, a school's approach to an exclusion, or to the curriculum. Although it could be possible for the law to be on the academy's side, in every case he had dealt with so far the academy had reached a settlement or agreement with the parent, perhaps, he said, because they did not want the courts to be scrutinising the detail of the legal framework. Mr Wolfe said he was handling about one case a week from parents in relation to academies. He gave an example of a recent case. A year 10 child had been causing “low level disruption” in his school. The school did not exclude him, but organised for him to go and “work on a farm” for one day a week, and then to “teach himself GCSE” at home for the other four days. In a maintained school, the school would have to write to the parent to explain why it did not plan to teach the child the national curriculum, and the parent would have a right of appeal. In this particular academy, the funding agreement said it had to teach the core subjects of the national curriculum unless the head teacher thought it was not appropriate for a particular group of children. This was an apparent get-out in this case, and there was no need to write to the parent to state what was happening. But Mr Wolfe said: “We threatened legal proceedings, and in the end they agreed to provide catch-up support [for this pupil].” Frightened about the prospect of adverse publicity, the academy was, then, having to make alternative arrangements for the boy. I must admit to being slightly puzzled as why this happened if, in theory at least, the boy and his parents may have no legal rights in terms of the academy funding agreement, but the practical implications for the school of a dispute also seem to be being taken into account here. In any case, the position for schools on this, then, Mr Wolfe said, could be very messy and the central problem was that there was not a process setting out the rights of parents and pupils: the key document was an agreement between the Secretary of State and the academy. All of this leaves me thinking that the title of the blog Mr Wolfe has recently set up on the subject of academies and the law – “A Can of Worms” – is probably apposite. There are ways of defending this arrangement. Politically, ministers will argue that the vehicle of academy status gives schools extra flexibility in the approaches they take to aspects of school life such as the curriculum. Legally, in terms of ensuring pupils' and parents' rights, I guess the Government would have to argue that the Secretary of State is a democratically-elected politician and will be…erm…the guardian of the public interest – and thus the parent and pupil interest – in negotiations with academy trusts over what goes in the funding agreement. But that seems, to put it politely perhaps, quite a long chain of causation or influence between the leverage voters have over any politician at the ballot box and the intricacies of what might be in an academy funding agreement. I checked with Mr Wolfe and it is still possible for the Government to include academies within the law governing mainstream state schools, on any issue the Government chooses. This would have the effect of superseding any individual academy funding agreement. But the Government would have to specify that academies were covered in any future education law, or else it would only apply to maintained schools; so academies are covered by their funding agreement unless the law says differently. The whole process seems potentially very messy, and also opaque. Although individual academy funding agreements (Note two) are available online, trying to make sense of the law as it applies to the hundreds of academies now in existence is now no longer, as it would be in the case of state schools, simply a matter of looking at and trying to understand what an Act of Parliament says. It is also difficult to imagine widespread coverage in the media – or a national debate – around the contents of individual academy funding agreements. This was brought home to me powerfully a couple of years ago, by the way, when I did try to look at what individual funding agreements said under Labour. Reading through these documents, I came across “side agreements” between the Secretary of State and a chain of academies, which seemed to be designed to lessen the short-term financial burden on a sponsor. These arrangements seemed to have received virtually no public or Parliamentary scrutiny or debate, despite the public money seemingly implicitly at stake. ( I wrote about this here: http://bit.ly/5mqHWN Note three) As more schools become academies, of course, a larger section of our system will be being run according to these piecemeal, largely unscrutinised, arrangements made between the Government and academy trusts. Does an education system run according to the contents of individual contracts between the Secretary of State and private organisations ring alarm bells for you? Well, it does seem rather a strange way of going about matters to me. A can of worms indeed. David Wolfe's website is: http://www.acanofworms.org.uk/ Note one: On his website, Mr Wolfe puts up what seems to me to be a contrast with the fee-charging sector. Although academies are said to be state-funded “independent” schools, the important contract is between the academy trust and the Secretary of State. By contrast, in conventional independent or fee-charging schools, there is essentially a contract between the school and the parent paying fees. Note two: But not, yet, it seems, funding agreements for “free schools”, which is another issue again, and outrageous given that public money is being spent and yet the public seems not to be being told either exactly what the rules are for its spending in these schools, or how much is being spent. Note three: I wonder whether the sums which sponsors appeared to pledge in funding agreements set up under Labour could now be legally enforced, even though the academies policy no longer requires a financial commitment from sponsors. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=510 Fri, 02 Dec 2011 14:25:44 GMT 20111202142544 Fri, 02 Dec 2011 14:25:44 GMT 20111202142544 Fri, 02 Dec 2011 14:25:44 GMT 201112021422544 It is one of the most important and far-reaching reforms currently under development in education in England. In June, teachers' subject associations were promised, in a generally well-received move by Michael Gove, that they would get a first glimpse of how it was shaping up by August. (Note 1) But what exactly has happened to the national curriculum review? This, remember, has been given the task of coming up with new curricula covering education right through from the start of key stage 1 to the end of key stage 4. Yet here we are, now in the second half of November, and there has still been no glimpse of early drafts of the new curricula in English, maths, science and PE, which are scheduled to start being taught in schools in less than two years' time. No-one within teachers' subject organisations seems to know, or have been told, the detail as to why there has been a delay in this first, unofficial, phase. Formal consultation for the curricula for these subjects is supposed to start in January, with first teaching from September 2013. There has been no indication that that official timetable has slipped – although one source now tells me the phrase is “spring 2012”, rather than January, for the start of formal consultation on English, maths, science and PE – but the mystery as to how the review is currently going is ringing alarm bells with some. I was unable, looking into this topic last month, to get meaningful information from the government on who had been involved in any drafting of detailed new national curriculum programmes of study, and even those who have at one stage been close to that process seem baffled now as to whether work carried out over the summer will end up seeing the light of day. The Advisory Committee on Mathematics Education's website makes clear that that organisation was gearing up to consult the maths community on draft Programmes of Study for the new maths curriculum over the period September 5th – 21st. This was part of what was being hailed at the time as an admirably transparent approach from the Department for Education. Yet the website has since been updated to say: “The process [as described above] will no longer be taking place on the timescale described. The Department has decided to delay this pre-consultation phase until the Programmes of Study have been developed to a more advanced stage. The intention remains to make drafts available to the community prior to the formal consultation in early 2012, but in order to make this process more meaningful it is expected that this will now take place later in the autumn . The overall timeline for the national curriculum review and its implementation remains the same, with formal consultation early next year. “The Department has asked ACME to pass on its apologies for any inconvenience caused.” I gather science organisations were then preparing to do a similar exercise with their members in October, only for, again, nothing to appear. Although civil servants have told them that the first versions of curricula are being held back until they are at a “more advanced stage”, there seems no more information than this. “We are in the dark. It's a case of not knowing, and that's the whole community not knowing,” Annette Smith, chief executive of the Association for Science Education told me. Another science source said: “The last I heard from anyone involved was the beginning of October. The whole thing is shrouded in too much secrecy.” It seemed as if individuals were being asked to write sections of the new curriculum, I was told by this source, but there was no communication around who they were. On September 29th, I asked the Department for Education who had been involved in working groups which, over the summer, had been given the task of detailed writing of new draft Programmes of Study. On October 3rd, I only got a vague answer back from the DfE. It said : “Since the summer, we have engaged with a number of experts - including practicing (sic) teachers and members of the subject organisations. These include: the Royal Society, the Advisory Committee on Mathematics Education, the Association of Science Education, the Institute of Physics, the Royal Society of Chemistry and the Society of Biology, the Science Community Representing Education (SCORE), the Association of Physical Education, and Sports England.” This, however, would seem to refer to general information-sharing events which took place in the spring and early summer, rather than the more specific business of actually writing the new curriculum. At that time, the DfE could not even tell me whether the new curriculum is to be introduced in all year groups simultaneously, or more gradually, with certain year groups going first. Ten days ago, I asked someone close to the review itself when we might expect to see early drafts. I was told that something would be available in about two weeks' time. Another subject association source said that delays might not be a bad thing: a danger was that reviews rushed towards implementation, and it was better to get any draft right – or nearly right – before releasing it than simply to put something out too quickly. However, rumours are circulating that the hold-up is at a political level, with ministers unwilling to sign off the recommendations of the curriculum review group. I wrote (http://bit.ly/qWs79B ) last month that some views of ministers – especially, according to most people I ask, the schools minister Nick Gibb – in support of the teaching of long division in primary school, seem at odds with the opinion of, as far as I can see, the broad spectrum of maths educators. Although ministers' enthusiasm for the teaching of systematic synthetic phonics is well-known and open, there seems to this observer also to be a hidden agenda around an encouragement of more traditional methods of teaching in mathematics. Earlier this month, Ofsted published a report of an investigation it was asked to make into the teaching of maths in “successful” primary schools, half in the state sector and half independent prep schools. This, says the report, was following a “ministerial request”. What interested me was that the inspectors seem specifically to have been asked exactly when and how these schools introduce “traditional” methods of teaching the mathematical functions, including long multiplication and long division. Now, I am not going to take sides in this debate – I don't feel qualified to take a view as to whether “traditional”or “modern” is best - but there is certainly a lively argument within the maths community as to whether these traditional approaches or alternative methods, which have been widely used in recent years, are more effective. (Note 2) If there is an agenda to promote a particular type of teaching, ministers should at least be upfront about it, set out the case for it and preside over an informed, transparent debate, rather than simply making moves behind the scenes. I've had another look at both the Conservative Party manifesto and the two coalition agreements, however, and there is no mention of the promotion of “traditional” maths teaching. If there is one, of course, it would sit rather oddly both with repeated assertions by ministers that they are giving teachers more freedom and with what I thought was the philosophy behind the review: that the detail of pedagogy will be off-limits. Another view that has been put to me is that problems have followed the decision to scrap the Qualifications and Curriculum Development Agency and have civil servants employed by the Department for Education, rather than subject experts working for the QCDA or its predecessor body the Qualifications and Curriculum Authority, providing the administrative infrastructure for the review. For all the initial commitment to openness of this review, and the willingness of the leader of its expert group, Tim Oates, to tour the country talking about it, this inquiry currently seems decidedly un-transparent. - Note 1: See Michael Gove's speech to the Royal Society: http://bit.ly/k6dZjb Note 2: As I said, there seems widespread agreement among maths educators against the teaching of long division in primary school. With regard to more “traditional” methods of teaching the functions such as multiplication, there is a split among the experts. Note 3: timetable: - August 2011: First unofficial drafts Programmes of Study in English, maths, science and physical education were expected to have been made available to subject associations. But they have yet to be seen. - “Early 2012”, according to the current timetable on the DfE's website: Official drafts in these subjects to be made available for public consultation. - Spring 2012: Ministers make final decisions on these subjects, and announce which other subjects are to be included in the second phase of the national curriculum review. - September 2012: New Programmes of Study in these subjects available in schools, to prepare for first teaching in September 2013.. - Early 2013: Official drafts in other national curriculum subjects, as part of the second phase of the review, put out for public consultation. - September 2013: Materials in these subjects available in schools. Meanwhile, teaching of new Programmes of Study in English, maths, science and PE begins. - September 2014: First teaching of new Programmes of Study in other national curriculum subjects. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=506 Mon, 21 Nov 2011 12:44:20 GMT 20111121124420 Mon, 21 Nov 2011 12:44:20 GMT 20111121124420 Mon, 21 Nov 2011 13:49:21 GMT 201111211314921 What is it with the TES and teacher unions? 10 days ago, my former employers had a cover which led on an article asking the question, above a picture of …ahem…a hearse, “do the unions have a future”? I think that was a strange question to ask, especially at this current moment in the history of the teachers' associations and the wider union movement. Therefore, I wanted to try to analyse and deconstruct the arguments in the coverage. The place of unions in schools, of course, is central to education in this country and I wanted to seek to make some comments as an observer. The coverage – the front page question was followed by a feature and also a leader column (http://bit.ly/ttUIVJ) written by Gerard Kelly, the editor –came despite a sharp rise in teacher union membership in recent years, as I discovered after looking at official data after reading the pieces. By my calculations, official, fee-paying union membership has grown by 10 per cent in the past seven years. Figures from the Certification Officer – which compiles the data on union membership, and was the organisation cited in the TES article, though not in this way – show total fee-paying membership of the six unions operating in England rose from 706,866 to 776,039 from 2004 (note 1) to 2010. This appears to be an improvement from a high base, too: in 2002, I wrote an article for the TES (http://bit.ly/vBLZwl) on how union membership was at that time at its highest level since the 1970s.(note 2). The TES feature article did admit, at its start, that “as the pensions row brings thousands of new recruits to the teaching unions, their future may seem secure”. But there was no sense, in the piece, that this growth in numbers easily pre-dated the current pensions row, or of the scale of the rise. The piece itself seemed to be listing four threats to the unions' futures, or to their very existence if you take the front page at face value. First, the rise of academies was putting at risk national pay and conditions arrangements. “The consequences are serious: an erosion of union leaders' power to negotiate on a national basis,” warned the piece. National pay and conditions were “historically a key raison d'etre for organised labour in schools,” it added. “National pay bargaining may limp on for a few more years but it won't be binding on large chunks of the system and it can't be long for this world,” the leader added. Now, of course it is true that academies do not have to abide by the national pay and conditions agreement, and that their numbers – in the secondary sector, at least - have grown “explosively” in the past year, as the leader says. It is also true that classroom unions have long feared that the academies policy may erode national pay and conditions arrangements in the long run. Yet what is the evidence that this is actually happening, at school level across the country? This is an empirical question, which as far as I can see the TES hasn't looked into in detail at the school level. It is certainly not hard to find academies offering salaries quoted with reference to national pay scales in adverts printed in recent editions of the TES. And looking into the funding of academies in the past six months, I have had academy heads tell me they won't touch teachers' pay and conditions for fear of the reaction among staff. How representative is that reaction? I don't know. I'm not sure anyone does. Lord Hill, the schools minister, was presumably so worried that there would not be big changes that he warned in January that schools signing up to national pay and conditions agreements could be stopped from becoming academies. (http://bit.ly/eDT5po) Classroom unions are clearly worried. They may not be inclined to take all academy principals' statements of abiding by national pay and conditions at face value. Or they warn that, when the principal changes, the policy might change. On the other side of this argument, government sources, and some academy principals themselves, have been known to suggest that pay and conditions can be higher in academies than in conventional maintained schools. Summing up, the actual future impact of the academies scheme on teachers' pay and conditions is, it seems to me, an open question. And unions' raison d'etre is not the existence of national pay structures, but to represent their members, in whichever way they can. Even in academies which do not recognise national pay and conditions, that will continue. Related to this, the piece says, as a second threat, that the rise of academies - allied to cuts in local authority budgets - is putting at risk “facilities time”, during which union reps are paid to carry out union business. This, clearly, would seem to be another challenge for unions. But a threat to their very existence? In what way? There's not much detail in the piece about the organisational challenges the unions would face. Third, the article cites a drop in influence with ministers. It says: “the fact is that, since the departure of the Labour ministers from Downing Street last May, unions' influence on politicians has dramatically dropped”. This is backed up by the leader. It warns: “The unions face a Government that is cordial but cool. Their clout is not what it was. The days when general secretaries wafted in and out of Whitehall are long gone.” It is hard to know where to begin with this. Some political context is needed: we have, of course, moved from a Labour to a Conservative-led government over the past 18 months. This might have something to do with this attitude shift among ministers. The implication of the leader is that Conservative ministers are neutral, dispassionate bystanders on the scene of union affairs, waiting to be impressed or not as to whether union leaders will act in the best interests of the country as well as of their members, and that approval from the Secretary of State – seemingly whatever his intentions for the schools system and for teachers' pay and conditions – is the criterion by which they should be judged. Again, this is a strange take on the way the relationship works. Of course, unions always have to decide how closely they want to work with government – when to rattle sabres, and when to seek to negotiate. And a case can be made that the closeness of some unions' relationship with ministers in the “social partnership” years under Labour would make things more difficult under the Tories. But that is very different from suggesting, as here, that the fact that a government is not supportive of unions is a failing in itself. Unions have to support their members. Their strategies in doing so will, inevitably and obviously, be heavily influenced by the actions of government. And there is little mention or analysis in this TES edition, as I want to go on to say later in this piece, of the substance and detail of the unions' current dispute with ministers over pensions. Any dispute carries risks on both sides, and that includes on the Government's. Ministers' actions are as worthy of detailed analysis as the unions' stance. The fourth threat to unions' futures, this piece argues, is the fact that they are not joining together as one. Well, the arguments for unity have been well-rehearsed, and to be fair, both sides of this debate are presented in the feature. The case for unity is that it presents a stronger negotiating bloc with government. The case against it is that having different associations represents healthy competition for the representation of the varying wishes of the teacher workforce. Whatever the merits on both sides of this argument, the fact is that teachers' unions have survived without unity for a long time, and seem numerically to be growing stronger. So this is not a threat to their existence, either OK, so I don't think any of the reasons cited suggest an existential threat for unions. If there is a threat to their existence in the current form – through, for instance, the rise of academies - I don't think a case is made here as to how exactly this means they have to change, or even as to the scale of the threat. If ministers are planning fresh moves to reduce the power of unions, we are not told about them. Teacher unions remain, as charts illustrating the feature show, far stronger numerically than they are in the rest of the UK workforce. The quoted 74 per cent of teachers “unionised” seems a huge figure. It would be interesting, actually, to explore the reasons for this difference with other parts of the economy. I would also like to read about the impact of unions at individual school level. But there are bigger issues here, too. First, it seems, again, odd that this question is being asked when the union movement would at least seem to be going through a period of strength, or at least of opportunity. As mentioned, the feature does highlight the fact that the pensions issue has gained unions members, but there is no acknowledgement that wider events over the last three years might have strengthened activism. It is currently not hard to find support, in glimpses at least in almost all parts of the media, for protest against the inequities of the financial system and wider economy. For example, the Financial Times regularly carries articles questioning the disconnectedness of rising executive pay, while the largely right-of-centre London Evening Standard runs columns taking the side of those camped outside St Paul's against the now shelved plans by the church to move them on. The rising inequality of recent decades also leads some to suggest this is linked to the decline of organised labour in the private sector over the corresponding period. And, of course, it was estimated that 250,000-500,000 people took to the streets of London in March to protest against the cuts. Activism, then, hardly seems on the wane. But the final “big picture” point is this: what about the substance of the unions' current dispute with ministers over pensions? For all the coverage as to the future of the unions themselves, where is the analysis of, essentially, the merits of their latest case? An issue of similar importance to, say, readers of the FT, would have been analysed to death by FT commentators as well as its reporters. There seem, also, to have been few human interest stories as to how individual teachers, and TES readers, are going to be affected by the pensions changes. Although the magazine has covered the intricacies of the pensions row in some detail, this approach seems to have been taken only occasionally. (See here: http://bit.ly/pzmDi8) Maybe the teachers who are angry about this central aspect of Government policy are wrong. Maybe their positions need to be tempered by a consideration of the overall impact of pensions provision on national finances. But I'd like to hear their voices anyway. Often, in the pages of the TES - although the magazine continues to break interesting, important exclusives which grapple with policy in ways often not offered by the mainstream dailies - unions can seem to be getting a tougher ride than coalition ministers. Given the scale of the changes being put through by this government, the lack of a detailed critique is surprising. - It was interesting to read last week that Michael Gove had told heads and local authority leaders to stop “whingeing” about a lack of resources and not to “reach for excuses” instead of getting on and improving their schools. (http://bit.ly/t7QZpa). Yet, as the latest set of economic growth figures were released a couple of days later, I found myself listening to the usual stock of “explanations” from Conservative MPs for the less-than-fantastic position of the economy over the past year. Chief among them were the “mess Labour left” (despite the fact that in 2007 the Conservatives signed up to Labour spending plans http://bbc.in/a0ohL8 , lest we forget) and the current problems in Europe. In recent months, we've also had slow growth blamed on the snow, on the Royal Wedding and on the Japanese tsunami. Reasons, or excuses? Accountability should work both ways. Note 1: 2004 was the earliest year on the Certification Officer's website for which I could find comparable figures for all the unions. By the way, DfE figures show teacher numbers in state-funded schools in England grew by three per cent over the period 2005-2010. With support staff numbers rising by 27 per cent over the period, the total growth in the schools workforce was 14 per cent. Note 2: These figures are not strictly comparable, as my reporting in 2002 was based on total membership, which includes non-paying members. The 2004-2010 stats are based only on members who boost union revenues. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=502 Mon, 07 Nov 2011 12:53:49 GMT 20111107125349 Mon, 07 Nov 2011 12:53:49 GMT 20111107125349 Mon, 07 Nov 2011 12:53:49 GMT 2011110712125349 I have been thinking a lot about the above questions in the past few weeks, spurred in part by the announcement of Ofsted's new draft inspection framework for 2012 onwards. There has been some excellently detailed coverage of this in the TES in recent months, with inspectors rightly challenged on the minutiae of how data is to be used to hold schools to account. But, reading the new framework and its accompany guidance to inspectors (http://bit.ly/ngjFLT), I can't help also wondering about the philosophical problems with Ofsted's official approach to education, and why these documents, supposedly talking about a sphere of life which should, surely, be about potential in its broadest sense and enrichment, come across as so frankly dispiriting. Unsurprisingly, perhaps from an inspectorate, part of the problem is the authoritarianism within these papers, I believe. But I think it is also to do with a misplaced sense of certainty and precision, and of a persistently narrow vision as to the purposes of education. I think I need to expand on this, and also to be a bit more specific. And predictably perhaps, I am going to link this discussion to the role of achievement data in inspection verdicts, as Ofsted sees it. What has got me thinking has been the at-face-value welcome pledge by Ofsted and ministers that the new inspection framework is to put more emphasis on the quality of teaching, as observed in the classroom, in judging schools. Launching the new framework on 30th September, the outgoing chief inspector, Miriam Rosen, highlighted the fact that inspection judgements on the quality of teaching and learning in schools would be one of only four main verdicts in each report, the others being pupil achievement; behaviour and safety; and leadership. This focus on a reduced number of inspection criteria – the current framework has three main judgements, and 18 sub-categories of judgement – would allow inspectors, she said, to spend “even more time” in classrooms observing lessons before making their judgements. This has been part of a move, I think, to address one of the criticisms that people including me have been making of the system since 2005: that its reliance on test and teacher assessment data – provided to inspectors before they reach the school – can lead to schools effectively being pre-judged before anyone sets foot in a classroom. Allied to that were worries about inspection judgements having been made on the basis of much less time spent in classrooms than used to be the case pre-2005. (Note 1) I think the move to emphasise judgements on just four aspects, all of which will be easily understandable to the public, looks admirably clear. However, dig a bit deeper beneath the surface of Ofsted's comments, and it becomes apparent that this is still a very data-oriented system. Consider the way the judgements work. The judgement for both “teaching” and “leadership” are both clearly also linked to pupil results. They even risk being defined by them; the extent to which they are will depend on the actions of individual inspectors, I suspect. To consider “teaching” specifically, the evaluation schedule, which provides guidance for inspectors ,s ays that “the judgement on the quality of teaching must take account of evidence of pupils' learning and their progress”. (The highlighting in bold is Ofsted's). The obvious evidence of pupils' learning and progress will be assessment results. There is also a section of the evaluation schedule on “grade descriptors”. This provides guidance to inspectors as to what outstanding, good, satisfactory or inadequate teaching in each school will look like. Against “outstanding”, it says: “Much of the teaching in all key stages is outstanding and never less than consistently good. As a result, almost all pupils are making rapid and sustained progress.” The grade descriptors for teaching continue in a similar vein. For example, against “satisfactory”, the grade descriptor begins: “Teaching results in most pupils, and groups of pupils, currently in the school making progress that is broadly in line with that made by pupils nationally with similar starting points.” Qualitative judgements of teaching quality, then, are expected to be reflected in statistical measures of pupil progress. Against “inadequate”, three of the four bullet points – in illustrating how good teaching should be judged - seem to be referenced against measures of children's achievement: either raw attainment data or progress measured. I would argue, then, that although inspectors are being encouraged to spend more time in the classroom observing lessons before reaching their judgements, a or even the key part of the evidence will continue to be pupils' test and teacher assessment results. Similarly, under leadership, against “outstanding”, the grade descriptor begins: “The pursuit of excellence in all of the school's activities is demonstrated by an uncompromising and highly successful drive to strongly improve achievement, or maintain the highest levels of achievement…” In her speech, Miriam Rosen said: “The new framework will focus on how school leaders are improving achievement for pupils by helping them to overcome specific barriers to learning.” It might seem difficult, then, given the above, for a school without good results to gain a grade of having good leadership. It goes without saying that the judgement for “achievement” will focus on pupil results. This means that for three of the four judgements in each Ofsted report, and for the overall verdict, pupil results are going to be central. Why does this matter? It seems to me that it does for two reasons. First, the suggestion and implication which is clear within the inspection framework and guidance is that teaching quality will always be reflected in good results seems to me to be, while powerful politically, not a reflection of the reality. Remember that, in the guidance to inspectors, it seems to me that there is the suggestion of this one-to-one relationship: good teaching, as observed in the classroom, will be reflected in good outcomes (Note 2) for pupils. Good results will follow good teaching. Or the reverse. Am I over-interpreting what Ofsted is saying, here? I don't think so. Look, again, at those inspection “grade descriptors”. Satisfactory teaching “results in most pupils, and most groups of pupils... making progress which is broadly in line with that made by pupils nationally with similar starting points.” The quality of teaching will simply be captured by the quality of the pupils' results, the inspectorate is saying. I think the faith in this one-to-one relationship is probably mistaken. In the real world, I suspect, children's motivation will rise and fall over a period, some children will be more motivated than others and everyone will have periods when they make leaps forward in their learning and times when they plateau or even slip back. A child with good results might owe them to the good quality of teaching in their school over the immediate period for which they are being assessed, to outside factors such as support from their parents or, in some cases, from tutors, or to the groundwork done by teaching earlier in their schooling. A group of pupils taking national test results might benefit or lose out from lenient or harsh marking, remembering that each set of tests is marked by a single individual. This complex relationship between the quality of input – teaching, in this case – and pupil outcomes was discussed in a fascinating speech I came across a few weeks ago by the assessment specialist Wynne Harlen. This speech (http://bit.ly/qU0kXn) was made way back in 1978, at a seminar on accountability at the British Educational Research Association. Wynne was discussing the merits of systems of accountability based mainly on what I might call “inputs” – looking at the sort of activity or transactions that went on in a school, with the goal of improving pupils' learning – and “outcomes”: the notion of judging schools by their pupils' results. Both had their strengths and weaknesses, she said, but judging schools solely on outcomes, even when outcomes are couched in such general terms as “what children [actually] learn”, could be problematic . She said: “The notion that 'what is happening in a school' the transactions, can be measured by what pupils learn is a simplistic one. It assumes that what is taught well is learned, that what is learned has necessarily been taught in school and that what is not learned has not been taught or not taught well.” In other words, that relationship between good teaching and good outcomes is, inevitably, complex, however much we might like it not to be. (Note 3) The complexity is not reflected in Ofsted' s world, though, where the implication is that good inputs always lead to good outcomes and, perhaps more to the point, good (or bad) outcomes are always a reflection on the quality of the inputs. I think Ofsted's system, if more effectively designed, could actually help us to understand the relationship between quality of input and quality of outcome much better than it does now. It could have separated out inspectors' judgements on the quality of teaching – ie the quality of the input – from pupil results – the quality of the outcomes. In the same way, there could be a separate judgement on leadership, without considering results at all. Such an approach would have to be carefully designed: inspectors would have to reach a judgement on teaching quality by, for example, classroom observation, talking to pupils and looking at the quality of their work without considering the children's assessment results, and give a verdict for “teaching” before having seen the data. “Achievement” would simply be recorded separately, and then these judgements looked at holistically before reaching an overall verdict on the school. Such an approach might be more tricky for “leadership” – many in the system now might argue that the use of data is such an integral part of leading a school that it is in reality difficult to separate the two – but it would be interesting to explore. Coming back to the judgement on “teaching” however, separate judgements for “teaching” and for “achievement” might help us to explore the relationship between input and outcome. Inspectors might even have to explain why they reached different verdicts on the two, which would be no bad thing, in my view. We might find that my suspicion discussed above, that there is not or not close to a one-to-one relationship between good teaching and good outcomes, was not true, of course. But the key thing is that we cannot check. This approach is not possible, because the assumption of a one-to-one relationship between quality of teaching and quality of pupil outcome just continues to underpin Ofsted's systems. Some inspectors might challenge that view: they might argue that, for the teaching judgement, if they saw evidence of good teaching and yet the pupils' results did not back it up, the school could still receive a good verdict on teaching. Yet I wonder, given the fact that numbers can seem to be more “objective” than qualitative judgements, how often this would happen. If you view the path from good teaching to good results as a collaborative venture between teacher and pupil, with many possible slips along the way, including inevitable error margins around any assessment system, I think there is the following implication. There is uncertainty in this process: a teacher might worry that good teaching might not result in good outcomes, through no fault of the teacher. But Ofsted accepts no uncertainty, or complexity in the relationship between teacher input and pupil outcomes, beyond its acknowledgement that a pupil's starting point will influence his or her attainment at any one point. Uncertainty is simply something the teacher has to deal with, rather than the inspection system to accept, manage and possibly investigate. The system is authoritarian, then, and in that sense may feel oppressive, in effectively seeking to deny uncertainty in the link between good teaching and good results, making it a problem with which the teacher, rather than the inspection system, has to grapple. Those questioning this approach can even be made to feel that it is Ofsted, rather than them, who have children's interests at heart. The obvious response from Ofsted is that this tough approach from the inspectorate and others with power over what goes on in schools is entirely appropriate: test and exam results just are important for young people, and we should never shirk from emphasising this fact. The system, then, needs to focus on outcomes achieved by pupils. Yes, there may be uncertainty, but we need teachers to try to negate it: to do all they can to squeeze it out of the system. But England's system has been taking a hard line in emphasising results, almost as ends in themselves, for 15 to 20 years now. The pursuit of results in this way, encouraged by our accountability system over that time, has been of questionable success: although the picture with regard to the evidence of different international testing systems is mixed, rising national test and GCSE scores are often not replicated when alternative ways of measuring our system are used. This is crucial, because the narrowness of focus on particular test and exam measures needs to lead to improvements on other measures to show that underlying skills and understanding have actually risen. Indeed, it is possible to view one of the often-reported anti-educational downsides of England's system – teachers spoonfeeding or guiding pupils towards test answers – as a logical response from many within the profession. It could be seen as the need to take more control over the process of “delivering” results to pupils by squeezing uncertainty out of the process, given that uncertainty itself, in the passage from good teaching to good results, is what the system wants to deny. This narrow focus on maximising outcome measures as ends in themselves, then, needs to be questioned. My second point is that Ofsted's new measurement system continues to reinforce a very narrow view as to the purposes of education. Consider, again, the judgements which are central to the inspectorate's new regime. Pupil results seem to be the key not just to the “achievement” judgement, but to “teaching”; to “leadership” and then to the overall judgement. It is as if the system is telling schools: you need to raise your pupils' results. And you need to raise your pupils' results. And by the way, in case you've forgotten, you need to raise your pupils' results. This intensive focus on assessment scores is reinforced by recent Ofsted pronouncements. In its inspection framework document, it says, simply and without discussion: “The most important role of teaching is to raise pupils' achievement.” No debate there, then, it seems. And, in her speech, Miriam Rosen said: “Our key challenge is to continue to raise pupils' achievement, achieve better rates of progress and secure higher standards of attainment for all pupils.” Well, no-one would advocate lower standards or poorer results, but is this really all that education comes down to? Ofsted seems to be proposing: learning's central purpose is to raise the numbers. I look at this, as someone who has loved learning throughout my life – and has never come close, by the way, to capturing its potential – as something close to tragic. Given the huge scope and breadth of human knowledge and understanding in the 21st century, why do we limit our ambitions of education to watching the percentage of children meeting Government expectations inch upwards, or not? It does not sound to me to be either a great advert for summing up what schooling could represent for children, or for attracting talented people into teaching. I think the inputs – the richness of the learning experience on offer for children, not just the results at the end of it – matter. Ofsted's systems are, of course, not marginal to what goes on in schools, but very powerful drivers. Many schools, I am sure, resist the tendency towards reductionism implicit in its measurement mechanism with its mutually reinforcing judgements but they will do so in spite of it, rather than because of it. There are many calls for a proper debate on what education should be for, and this underscores for me the need for Ofsted's systems to be aligned with what the profession and the public view as schools' overall purpose. Results are a part of that, but we need to ask the extent to which our young people are actually served by their pursuit almost as ends in themselves, which is continuing under this government. I think Ofsted's ultimately life-sapping – because it is so narrow - underlying philosophy (Note 4) compares badly both to alternative visions of education articulated, for example, by inquiries such as the Cambridge Primary Review and the Nuffield Review of 14- to 19 education, and also to frankly more sane inspection reports I read regularly on international schools abroad. The take of the new inspector, Sir Michael Wilshaw, on all of this will be interesting. (Note 1): Probably because Ofsted has been under pressure to cut costs since at least then, I don't think inspectors will be spending as much time in lessons even under the new regime as they did pre-2005, but I may be corrected on that. (Note 2): I should state, here, that when I say “good outcomes”, the inspectorate is not talking solely about “raw” results. Its inspection system now gives at least as much scope for the progress pupils make between key stages, when deciding whether outcomes are good, as to simple measures of a child's attainment against national averages. Yet the underlying point remains: the inspectorate is judging schools on the assumption that good teaching will always be reflected in positive assessment results, following that teaching, from the children. Even measures which include children's starting points will fail to capture uncertainty in this process for teachers, including the differential support pupils are receiving in the home. (Note 3): Wynne Harlen's argument might look almost quaint now, given the often unquestioned “outcome” focus of the current system. But I do think it has some power. If you are trying to measure the quality of an “input” in the process of helping children to make progress in their learning, such as the quality of teaching, then the outcome at the end of the process may give clues as to the quality of the input. However, it would be strange to think that, if you were seeking to judge the quality of the input, all you needed to do was to look the outcome at the end. Or even that you could rely on outcome measures as your main form of “objective” evidence as to the quality of the input. To put it another way, as argued above, a system which did this would assume there would be a perfect or one-to-one relationship between the quality of input at the start of the process and the quality of outcome at the end. Real life being what it is, however, this is unlikely to be the case. Alternatively, advocates of outcomes accountability might argue that, in fact, this does not matter: in education the measured outcome at the end of the process is all that is important, and therefore the system should not really be too worried about trying to ascertain the quality of the “inputs” at all: good teachers and good schools are, by definition, those where pupil outcomes are good, so long as those outcomes are defined appropriately. I think that, despite the fact that the inspectorate is now putting more stress on lesson observation, in the end this must be close to Ofsted's view. (Note 4): I should say that I have one reservation when it comes to this view: I have always had a lot of respect for Ofsted reports on the quality of the teaching of subjects across the country, probably because they talk in detail about the substance of what is being taught, alongside results, rather than being entirely dominated by statistical considerations, as I think is largely the case with individual school reports and with the overall inspection framework. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=498 Mon, 24 Oct 2011 14:10:42 GMT 20111024141042 Mon, 24 Oct 2011 14:10:42 GMT 20111024141042 Mon, 24 Oct 2011 16:29:46 GMT 201110241642946 “Roll up, roll up: get yer academies and free schools here”. A slightly facetious start to this blog, admittedly, but this is the impression I get when clicking on the Department for Education's homepage. This could be seen as the online focal point for all of England's 22,000 schools; a chance for a government to offer parents an attempt at a genuinely objective, public-spirited guide to our education system for 0-18-year-olds in its entirety; an opportunity for ministers to showcase what we know about how teachers can build children's understanding; or even – who knows? - a gateway to professional development opportunities for teachers and heads. Instead, its main function seems to be as a selling platform for the current pet schemes of politicians: academies and free schools. In that sense, instead of being viewed as the flagship site for our schools, it should be seen as a national embarrassment. On Thursday, I clicked on the home page to find my jaw dropping as, top of the list of press releases and therefore in a prime position on the site – searching for something else on the site, this was the first thing my eye rested on - was a release published on Tuesday proclaiming the benefits of academy status in the light of statistics showing that more than a million pupils are now being educated in academies. This was in line with comments at the Conservative party conference, made by Michael Gove on the same day, that there are now 1,300 academies. The only immediately visible paragraph of this press release, which is still on the homepage as I write, reads as follows: “Nearly 1.2 million children in England now attend academies – schools with the freedom to meet the needs of their pupils, rather than answering to local or national politicians and bureaucrats.” It is hard to know where to begin with this sentence. So, 1,300 schools are now academies. That leaves more than 20,000 state-funded schools in England which are not. On what is to all intents and purposes the official homepage of English state education, then, all these other schools are being denigrated, since, by implication, the DfE is presenting them as not fully “free to meet the needs of their pupils”. This is mindblowing stuff from a department supposedly – at least notionally - charged with supporting all state-funded schools in doing the best for those they educate. The “free from answering to local or national politicians or bureaucrats” slogan, though powerful politically, also strikes me as bizarre to the point of incoherence when it features on the website of the education department. For if ministers and civil servants are genuinely concerned about freeing up schools from the priorities of “bureaucrats” and their political masters, why are they bothering to spend years investigating and implementing a new national curriculum, for example? Surely, by this logic, this is getting in the way of schools' being “free to meet the needs of their pupils”. Why are ministers so keen to insist that schools teach synthetic phonics? Why is at least one minister thought of as such an enthusiast for pushing, within the national curriculum review, for traditional methods of maths teaching to be used in primary schools? These plans are being given high priority by politicians, presumably with the prospect of being implemented by “bureaucrats”, with the result that what goes on in schools which are not academies – still the overwhelming majority at primary level, and currently the majority at secondary – will be heavily influenced by them. The reality is that there will always be a balance to be struck, in state-funded schools of all kinds including academies, between the need of the central state to have at least some oversight over what happens, and the ability of professionals to exercise autonomy. How that balance works will always be subject to debate. Crude caricatures such as the one shown here are as dishonest as they are internally contradictory. There is also, of course, an Alice in Wonderland quality to a press release which proclaims the ability of some schools to break away from the needs of politicians and “bureaucrats”, given that this very release was presumably penned by “bureaucrats” working to a highly political agenda presumably laid down for them by the political masters. But the surreal quality of this website does not stop here. Top left is a statement which begins commendably simply: “The Department for Education is responsible for education and children's services”. Excellent. Unfortunately, the next and only other sentence reads : “Find out about the academies programme [the bold type is, of course, the department's] which provides schools with greater freedoms to innovate and raise standards.” This, then, is the two-paragraph summary of what the education department - funded by your and my taxes, remember – is supposed to do. Top-centre of the homepage, in a large window, currently sits a rotating series of images flanked by a static screenshot, the top one of which is “2011 free schools”: an invitation to click on “a short film about free schools”, which looks like an advertising video, set over soothing music, for this small number of institutions. Immediately below this advertising space is the heading “schools”. The following types of school are listed: academies, free schools, “technical academies” and university technical colleges. All four of these presumably (I have to confess “technical academies” is a new one on me) are types of schools backed heavily by ministers. Visiting Martians might wonder why some schools are felt more worthy of ministers' and public attention than others, but this is the position. For astonishingly, there is no mention, then, under “schools”, for all these other institutions which still make up the overwhelming majority in England, including community, voluntary aided and voluntary controlled institutions. The political spin goes further. On the homepage, on “schools”, and then on “types of schools”, you might think that there would at least be a guide, perhaps for parents wanting information, on England's exceedingly complex array of different varieties of institution. But instead, on this page the first linked document reads “How to Apply to Open a Free School”. To be fair, on this page there is a link to the “directgov” website, which, after another click, does offer a more comprehensive list of types of schools, although bizarrely this offers no mention of free schools. But that's not within the DfE website itself. This would be comical in its politically-skewed crudeness, if it were not so depressing. The department clearly sees its job not as one of providing a neutral or inclusively supportive gateway for information on all schools, but of manipulating opinion – including, presumably, professional opinion - towards a particular political vision as to how schools should be organised. Yesterday, I conducted a brief tour of education department homepages around the world, and it is difficult to find any that is politicised in quite this way. Look at the French education ministry's website (http://bit.ly/pK2ET5) and you will find that, though ministers are prominent on the homepage, single clicks flagged up from there take you to detailed factual information on how the state education system is organised, with separate pages for “schools”, “colleges” and “high schools”. In Germany, education is the responsibility not of the federal government but of individual states. I looked on the website of the largest state, Bavaria, and although its homepage( http://bit.ly/nD1Tss) does feature press releases prominently, a click on “school types” offers information couched in neutral, seemingly informative, terms. The Swedish and Finnish ministries also seem straightforward. Interestingly, in Australia, where education again is still primarily a state responsibility, I clicked on the website of Western Australia (http://bit.ly/eKEZkp ) , which seems to have its own academies-style policy of “independent public schools”. But this is given less prominence on the website than central images of “good news stories” of achievements of pupils and teachers in all types of schools on the homepage. There are a couple of links here to a page on the “independent public school” policy, but the information you find is couched in fairly neutral terms, with none of the denigratory language towards other schools you find on the DfE website. Closer to home, the education websites of the Scottish, Welsh and Northern Irish governments are also seem to be mainly about providing information on the system as a whole. Arguably, only the American department for education website rivals our own for politicisation but again, the role of the federal government there is different from our own: the states themselves remain responsible for schools. I clicked on one, New Jersey, (http://1.usa.gov/gvbvqy )and found a small box highlighting “governor's reform agenda”, but the main banner headline being the neutral “Choose the Best Education Possible for Your Child” My personal favourite homepage at the moment is the New Zealand education ministry's website (http://bit.ly/9Hhxsx ) which is simple, clean, emphasising the provision of information for parents and with media releases promoting particular policies kept quite discreet, in small type on the right hand side. Of other UK departments, there seems to be a bit of a contrast: while I found departments such as that for Communities and Local Government, and arguably Business, Innovation and Skills, relatively politicised, those for HM Treasury and the Department of Health seem to keep the politics very low-key. I have raised this and the related issue of the sometime politicisation of the Department for Education's twitter feed (which has in the past been guilty of highlighting positive news and highly political comment articles about ministers' reform plans, but strangely not the more critical ones), and sometimes I hear the argument coming back that this is what we should expect: Government departments are necessarily political. The Department for Education press office have tried to tell me that it is the duty of civil servants to promote government policy, and thus that my view – that there should be a concept of the public interest beyond the interest of elected politicians, and that at least one aspect of the public interest is clearly for the department to support the education system as a whole – is in some way strange. Quaint as it may seem, I cling to this view. The fact that almost all of us pay taxes to fund the whole of our education system, including this website, and schools of all types, suggests to me that ministers should treat them in an even-handed way. I don't think it is too much to ask for information to be presented with at least the pretence of even-handedness, rather than skewed information – let's call it as it is: propaganda - being put out as it is now. Once again, I don't think I am being old-fashioned in saying that the purpose of state-funded public service websites should be to help the users of these services and the general public, rather than servicing the needs of ministers. This website just underscores, in fact, despite the politicians' claim to be getting out of the way in the running of schools, just how pervasively politics infects our system. It is not too strong to call it embarrassing. The concept of an education service to be run in the public interest, rather than to serve the requirements of politicians who are always, these days, elected on well under half of the national vote, seems more essential than ever now. Finally, there is some history to all of this. The academies policy, because it has been pushed so hard now by successive governments and has always been, in that sense, a top-down initiative, has never been presented honestly by the education department. Having covered the policy in detail for getting on for 10 years now, the bias in official press releases in favour of academies – and thus to the disadvantage of other schools – has been obvious. It was certainly not unknown, under Labour, for press releases of GCSE exam results to go out highlighting the fact that those of academies were improving, on the headline measures, faster than the national average and that academy results represented improvements compared to the schools the academies replaced. Yet basic statistical caveats – such as the fact that there were no checks on whether the characteristics of the pupil body in academies compared to their predecessor schools had changed – were never included. Under Labour, the department also routinely presented the schools that academies replaced as “failing”. Yet, when we investigated this on the TES back in 2005 ( http://bit.ly/qdNF8p), I found that at that time none of the predecessor schools had, actually, been failing in the strict, Ofsted definition of the term. Again, the department was unhappy, accusing me of pedantry. But the difference between a school with low results and one which is said to be “failing” is important, not least for the staff who once worked in the schools replaced by academies. It suited ministers to ignore any caveats and make these schools look as bad as they could. It would be nice to hope that things would have changed under a new government. But the bias remains: witness Michael Gove's highlighting of the importance of his new English Baccalaureate measure, and how pupils from disadvantaged backgrounds need access to English Bacc subjects, but then failing to mention the poor results on the EBacc of many academies set up under Labour. (http://bit.ly/oAKNtd ) It might be, of course, a bit too much to expect the Education Secretary to preside over a system which is even-handed with evidence when he also talks about the need to respect what international studies tell us about the state of English education, but fails repeatedly to mention results from an international testing study which places our schools' progress during the Labour years in a relatively favourable light. (See http://bit.ly/gam61I) Ironically, of course, all this might be seen as evidence that it is good to separate politicians from education, as the press release implies. Maybe if politicians and civil servants actually worked harder to promote the public interest, rather than their own interests, we would have a better way forward. -If you still have the will for more after the above rant, I've been looking at the recently-published evaluation (http://bit.ly/oyc2pK) carried out for the Government by academics at Sheffield Hallam University, of a pilot of its proposed new phonics check for six-year-olds. This is due to take place for all Year 1 pupils at the end of this academic year. The report, based on a survey of the 300 schools which carried out the pilot, and follow-up case study visits to 20 of them, offered some positive findings which were highlighted, alongside some caveats, in the TES. The stand-out figure for the TES was the fact that almost half of the schools (43 per cent) found that the check had “helped them to identify pupils with phonic decoding issues that they were not previously aware of”. This is important, of course, and, although the rest of the schools (strictly: 55 per cent) appear to have found the check of little use, the fact that a large number felt it would be at least some help may well be significant. However, the possible benefits of this check have to be weighed against the findings in the round, which are far from unequivocal. For example, the study found that “the majority of teachers in case study schools had faced difficulties in judging whether a word had been read correctly or not with some of their pupils”. While three quarters of survey respondents felt that the check accurately assessed phonic decoding ability overall for their pupils, fewer than half said they thought it did so for English as an Additional Language pupils (46 per cent); for those with speech difficulties (35 per cent), with special needs (33 per cent) and with language difficulties (28 per cent). For the check, children are expected to say some real words which are presented to them in writing, and some “pseudo words” or “alien words”. Some but not all of the latter were flagged up in the check for children with a picture of an alien against them, and it seems from the report as if the two types of words were mixed up in terms of the order they were presented to children. The majority – 60 per cent – of the surveyed schools felt that the “pseudo words” caused confusion for at least some children. Although evidence from the survey and case study interviews suggested that for most pupils the experience of the check was “positive” – with many children professing to enjoy the check - around a quarter of those surveyed felt “the experience was negative for pupils with speech and language difficulties, other SEN or weak phonics skills”. Most – more than 80 per cent – did not tell parents the check was taking place, partly for fear of making children anxious. There were also comments from teachers that the timing of the proposed check, in mid- to late-spring, was not ideal, partly because if it were earlier in the year, they could use the information it provided more formatively, to tailor their teaching towards' pupils needs, a fairly basic requirement, you would have thought. These findings suggest, to me, lots of issues to be ironed out with this check before it goes “live” in only just over six months' time, despite the positives of the report, which also included teachers largely finding the training went well. There are a couple of more things to say. First, the schools which took part in this pilot were not experiencing the test being used for accountability purposes. The fear must be that this might create problems, in terms both of anxiety for staff and possibly therefore pupils and washback effects on teaching (teaching to the test). A Government document before the pilot (http://bit.ly/oRBZXJ , page 6) said: "an accurate evaluation of the potential impacts of the phonics screening check will not be possible" during the trial, for this reason, with the impact in this important sense then only being monitored when the check went live. Yet the new Ofsted framework makes clear that the check's results are to be taken account of by inspectors (see http://bit.ly/nFvddG page 7) as part of evidence in relation to pupils' reading. Clearly the potential influence of the accountability structure on this test is the elephant in the room, yet uninvestigated in the pilot. Second is the issue of cost, both in financial terms and in terms of time. Research (see John Hattie's study of education initiatives around the world) suggests most interventions with children have some good effects. But are they cost-effective? This report offers no direct answers, since there is no estimate of the likely overall cost of this test within it as far as I can see. But it appears the time commitment for schools is likely to be larger than the Government was expecting. The report says 99 per cent of survey respondents reported spending more than the two to three minutes per child on the test that the DfE training manual indicated was the expectation. (Tsk: those “bureaucrats”, getting it wrong again…) For 53 per cent, it took more than seven minutes per child, while for one in seven schools, it was more than 10 minutes per child. So, in terms of total time per school spent on the check, the average was 12.5 hours in terms of administering it, while they spent three hours preparing for it. That looks like quite a large time commitment to me: 12.5 hours alone looks like around half the teaching week for the member of staff administering the test, and I think and some schools – most of which used their class teacher to administer the test – raised the issue of having to provide supply cover. Back in the summer, when I wrote an article about the check, the DfE told me they would be able to provide figures on its likely cost after the pilot evaluation was complete. So I asked the press office this week. Given the above, I was very surprised to learn that the likely figure the department now has for costs is that it will be only “in the region of £1-2 million to deliver”. That equates to only £2 to £3 per pupil in England, an amazingly small amount, I think. It appears, though, that this does not include staffing costs to schools: when I asked the press office, I was told this relates to “costs to the DfE”, and no further figures were currently available. The pilot schools were each given £250 towards the cost of “preparation and administration time”. But it appeared some struggled, even with this extra funding. The report concluded: “Many teachers spoke about the financial implications of the phonics reading check where funding was spent on organising supply cover to enable the lead teacher to be away from the classroom.” One teacher said: “The funding wasn't great, it didn't even touch it.” If the check costs £250 per school to administer, that equates to a larger national cost figure, of the order of £4 million, which presumably is in addition to the DfE's in-house costs for developing, monitoring and distributing the checks. The report says it is unlikely that “ringfenced” funding will be provided for this new check, meaning that schools will have to find ways of administering it within existing resources. Many will use teaching assistants to administer it, some teachers predicted. In these straitened times, then, the most relevant question is whether this is the best of school funds and, perhaps most to the point, of staff time, even if there are benefits in terms of information provided for some pupils. The concerns of one survey respondent should be taken seriously. They said: “Overall, the cost of running the phonics check is disproportionate to the outcome…it would be more efficient for the DfE to promote a list of alien words and new activities/resources that teachers can use to strengthen their current phonics teaching and assessment. This would remove the need for a very expensive formal test.” That's only one view, of course, and it's still possible this new check will prove a success; certainly the pilot evaluation was not damning, and £250 per school could still be seen as a relatively small figure. But it is an initiative to watch closely, I think. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=495 Mon, 10 Oct 2011 11:33:43 GMT 20111010113343 Mon, 10 Oct 2011 11:33:43 GMT 20111010113343 Mon, 10 Oct 2011 11:33:43 GMT 2011101011113343 On Friday, September 16th, a very big claim was made on the front page of the Times Educational Supplement. “Decimate to accelerate: how dealing with the bottom tenth of teachers would improve education out of all recognition” read the headline. Beneath it, puzzlingly, was a diagram of 10 eggs, one of which had cracked open. Inside, one would believe, would be the details on how one seemingly simple reform could do what our education system has arguably never achieved before: moved an established system forward in a – for want of a better phrase - “step change” of improvement. Not only is it debatable whether this has ever been achieved in the English context before; a paper I read recently mentioning international test study evidence suggests it has rarely if ever been achieved anywhere, at least in maths (Note 1). This was quite a scoop, then, for the TES as it relaunched itself as a magazine. Turning to the inside pages, it emerged that the evidence for this bold assertion was a paper just published by the Sutton Trust, the think tank which campaigns for more young people from disadvantaged backgrounds to go to the most prestigious universities. The TES feature quoted its most eye-catching conclusions in a bit of detail, before moving on to quote union leaders on whether there really were lots of underperforming teachers in English classrooms, and a general discussion about how easy it is for schools to rid themselves of classroom staff who are not felt up to the job. But nowhere was there a discussion of the validity of the claims in the Sutton Trust research itself. This is surprising, given not just the confidence with which they are put forward in the paper but the in-some-cases revolutionary and certainly controversial pay and conditions proposals the paper makes on the back of the evidence and arguments it marshalls. I want to try, here, to look at that evidence, and then suggest – deep breath - six major problems with it. OK, the central argument argument of the paper - by the London School of Economics academics Richard Murphy and Stephen Machin “advised” by Eric Hanushek, a high-profile American education economist based at the Hoover Institution at Stanford University – is that we could greatly improve the performance of the English education system if we could just raise the game of those teachers who perform the worst. The TES said the study suggested that 64,000 teachers were “not performing as well as they should”, and that replacing or improving the performance of the lowest-performing 10 per cent – some 40,000 – would improve England's position in the main international testing study dramatically. Not only that, but the Sutton Trust paper offered some quantifiable financial returns to improving teacher performance, in terms of future earnings returns to individual pupils. Bringing a “poorly performing teacher” up to the average, and therefore improving the grades of the class he or she taught, would raise the lifetime earnings of a class of 30 by £240,000 to £430,000, as employers rewarded these achievements through higher salaries. Wow: perhaps more than £10,000 each over a lifetime, then. It also quoted evidence suggesting that teachers with better qualifications on entry to the profession – such as better, or higher, degrees – tended to achieve not much better test and exam grades for their pupils than those with less good academic cvs. Finally, it used this evidence to argue for some very radical policy changes, the most interesting of which were making all teachers' salaries much more closely linked to pupil test results, and secondly introducing a system in which all staff would be given the chance to opt for a “high-stakes” pay route, which “rewards high performers with extra pay and opportunities for faster career progression, but penalises under-performance”. Coming across this, I suspected a number of what to this observer look like dodgy assumptions and overblown conclusions. It is a pity that studies like this are not better founded, and that they are not handled, published (Note 2) and disseminated more responsibly, especially given the debate they inevitably provoke, entirely misleadingly I think as will become clear, about the overall quality of the profession. Here are six reasons for scepticism: 1 Very big claims are being made on the back of what are, in fact, small-scale studies, some of them quite old now and at least one –arguably the most influential to this paper's arguments – of which itself warns against the dangers of over-interpretation. The fact that the studies on which its claim are made are small-scale and in the main do not relate to the UK is not made clear in the report for readers. The first bullet point in the Sutton Trust paper's executive summary says: “The difference between a very effective teacher and a poorly performing teacher is large. For example during one year with a very effective maths teacher, pupils gain 40 per cent more in their learning than they would with a poorly performing maths teacher.” Again, this is a very bold claim, also reported in the TES. And at face value, it's persuasive: if the evidence suggests this sort of effect for more effective teachers, who could possibly argue with it? But what's its source? Four studies are referenced in the paper in relation to this paragraph. These seem to be part of a small but, to this observer, still relatively undeveloped field which has sought to quantify teachers' “effectiveness” through value-added test data. The only study which seems to be the basis for the claim about the effects of good maths teachers, seemingly put forward in the report as universally true, is the first one listed in the note to this paragraph: work carried out by Daniel Aaronson and Lisa Barrow – both from the Federal Reserve Bank of Chicago – and William Sander, from DePaul University in Chicago, which I've now read. This looked at maths test scores of 53,000 American ninth grade pupils in one school district of Chicago in the years 1997-1999. Although the model includes a lot of statistical checking, in essence it compared the results of these ninth-graders with those they achieved in eighth grade, producing “value-added” scores for each child. It had records of who each child's ninth grade teacher was. It then concluded that (Note 3), if you looked at teachers whose pupils had average value-added and then compared them to those with significantly higher scores, teachers with the higher scores would see their pupils making 20 per cent more progress over the year than those at average level. If you compared the teachers with the high VA scores with those with significantly below average scores, you reach the 40 per cent difference quoted here. Three other studies are referenced in the note to this paragraph and elsewhere in the Sutton Trust report, but none of them seem to reach this precise finding. So that's it: one study, not in England but in a part of one US city more than 10 years ago, has been used to justify this seemingly all-embracing claim. Only one of the other papers referenced in the note to this study was carried out in England, and helpfully it also includes a discussion of the three other papers referenced in the note, all of which were carried out in the US and all of which it appears to be critiquing as being imperfect. The English study itself was conducted by Helen Slater of the (UK Government) Treasury and Neil Davies and Simon Burgess from Bristol University. It found that teachers with higher value-added GCSE scores could contribute 33 per cent of a GCSE point in extra achievement to their pupils, based on comparing the GCSE results of 7,305 pupils in just 33 secondary schools in the years 1999 to 2003 with their results at Key Stage 3. No mention of maths specifically there, as far as I can see. And, amazingly, this research also includes the proviso: “there is no presumption that the sample is representative of all English secondary schools”. These don't look exactly like universal findings to me, despite the undoubted statistical expertise of the modelling in the two studies I've read. Yet the paper reports as “fact…the difference between good and bad teachers is very large”, as the basis for its radical and largely unqualified policy proposals. Interestingly, the Chicago study itself says that moves to use its findings to link teachers' pay to performance “would require serious attention to implementation problems…including, but far from limited to, important measurement issues associated with identifying quality[among teachers]. The English GCSE results paper also includes the following warning that the “folly of basing important decisions [ie, on teachers' pay and progression] on the small samples of such data in a single school need always to be borne in mind”. These caveats should be remembered when we come to the policy proposals in this paper. 2 Problems with the value-added model of “teacher effectiveness” itself, and the…highly questionable [ polite phrase] move to make links to England's performance in the OECD's PISA tests. OK, a bit of technical detail is needed here, although I am no statistician myself. All of the above studies, including the Sutton Trust paper itself, define “teacher quality” or “teacher effectiveness” entirely in terms of “value-added” test and exam statistics. Now, of course, I have problems with that, given that most people would say exam results are only a part of what a good education should be about. But leaving that aside, the model itself is interesting, strange and limiting, I think. I think a reasonably scientific way of trying to find a link between “good teaching” and “a teacher who achieves good exam results for his or her pupils” would be to analyse the relationship between two independent variables. For example, it is possible to think of an experiment where, say, observers with no prior knowledge of a group of teachers' results were asked to look at evidence of the quality of their teaching – most obviously, by observing them teach - and to rate them in terms of effectiveness. This could, perhaps, be supplemented by seeking the views of the teacher's pupils on their quality. Perhaps, then, teachers could be grouped into broad bands and defined as “good quality”, slightly less good, and so on. It would then be possible to look at the results achieved by pupils taught by each of these groups of teachers, and then to try to reach conclusions on the strength of the relationship between this first measure of teacher quality and the second, seemingly independent, variable: grades. (Note 4) This model could be subject to debate, but it would seem to me to be capable of identifying some interesting patterns: is our measure of quality captured by results, or not, and if so, why not? But this is not the methodology used in these studies when they seek to reach conclusions about the “quality” of individual teachers. Instead, they define this entirely and self-referentially in terms of test results. Basically, the studies look at a group of pupils' test results at a particular age, before they were taught by a particular teacher. They then compare the results of that teacher's pupils after the teacher has taught them. Some of them seek to take into account extra influences on a pupil's performance in his or her exams, such as social backgrounds. What is remaining, unexplained in the model as a result of other possible factors such as pupil prior achievement or backgrounds, is then said to be the “value-added” to each pupil's grades by the teacher. In other words, teaching quality in these studies is defined as that which doesn't seem to be explained by other factors within the model. A “good teacher”, then, is one who produces good results for which there doesn't seem to be any other explanation, meaning the obvious explanation is good teaching. A “bad teacher” has correspondingly below-average results. Teachers reading this will realise that that probably doesn't work with, for example, single pupils or even single groups of pupils over a single exam year. Any pupil – really, any group of pupils - can over- or under-perform on the day, meaning that a teacher's VA results for that pupil may, in reality, say more about random variation than any qualities within the teaching. And, despite the respect I often have for the work of individual examiners and the expertise within exam boards, I've reported on too many cases of schools with clearly very unusual test and exam marking to be sure that, if a teacher's grades are above- or below-average, it is the teaching rather than the exam marking which explains the difference. Claims of the reliability of the VA model are hardly helped by the finding within the above-quoted Chicago study that, while those teachers who had the highest VA scores tended to remain fairly stable over successive years of this research, those with the lowest scores changed dramatically from year to year. Does “teacher quality” really move around in this way? Finally, the VA model may suffer, I think, because, although these studies look at pupil progress over a particular period of time where the pupil might have one teacher for a particular subject– key stage 4, for example, or grade nine in the Chicago study – it could be argued that the work of teachers earlier in the pupil's school career , and indeed, of the school as a whole, is contributing to this progress. So to what extent is a good, or bad, VA score really the teacher's own, rather than that of his her colleagues? Ironically, the findings that are more robust, I think, in the papers cited by the Sutton Trust report are those which found very little link between a teacher's qualifications, and – indeed – experience after the first few years in the job, with their pupil's exams results. This is a more solid finding because it involves two independent variables. To put it another way, it is possible to check whether pupils taught by a teacher with good qualifications, for example, achieve better grades than those without. It is not possible to check in this way whether “good teachers” as defined in these studies always get “good value-added exam results” – or whether, for example, some uncertainty in the examination process or other factors means this sometimes or often does not happen – because “good teachers” have simply been defined as those who get “good value-added results”. This circularity in the definition is vital when we come to the way the Sutton Trust paper tries to use the findings from the VA studies it cites to make conclusions as to what the impact on England's performance in PISA – the comparative reading, maths and science tests now taken by pupils in most countries – would be if we could just find a way of improving “underperformers”. The Sutton Trust research, it seems to me, just looks at England's results in PISA and then asks how they would improve if the “lowest performing” 10 per cent of teachers, on value added measures, were to improve the quality of their teaching so that their pupils' results improved enough to make these teachers now average performers. “All other things equal” (a clue that there are some underlying assumptions at play here), the paper says that our position in reading would improve over 10 years from 21st to 3rd in reading, and from 22nd to 5th in maths, making England one of PISA's star performers. But because “teacher quality” is defined entirely in terms of test results, this is akin to this country saying: if we could just replace the worst 10 per cent of this country's results with more of its average results, England would be in such a better place. That's a great aspiration, but I'm not sure it takes us very far. It's a bit like saying Manchester Utd would be an even better team if they could just replace their two lowest-scoring players with two who went on to score more goals in games, or that if Lewis Hamilton could just find a way of speed up a bit on his slower laps, he'd be more of a match for Sebastian Vettel. It may be that you believe it is the bottom 10 per cent of teachers, as measured by VA scores, that are holding this country back. But I don' t think this data gives much that is new to say on that front. Yes, it would be better if all pupils were better taught. It might be good if they went on to do better in PISA. No, I don't think this gives us information either on how many are badly taught, or, anything on how we improve the teaching of the “worst” teachers. It's simply an attempt to back an argument by doing a bit of number crunching. There is, of course, another elephant in the room with this PISA claim. If teacher “quality” did indeed improve, on GCSE-based VA measures, that would be reflected – all other things being equal – in improving national GCSE results. As these rose, then, in line with VA improving, so too must the PISA scores, must be the claim. But GCSE results have been rising incrementally for many years (Note 5). As I wrote here, http://bit.ly/pZpBZg) headline GCSE five-plus A*-C rates increased 25 percentage points over the years 2001-2010. PISA data were broadly flat over that period. GCSE results, then, would suggest teachers have become more effective. Yet England's PISA scores, actually, have not risen to reflect this. 3 There's another technical problem, I think, with one of the assumptions behind the PISA data. It assumes that pupils benefit from “an effective” teacher – ie one that has good value-added test scores – not just in a single year when they are taught by that teacher, but potentially for many years afterwards. It seems this assumption is needed to generate the seemingly big predicted improvements in England's PISA results which are central to the paper. It says: “It assumes that…the impacts of good instruction are cumulative over the 10 years of education prior to PISA testing”. In other words, a five-year-old taught by a “good” teacher 10 years before taking the PISA tests at 15 would still be feeling the benefits of that quality when taking the PISA tests. Another “good” teacher as they get older will add to that effect, and so on. Now, don't get me wrong: of course we would all at least hope that good teachers can make a long-term difference; this must be the core assumption of the job. But I'm not sure we can make quite such a big assumption as is made in the Sutton Trust survey: that all of this measured benefit persists over time. I recall seeing research – for which sadly I can't find the reference now - by academics at Durham University's Centre for Evaluation and Monitoring which showed that, I think, while the assessment gains a reception or year 1 teacher makes with children can last throughout primary education, the effect tends to diminish (though not to nothing) over time. In a book published last year Diane Ravitch, (Note 6) the high-profile US educationist and teacher champion who has argued against the simplicity of similar statistical interpretations used in America, cites research which found, she said: “Learning gains do not persist over time. Students forget, gains fade….After two years, unless there is continual reinforcement of learning, only one-eighth of the gain persists”. I don't have any way of checking those stats now. But it seems to me that the Sutton Trust paper's assumption is likely to be, at least, highly contentious and worthy of further investigation. 4 “Blimey, are we nearly there yet?” I hear you ask. Almost. I'm not going to spare too much space going into the Sutton Trust paper's claims of pupils' future financial earnings being boosted by improving the “effectiveness” of their teachers. It should be clear from the above that there are already a large number of probably contentious assumptions behind these precise-sounding numbers. But I would just say a couple of things. First, if a pupil did have the earnings of his or her class boosted over a lifetime of £240,000-£430,000 thanks to the effectiveness of a teacher, it could be argued that's not actually a huge sum. So, to take the higher figure of £430,000, first you have to divide it by 30 to get the claimed financial gain per pupil. That gives you £14,333 over each pupil's working life. That sounds quite significant, but if you assume a working life of 40 years (I've not been able to find what the actual assumption behind this stat is; hands up: maybe I didn't look hard enough), it's only £358, in today's prices, per pupil per year. Better than a poke in the eye, you might say, and perhaps if a pupil had several very “effective” teachers for each GCSE class, then it would mount up. But these are the sorts of numbers we are talking about. The bigger point, however, I think is I would question whether the research on which these figures are based takes into account any effect of improving results not being rewarded so much by employers if there is a general national improvement in results. In other words, results may only be valuable to pupils if they improve faster than competitors in the job market. If grades rise nationally, will the financial returns from any individual's better results continue? It's not clear to me that this paper considers what seems to me to be this potential issue, but, again, I may need to look harder in the references. 5 Almost finally, I have concerns around the policy prescriptions set out, again very forcefully, in the paper's conclusions. These include the fact that, shockingly I think within the very thin 11 pages of text in this document, no extra empirical evidence seems to be being offered as to whether these policies actually are likely to lead to the improvement in teacher effectiveness the paper says is clearly so vital. The paper says that, because of the evidence it outlines, “major reforms are needed to the performance and pay system for teachers, with assessment based on three core factors: improvement in results in the classroom, reviews by headteachers, and external appraisals”. Because of what the paper has shown, then, we need a sharper system of payment by results, with “previous qualifications, previous experience, or years spent teaching…given far less significance”. As should be obvious from the above, although the findings on there not being strong links between teacher qualifications and the results their pupils achieve are interesting, as a whole the limitations of the evidence outlined above undermine the boldness of this claim. In perhaps the only major concession to sceptics, the paper does, to its credit, acknowledge limitations with making value-added test scores the main judgements of teacher effectiveness, including that this would encourage…ahem: another major elephant here… teaching to the test. Thepaper attempts to get around this by proposing making test scores only one of those three aspects of teacher evaluation. Fair enough, except that, in reality I think, test data is likely to have a major effect on the other two aspects: evaluations by head teachers and in external appraisals, unless this is specifically guarded against in ways which, I think, would be very difficult to imagine (ie the head making an appraisal decision without knowing a teacher's test scores). I have seen heads advised to take teachers' test scores into account in performance appraisals; hence my scepticism here. So this would essentially be a payment-by-results system in which value-added data would play at least a very major, if not the overwhelmingly important part. The paper goes on to suggest a “high-stakes” option, in which teachers would be able to progress very fast up the pay ladder if they could demonstrate consistently good test scores, but be penalised – perhaps, although it is not spelt out clearly here, actually losing money – if their pupils achieved bad scores. It says: “We believe that if teachers were given the option of a more flexible promotion and pay system, it would have the potential to attract and retain more high quality applicants into the profession”. For a sceptical view of whether such moves – and the increased pressure on “underperformers” in particular – would actually improve recruitment and make the profession more enticing for would-be talented teachers of the future, see the debate within the American context (similar calls have been made by Eric Hanushek in the US) here : http://bit.ly/gnfFZD (Note 7) That's another long blog, but it highlights another huge question implicit within the Sutton Trust proposals and associated claims in the US: even if you could identify and get rid of “underperforming” teachers, how can anyone be sure that those replacing them will be any better? This leads on to the larger point: the quotation above is just an assertion. Where is the empirical evidence to suggest what is being advocated here will work? Has it been tried before, and if so, what were the results? As far as I can see, no evidence is presented here. If it hasn't been tried before, would it not be a good idea at least to suggest a trial first? I can think of other questions which could at least be investigated empirically. How quickly, in reality, do teachers in England actually progress up the career ladder? Do many of them feel held back by the current system? Would it put them off entry if they had to enter teaching again, knowing what they know now? Would the system proposed appeal to them, as young teachers? (To me it might seem a bit daunting, but then again, I'm not a young teacher…) I'm not saying that all of the answers that might be generated from teachers would necessarily carry unfavourable implications for this study. It would just be nice to be presented with some evidence, one way or another, if we are really contemplating such a big change. 6 And finally: there is the big picture. And here I'm afraid I'm going to have to come back to the TES interpretation. The article says: “The [Sutton Trust] study suggests that some 64,000 teachers working in England's schools are not performing as well as they should”. It then highlights the report's claim of the benefits of “replacing or improving just the lowest 10 per cent of those working at the chalkface – roughly 40,000” teachers, before asking “If there are 40,000 sloshing around the system, what's to be done about it?” In its puff for the article on page 3, the TES says: “Forget Chris Woodhead's 15,000 underperforming teachers – according to the Sutton Trust, there are 64,000 working in England today and they are stopping our education system from rivalling the best in the world. What should be done about them?” This is a variation of a statistical fallacy I'm finding increasingly often in education. It runs along the following lines: x per cent of schools/teachers/education authorities/whatever have results which are worse than others. Therefore (is the implication your average reader is being invited to draw, I fear), the whole system has problems/is rubbish. But the data is actually self-referential: any system measured in this way will always have a certain number of below-average performers,and that's a function of mathematics, not “low expectations” or anything else. So, actually we're being told nothing about the system as a whole. This fallacy can be very useful to anyone wanting to suggest standards are not high enough. Coming back to this particular example, I have to say this is a strange interpretation of the data. Where does the 64,000 teacher figure come from? This is clear, I think, from the Sutton Trust report. It bases its calculation on the difference in maths performance by pupils taught by a “very effective” and a “poorly performing” teacher (the calculation used in the Chicago study), by looking at the performance of teachers at the 84th percentile, and at the 16th percentile. Those at the 84th percentile are at the boundary of the top 16 per cent “most effective” teachers. Those at the 16th percentile are at the boundary of the bottom 16 per cent “most effective” teachers (ie they are in the 16 per cent “least effective” performers). There are just over 400,000 teachers in England, the paper assumes. And lo and behold, 16 per cent of 400,000 is 64,000. So the key thing here is that the 64,000 lowest “performers” on value-added data in England will be defined entirely relative to other teachers within England. No matter how high teaching standards are overall, there will always be 10 per cent of teachers at the bottom of the value-added pile, and because 16 per cent of 400,000 is 64,000, that number will, all other things being equal, always equate to 64,000. To put it another way, every teacher in England could be a terrible performer, desperately in need of being eased out of the profession. Or none of them might be. This data tells us nothing about which of those statements is true, or more true, since however high overall teacher quality is, the bottom 16 per cent will always be the bottom 16 per cent. Of course, the researchers could have chosen another cut-off point. It's quite likely that, if they wanted to look at the “performance” of teachers at the 49th percentile, they would find all of these had below average VA results (I say only quite likely because of them might actually be at the average). 49 per cent of 400,000 is 196,000. Would this then lead to suggestions that nearly 200,000 teachers are not much good, and “sloshing around the system”? Again, it may be the case that they are not very good, or they may be brilliant, but just with not quite as good results as others. The point is: this data provides absolutely no evidence on overall teacher quality. Similarly, the 40,000 figure quoted in the TES comes from the Sutton Trust's decision to focus hypothetical PISA calculations on the idea of improving the scores of those teachers with the 10 per cent lowest value added measures. And what it shows is…40,000 is 10 per cent of 400,000. That's it. This would be laughable if it weren't so potentially serious. The final, final thing to say about this paper and its interpretation is that it certainly helps feed a narrative that says that, in general, the teaching profession is not very good and that the challenges it faces can be solved easily by getting rid of those who are simply not up to the job. (Note 8) Our state schools, or at least a large proportion of the teachers who work in them, are not very good, is the implication. Is that a harsh interpretation? Well, in an online response to that US blog I linked to above, Hanushek himself seems to take issue with the teaching profession, questioning why good ones would ever worry about an analysis which says that their under-performing peers have to be “dealt with”. He says: “I…do not understand why the vast majority of hardworking and able teachers are willing to be lumped together with the small number of truly ineffective teachers.” A teacher responds: “Because, there, but for the Grace of God, go I….We see through the smokescreen and know that the data is faulty and not a true measure of a teacher's worth”, before claiming that these studies are part of a move to “dismantle” the teaching profession and privatise schools. Do we really want the polarising US education debate in this country? It seems this paper is an attempt to move more in that direction. Whatever the truth of claims about the numbers of “underperformers” and the need to replace or retrain some of them, this debate has to take place on a fair, evidence-based footing. This brief, unjustifiably sweeping paper offers anything but that approach, I'm afraid. I worry both for the state of education research and for the future of the teaching profession in this country if this, really, is the best that we can do. Note 1: A paper presented at the British Educational Research Association's annual conference by academics at King's College, London, includes the following paragraph: “In other countries there is little evidence of great change in mathematical standards over time. In the international tests even though the rank orders may shift slightly over time there are no examples of countries making radical shifts.” Note 2: The Sutton Trust study is said only to be an “interim” paper. Maybe the final version will be better grounded in evidence. If so, I wonder why this version has been publicised now. Note 3: I do simplify here, of course. If you want the full paper, with a lot of statistical analysis, it's here: Note 4: The model would need to use some kind of “value-added” methodology – such as the progress pupils make over a particular period, rather than raw results – so the different results did not largely reflect pupil backgrounds. Note 5: Interestingly, GCSE scores have been going up in recent years despite KS2 data for the same pupils having remained largely flat since 2000. This would suggest KS2-GCSE “value added” measures, if calculated against a baseline of several years back, have gone up nationally. Note 6: “The Death and Life of the Great American School System: how testing and choice are undermining education” by Diane Ravitch. Note 7: Thanks to @tothechalkface on twitter for this reference. Note 8: The TES article appears dismissive of union claims that there are not a large number of bad teachers out there, claiming that in the light of the Sutton Trust report's evidence, that many [unspecified] people find the union response “risible”. But there is actually evidence that many parents are impressed with what happens in their child's school: in a little-reported stat, Ofsted's annual report last year found 93 per cent of parents agreeing with the statement: “I am happy with my child's experience at school”. Another survey (http://bit.ly/p6Ya78) of more than 1,000 parents of 3- to 16-year-olds in July found 92 per cent were either “very satisfied” (55 per cent) or “somewhat satisfied” (37 per cent) with the quality of teaching at their child's school. I know that leaves eight per cent not very satisfied or not at all satisfied, but it is important to get these figures in perspective. The Sutton Trust paper is here: http://bit.ly/ov1EUH http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=491 Mon, 26 Sep 2011 10:00:39 GMT 20110926100039 Mon, 26 Sep 2011 10:00:39 GMT 20110926100039 Tue, 27 Sep 2011 15:43:51 GMT 201109271534351 It couldn't last, could it? After, in my last blog, suggesting that one particular criticism of league tables – that they explained the large recent fall in pupil numbers taking French and German GCSE – was overblown, I'm back on more familiar territory this week. In the last couple of weeks, I've come across information which somehow offers fresh, and to this already battle-weary observer, highly concerning insights into the way school-by-school results pressures appear to be impacting negatively on pupils. Despite years of trying to track the multiple ramifications of all this stuff, even for me this subject retains its capacity to shock. Exhibit number one was a Government survey of 692 secondary schools about the impact of the English Baccalaureate, the new league table performance measure for English schools. http://bit.ly/pv01oM This poll, conducted for ministers by the National Centre for Social Research, was greeted enthusiastically by the Government, after its results appeared to show a big rise in pupils lined up for entry to Ebacc subjects following the announcement of the new measure last November. Yet, on closer reading I found two statistics which appear to me to be absolutely outrageous, in the evidence they put forward of schools seemingly putting their own need to raise performance statistics - arguably not surprisingly, given the pressures on the leadership of these schools – above pupils' needs. Until someone puts forward an argument to convince me otherwise, I'm coming to believe that what league tables (and other performance pressures) have helped to create is an inherently corrupt system, with dire implications in certain circumstances for individual pupils. The BBC report of this survey (see http://bbc.in/qofBUI ) , a link for which was tweeted by Michael Gove's education adviser Sam Freedman, said it showed the number of pupils who would be taking EBacc subjects would rise from 22 per cent to 47 per cent by 2013. Mr Freedman and Government supporters would argue that this vindicates the decision to introduce the EBacc indicator, which ranks schools on the proportion of pupils achieving A*-Cs in English, maths, two sciences, a language and history or geography. But the first shocking statistic was that the survey found that the proportion of pupils due to take EBacc subjects in 2012, ie at the end of the academic year which has just begun, was 33 per cent. Without wishing to bombard you with too many numbers, that's 33 per cent compared to a figure for pupils taking EBacc subjects in 2011 of – according to the BBC - 22 per cent. That means the numbers taking EBacc subjects, if this survey is correct, will have risen by 50 per cent between 2011 and 2012, seemingly as a result of the EBacc change, since 33 per cent of any number is 50 per cent higher than 22 per cent of that number. Why does this look outrageous? Well, children taking GCSEs next summer are in year 11 now, and would have been in year 10 last year. But the Government only announced the introduction of the EBacc in November. So these numbers, if they have risen as a result of the EBacc announcement, mean tens of thousands of pupils were switched to EBacc subjects months after their key stage 4studies and GCSE courses had started. As has been pointed out to me, this does not necessarily mean that a pupil was stopped from doing a course they had already started, simply because of this new performance table indicator. That will have happened, clearly, in some cases. But it may also be that young people have begun EBacc subjects in addition to those they were already taking. Whatever the precise circumstance in each case, large numbers of pupils appear to have been steered towards starting EBacc subjects months after the normal start of these courses simply because ministers have made sure that results are now to be collected and published at the school level. I had already heard anecdotal tales of some schools choosing to take actions affecting their pupils mid-course following the EBacc announcement (see below). But these statistics suggest these are not isolated instances; they might have affected more than one in 10 of all English teenagers now currently in year 10. Assuming this survey – and the BBC's interpretation of it – is accurate, it is fairly easy to check the numbers. Assuming, very roughly, that there are 600,000 pupils in a year group, and 22 per cent of them took EBacc subjects in 2011, that equates to 132,000 taking the subjects this summer. 33 per cent of 600,000 is 198,000 due to take them in 2012. The difference between the two figures – 66,000 – is the number of additional pupils seemingly now lined up to take EBacc subjects in 2012 compared to the numbers who sat them in 2011 and therefore the number who, given the assumption that without the EBacc announcement only 132,000 would have taken EBacc subjects in 2012, have actually started taking EBacc subjects mid-course. Could this interpretation be wrong? Well, of course we have to take the survey's accuracy for granted. However, it is stated to have been a properly representative poll. It should be pointed out that while the survey report itself does mention the 33 per cent of pupils expected to take EBacc subjects next year, I cannot see any reference within it to the 22 per cent “baseline” number: the proportion of pupils who took Ebacc subjects this year, which seems only to appear in the BBC report. In any case, we won't have a true indication of the final possible impact of the EBacc on subject numbers until next August's GCSE results, which will give definitive data on individual subject entries. Until then, this appears to be the most detailed indication of possible trends. It could also be said that any rise in pupil numbers is not proof of the impact of the EBacc announcement itself. But I can think of no explanation for such a dramatic apparent shift. Meanwhile, anecdotes have reached me of individual school decisions. One, from a parent in February, said: “At my children's school, year 10 students studying photography, art and design, drama have been 'ordered' to switch to history to ensure they gain the EB. They have missed a term's work and have no interest in history having deliberately not chosen it. Some very unsatisfied young people and parents.” Another, sent to me on twitter, also seemingly from a parent, said: “This happened at my son's school. Kids made to pick up lang[uages]/ hum[anities] and finish IT course early”. Another, from a school improvement adviser, said: “[I] know of at least one school where kids have been changed from BTEC Science at [the] end of Year 10 to double science [GCSE] in year 11; in no-one's best interest.” Schools changing their minds mid-course on what is in the “individual's” best interest simply because ministers have made this announcement on a new performance indicator is as scandalous as it is depressing, in my view. Perhaps a school could argue that this new measure will be so important to the child's future that it justifies the clear disruption these kind of moves will cause. But that looks to me a fairly thin defence. Otherwise, this looks a clear case of schools feeling they have to jump, at no notice, more or less whatever the implication for the child, because of ministerial announcements and the likely feedback effects on the institution of low scores on this new measure. That is a very poor place for an education system to find itself, whatever the underlying explanations, again in my view. The second example is, I think, even more shocking evidence of results-driven perversity. The same survey asked schools whether some year nine and 10 pupils would complete some of their GCSEs early. Four out of five (81 per cent) said this was the case. Then the survey asked them whether they would allow pupils a resit if they took a GCSE early and gained a grade C. While 59 per cent said they would always allow pupils a retake, 35 per cent said this would only be allowed “sometimes” and six per cent of schools “did not allow pupils who got a 'C' grade at early entry the chance to retake”. That last statistic is, from my perspective, simply staggering. Six per cent of schools would not allow any pupil who had already achieved a C in a subject the chance to re-take that exam in the hope of obtaining a higher grade. The hypothesis must be that the C grade has become so important for the school that the incentive, from the institutional point of view, is simply to take the pupil off the course once this is achieved, so that they can concentrate on other subjects where it has not yet been attained. I suppose it could be rational for a school, faced with a limited budget, to prioritise resits targeted at the “most important” grade C. oHoHoweHoweverers However, better grades than a C in a subject clearly have currency for young people. It seems to speak to a madness at the heart of our system not only related to results pressures themselves, but to the fact that while the accountability regime has tried to tell us that the C grade is all-important actually GCSE exams themselves were set up with the intention of using the full spectrum of grades, from A (now A*) down to G. There was never any intention that there should be an absolute pass/fail cut off point where a C grade becomes so important that one could sacrifice the chance of higher in one subject in order to try to gain a C in another. Many have countered that the C grade is the crucial cut-off among employers. But even with that argument, I'm afraid I remain to be completely convinced. Teenagers attaining higher than C grades are likely to have more options later in life, and the fact that some schools operate blanket policies hardly suggests individual needs are being put first. An influential report from the Advisory Committee on Mathematics Education (http://bit.ly/kkY5YU) this year suggested early entry for maths GCSE – with schools hoping to “bank” the C for the pupil early and then concentrate on other subjects for that individual – was more prevalent in schools on the Labour Government's National Challenge programme: ie those under most pressure to improve headline results. This makes sense, with those schools told they had to improve on the five A*-C grade including English and maths, or face closure. Although that particular scheme has closed, the coalition is persisting with this approach, and raising Labour's floor targets. The third piece of evidence came my way in my role, over the last month, as press officer for the British Educational Research Association's annual conference, which took place in London last week. Several papers, and a keynote speech by the crossbench peer Baroness (Onora) O'Neill, suggested a collision between the demands of test- and exam-driven accountability and what subject experts saw as good teaching in their field. For example, a paper on English teaching which I saw found that most of the teachers surveyed loved literature but that teaching to exams got in the way of cultivating that enthusiasm among their pupils. This is in line with a tendency I have found throughout my years researching and writing on this subject: while political debate and school improvement/management research often focuses on the claimed benefits of a “relentless focus” on raising results, subject experts chart the side-effects and detailed implications for teaching for long-term understanding and engagement; ie the quality of pupils' actual experiences of the subject. Few quantitative researchers, who tend to be in the former camp, seem to want to genuinely engage with the arguments of those in the latter, even though this is at the heart of what it means to be truly educated. But my particular focus here is a paper by Birendra Singh, of London's Institute of Education, which looked in detail at how two secondary schools had reacted to the Labour government's decision, under Ed Balls, to scrap Key Stage 3 science tests in 2008. (Maths and science tests were scrapped at the same time). The paper sets out some trenchant criticisms of the teacher assessment approaches instigated in these two unnamed schools – one rated “outstanding” by Ofsted, and the other “good” – as Sats were abolished. Though obviously tiny in scale, in my mind it has to raise concerns about exactly how teacher assessment is implemented against the background of a high-stakes accountability system, and in a context where schools have become used to externally-marked tests. What also interested me here was that at least one of the schools had reacted to the scrapping of KS3 Sats by starting out on a three-year GCSE course, and thereby reducing Key Stage 3 from three to two years, because all the “league table” pressures were now focused on performance at 16. Mr Singh and a science teacher from one of the schools whom he interviewed clearly have reservations about that, from an educational point of view, in that too little time had been given to embedding the Key Stage 3 programme and, in Mr Singh's words “pupils were too young to appreciate starting GCSE in year 9”. I know there is a debate about the benefits of a two year key stage 3 for pupils. But my concern would be that schools are not taking decisions on this from a neutral perspective, in terms of simply being able to think about the effect for the pupil. They will also be conscious of their own need to improve GCSE results at an institutional level and this, as Mr Singh clearly believes, is not always consistent with building true subject understanding in pupils. The paper also sets out how 40 per cent of pupils in both schools were “pushed” towards BTEC courses in science because they were deemed to be equivalent, in league table terms, to two GCSEs at C grade. All candidates were expected to pass, even though many would not have been expected to gain a C grade in actual GCSEs. In perhaps the most damning passage in this study, teachers are quoted as saying that it did not matter that pupils in lower sets had done badly in science at key stage 3, as they were lined up to take the BTEC course – where all were expected to pass - rather than GCSEs. In other words, that pass result at the end was all that mattered. I include this next bit boldly – perhaps bravely – in a blog for head teachers, but a teacher emailed me a couple of weeks ago to argue that when people look at what has gone on and pin the blame on “league tables” for some of these problems, inadvertently they let school management off the hook. On the issue of the decisions made by schools with regard to pupils' English Baccalaureate choices, she said: “I'm really not sure why everyone is so surprised that schools are moving their children to EBacc subjects mid-stream. After all, head teachers have been allowed to play these kind of games for years under the guise of 'league table pressure'. “Guise sounds like a strong word to use, particularly when we know that league tables are unhelpful in this situation. But we must remember that most of the schools that play these games are not under threat of being put in special measures/closure/or imminent redundancy. They just want to see their schools, not their students, appearing to be very good.” I include this with little comment, other than to say that I make no apology for highlighting incentives which appear to have been created, at a system level, for schools to act in certain ways. Whether one chooses to blame individual heads or the system as a whole, the problem, I think, has been the over-emphasis of school-by-school results as – in the reality of the way those schools and their leaders are judged in the outside world – ends in themselves. Changing this would be a huge undertaking. The Government will say, of course, that it is addressing these concerns by ending modular GCSEs and thereby seemingly removing schools chance to “bank the C” early, and by abolishing multiple GCSE equivalencies for non-GCSE qualifications. But this idea of gearing education to the maximisation of performance indicators for schools will continue to distort behaviour. Instead of league tables being seen for the flawed and blunt instruments that they are, Government policy seems increasingly to be being built around them, while Ofsted inspections still give great weight to institutional indicators almost as ends in themselves and, as mentioned, Labour's “improve your results or else” targets for those at the bottom of the rankings remain alive and well. Four years ago, Martin Stephen , at the time the “high master” of St Paul's School, told me that league tables were a “cancer on the face of education”(http://bit.ly/nw9msM). I think he was probably being too specific: “league table” pressures are actually simply a shorthand, in the state sector at least, for the entire exam-driven measurement and management apparatus which I have called hyper-accountability. That said, I am increasingly convinced that Mr Stephen's seemingly hyperbolic statement is not too far from the reality. I admire school leaders and teachers who resist the reductionist and anti-educational tendencies clearly inherent in making simplistic institution-based indicators the centre around which our schooling must revolve. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=488 Tue, 13 Sep 2011 11:19:32 GMT 20110913111932 Tue, 13 Sep 2011 11:19:32 GMT 20110913111932 Tue, 13 Sep 2011 11:19:32 GMT 2011091311111932 Did I really just write that? I've spent the last seven years trying to chart the effects of results pressures on schools in all their often gory detail. But this week, after reading a couple of articles on this subject, I found myself shaking my head. The stories, in the Daily Mail and the Daily Telegraph, documented a large fall in recent years in the number of pupils taking GCSEs in the group of subjects now collectively tagged as the English Baccalaureate: English, maths, two sciences, a foreign language and either history or geography. A Parliamentary written answer to the Conservative MP Charlotte Leslie revealed that, in 1997, the numbers taking these subjects was 293,000, or 49.9 per cent of the cohort. By last year, it had fallen to 141,000, or 22 per cent of the cohort. Cue a headline, in the Daily Mail, saying “how Labour let down a generation with easy GCSEs”. The suggestion, in both this story and in the Telegraph, was that pupils had been pushed away from these central , or “key” academic disciplines towards “softer” subjects, with league table pressure on schools – the chase for 'easy' ways to boost the figures - implicated in the stories as a factor. Well, from the Government's point of view, this fits with a very convenient narrative: Labour was simply lax, in that its performance measures encouraged schools to move away from the central, academic and “hard” GCSEs it has chosen to highlight in the EBacc, and that new measure is the way to fix this problem. However, this is a very partial reading of the situation. It seems likely that the largest impact on these figures came from the unrelated decision by Labour in 2002 to make the study of languages optional at key stage 4. The fact that pupils were no longer compelled to take a language up to the age of 16 makes it completely unsurprising that the numbers entered for GCSEs in French and German in particular are not what they once were. New GCSE figures show that the numbers entered for French fell by 54 per cent in the years 2002 to 2011, from 338,468 to 154,221, while in German, the fall was very similar, by 51 per cent from 126,216 to 60,887. Over the period, the overall number of pupils taking GCSEs fell by only 3 per cent. Anyone not entered for a modern language will not count in the Parliamentary statistic, so these large reductions effectively place a reducing ceiling on the numbers who would achieve the benchmark that generated this week's headlines. It is very likely that league table calculations on the part of schools will have played some role in the move away from languages over this time, especially as languages are generally perceived as among the most “difficult” GCSEs. But they will have been far from the only influence on schools and it is strange to make these claims without mentioning the fact that pupils once had to take the subjects up to the time of GCSE, and now do not. And not only did we go, over this period, from modern languages being voluntary to being optional at key stage 4, but this was part of an overall move towards broadening out the curriculum by Labour, with the philosophy being to give teenagers more options in the latter years of secondary. Not only did the national curriculum change, then, but, if I recall correctly, Ofsted inspectors expected schools to engage with this agenda of offering pupils a broad range of options. (In fact, I've just come across this Ofsted report http://bit.ly/nvJc8J from as recently as this May praising a school for offering a “good choice of both academic and vocational courses”. More choice, of course, means greater ability for pupils to opt away from “key” subjects). Against this background, it is unsurprising that the numbers taking what are now defined as “key” or “core” fell. Consider again, entry numbers in the six subject areas that make up that seemingly damning statistic. Again, I've done a comparison going back to 2002, to give an idea of trends. Maths numbers over the period rose by nine per cent – possibly more pupils being entered multiple times? – while English figures are down three per cent, in line with the overall reduction in pupil numbers 2002-2011. For science, it is hard to do a direct comparison, because the structure of science GCSEs has changed over the period. But the numbers taking single science GCSEs (physics, chemistry and biology) separately, which is now seen as providing the best preparation for A-level, have trebled over the period. It is possible that the numbers taking only one (generic science) GCSE, or no science GCSEs at all and taking alternative qualifications such as BTECs, have risen. Geography is the other subject in this group to have seen a big fall, at 25 per cent. But religious studies, another humanity but for some reason not counted in either this statistic or in the English Baccalaureate, more than compensated, its entry rising 81 per cent over the period. Meanwhile, history numbers have held more or less constant, with a fall of less than 0.5 per cent. This headline stat, then , seems to have been driven by possible changes in science take-up; possibly by the fall in geography – perhaps only a national problem if you view geography as “key/central/traditional” but not religious studies; and by the modern languages change. That one statistic – bundling up a number of subjects in one indicator – helps to obscure, rather than illuminate, what has been happening; subject-by-subject data at least need to be considered alongside it. Don't get me wrong: many will argue that Labour's decision – in making languages optional and seeking to broaden curriculum options - was the wrong policy , and that what pupils need is to study this “central” core above all else. On the other hand, this change can and has been argued from the reverse position: that it is better to have teenagers beyond the age of 14 choosing which subjects they want to take, rather than forcing particular subjects on them. I make no judgement on that debate here, but would just argue that it wasn't just league tables pushing schools towards this move: it was the entire Government policy drive. And that, in this case, is what any debate about these statistics must reflect. OK, well I've got that off my chest then. But, surprise surprise, I'm not letting league tables completely off the hook. For there are still problems revealed within the latest GCSE data which do seem to reflect badly on the effects of results pressures on schools, of which league table considerations are a sub-set. The number of pupils entered early for maths and English GCSE continues to grow, and there are now widespread claims that some schools are encouraged to take decisions towards early entry because of their own need to maximise results – I reported on a shocking example of this here http://bit.ly/9evGy4- rather than simply considering what is best for the child. In maths, concerns about schools pursuing early entry because of results pressures was documented earlier this year in an influential report from the Advisory Committee on Mathematics Education (http://bit.ly/mCEONK) . This is, perhaps, one of the most extreme side-effects of having a system which puts such emphasis on results for each institution. And – although it is not clear that this is having a massive effect on the overall statistic reported on by the Mail and the Telegraph* – some pupils certainly seem to have been pushed towards science BTECs because of their league table value. Alison Wolf's review for the Government earlier this year said that science BTECs, hitherto deemed in the rankings to be worth two or four GCSEs (and another rival qualification, OCR Nationals in the subject) did not prepare pupils to go on and study the subject at A-levels. Yet, according to league table information I have analysed from 2010 GCSE data, last year 12 schools, including seven academies, had not a single pupil entered for any science GCSE. Given that they must take the subject to 16, they will have been entered for alternative science exams. Earlier this year, Ofsted criticised schools pushing whole year groups towards non-GCSEs in science, and that seems right: it seems inconceivable that in a school with more than 100 pupils, none needs to be prepared to go on and take the subject beyond A-level. Results pressures on schools, unsurprisingly given the stakes, again can push them towards some perverse measures in a bid to improve their published data. Of course, the government's response has not been that it is the fundamentals of league tables which need looking at, but the measures by which they are calculated. Its reaction to the perceived problem of low take-up of languages in particular is to change the way schools are measured, by introducing the EBacc. Yet as Russell Hobby said on the publication of the results: “Schools have to be more than exam factories driven by performance incentives.” Amen to that. *Statistics within the Wolf report say there were just over 60,000 entries for the GCSE-equivalent BTEC First science courses last year, or around 10 per cent of the year 11 cohort. Some of these pupils will either have not have taken a modern foreign language, or history/geography, I'm guessing, in which case they would not have counted towards our EBacc Parliamentary figure even if they had taken two GCSE sciences. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=485 Fri, 26 Aug 2011 09:36:42 GMT 20110826093642 Fri, 26 Aug 2011 09:36:42 GMT 20110826093642 Fri, 26 Aug 2011 09:36:42 GMT 201108260993642 I feel slightly daunted at the start of this blog. I've read some superbly insightful postings* on the implications of this week's disturbances for school, written by teachers, and wonder what I can bring to this particular subject, never having taught. Some have also observed that those commenting on the riots, with all their complex causes, have simply used them as an excuse to re-iterate whatever they think is wrong with a particular aspect of society they are interested in. Here, again, I plead guilty. But that clearly hasn't stopped me writing this blog, so please read on! I should say at the start that, like no doubt many others, I was appalled by the footage of violence and looting, some of which occurred not far from where I live. “Thugs” seems to be the word used by much of the media and political class to describe those carrying out looting, arson and the rest, although it's not one I particularly favour. No, “idiots” was the word going round in my head, instead. What kind of person thinks it's OK to set fire to things, or to steal from any business, let alone a small family-run shop? Commentators and politicians are therefore right to say that these events were not political, in the sense of not being part of an explicitly political protest, especially once events spiralled away from the initial catalyst for them in Tottenham. Much of the looting was simply opportunism and wilful lawlessness. So much is obvious, and of course, there must be consequences for these actions for the individuals concerned. But of course it is also right to seek to probe for deeper causes. While individuals have to be held to account for their actions – and the vast majority of young people were not involved in these disturbances – the question has to be asked as to how we have got to a situation where some think it a good idea to set fire to a building, or to rob. Punishment alone at the end of this process can never be the whole answer. And any attempt to understand will need to look at what has gone on in our education system, as one factor among many affecting the way people behave. In an attempt to go beyond the fairly superficial reaction set out above, I've been re-reading reports (http://bit.ly/reXJrU and http://bit.ly/mTZwf0) generated by an excellent investigation into educational disaffection among young people which was carried out by the Rathbone Trust with the Nuffield foundation's painstaking review of 14-19 education. It reported three years ago. The research looked specifically at the situation facing “NEETs” – 16- to 18-year-olds not in education, employment or training – and featured 36 workshops in which groups of eight to 10 young people discussed their lives. It also included another set of discussion groups for “practitioners” who frequently had contact with them, such as Connexions staff, magistrates, voluntary sector organisations, teachers and college lecturers, youth offending teams and even employers. I know that not all of the participants in this week's disturbances were “NEET”, but the investigation's final report, published in October 2008, contains findings which could be considered prescient, given the events of recent days. It included the account of one 16-year-old from Manchester who, when asked to describe a typical day, said it included: “In the afternoon, I'm out with the boys on the estate. Chilling – terrorising, tipping stuff off bridges.” The report also talked about the gang culture, which it said had featured prominently in workshop discussions in both London and Manchester. It said: “If a young person does not feel a sense of belonging within the family, they tend to seek to belong to other groups which offer them a similar sense of belonging. Whilst for some, positive activities such as youth groups and religion offer alternative means of belonging, others choose a more negative route, with a sense of belonging ensuing from gang membership and involvement in crime. “These young people live in a society where celebrity is lauded and represents a source of aspiration…Particularly when parental and local community role models are not present, crime and the gang culture offer not only a sense of belonging, but a route to local celebrity status through alternative, sub-cultural means.” It added: “The workshops highlighted the range and heterogeneity of the challenges faced by young people but also indicated the problems that they shared. They face multiple barriers to making progress in their lives – poor educational attainment; poverty; low self-confidence and esteem; inner city living and poor labour market experience for boys; and for girls, teenage motherhood and a lack of parental interest in their education; and above all, a feeling of failure. To have any chance of helping these young people meet the learning challenges needed to re-engage with society they have to be acknowledged as a diverse group with diverse needs which must be dealt with in a holistic manner.” Unsurprisingly, there were also some specific problems with many of these young people's engagement with education. The report quotes research from 2007 which found: “It is undoubtedly true that among those who are NEET, there is a substantial majority of young people who, after 11 years of statutory education, are united by their common experience of social and economic disadvantage, low educational attainment, relative underachievement and alienation from the education and training system. The educational reform process that has continued apace in England since the Education Reform Act 1988 has completely failed this group.” You may or may not agree with that final sentence, but the rest of it seems fairly hard to argue against. The report also found: “One of the key findings from our work with young people over the last year is the very pronounced feeling of alienation from schooling so many expressed. Many of the young people certainly have unhappy memories of schooling…However, the reasons for dropping out are far more complex. For many it is not primarily about the school curriculum, or about a lack of vocational learning opportunities, but an inability to cope with the necessary authority structures that must underpin the structure of schooling.” The words “alienation” and “failure” run throughout the report in relation to the feelings of these young people towards education, and I believe they should give anyone concerned with schooling pause for thought. The investigation found: “Some of those who are classified as 'NEET' have carried an ingrained sense of failure with them since secondary, or even primary, school. This sense of failure affects the young people's capacity for self-motivation, and for identifying, realising and implementing their aspirations.” An anti-establishment culture outside of school therefore had its obvious attractions. Among the report's prescriptions are for greater support for frontline staff working with disaffected individuals, many of whom cited an inspirational individual, including sometimes a teacher, as a positive influence. The report - and this is where my particular obsession starts to kick in – also raised the issue of the use of the current system of GCSE A*-C indicators in secondary schools. It said: “The emphasis on academic attainment and on qualifications, and particularly on the five GCSEs at A*-C benchmark, in schooling, has serious implications for those young people who do not succeed with mainstream schooling.” The investigation's interim report had found: “Young people who do not achieve good GCSEs at school must be offered a suitable second chance, which they perceive to be accessible and relevant to their particular situation.” While new measures being used by Ofsted at that time focusing on pupil progress, rather than simply raw attainment, might help, the danger was that schools would simply marginalise those children they felt would could not meet these indicators. I wonder if education policy-making hasn't failed to rise to the undoubted challenge faced by these issues in recent decades. First, we have had a system in which success has been very narrowly measured and accredited, with repeated messages inevitably sent to children as to which of them are meant to be viewed as the successes, and which, the failures. English and maths success in primary school, followed by achievement in traditional academic subjects in secondary, have been stressed so overwhelmingly, including through the accountability system, that it is, perhaps, unsurprising that some children failing on these narrow measures seek to disengage. This is now reflected in the new English Baccalaureate benchmark. The subjects within the Ebacc, including, of course, English and maths, are very important and, of course, we want more pupils to be fully engaged with them. But I do wonder if schools also need to be encouraged to emphasise a wide range of subjects and skills: heresy as this may sound to some, I do think children need to have something to succeed at, and narrowing down the focus in schools reduces the number of dimensions of success. Pupils will and do disengage – especially if secondary schools now react as seems natural by grouping students into “EBacc” and “the rest” - and the reaction of some to say that they simply should not do so is unrealistic. Of course, many will argue that this is hopelessly idealistic and fuzzy-headed. Maths and English in particular are vital for children's futures. And it is patronising to think that certain pupils cannot achieve in these subjects. I would not disagree, but I would say that, thankfully, people can and do succeed in real-world jobs that put the emphasis on, for example, artistic or musical creativity, craft skills, or teamwork. Stories of schools reducing teaching in these subjects because of the EBacc are worrying. To say to children that schools are mainly just about a limited number of “basic” or “traditional” subjects risks narrowing their experiences needlessly. If you have just one measure of success, the danger is that those at the bottom on this measure will disengage. Offer more fields at school in which pupils can genuinely find an interest, passion and some success, and you might engage more young people. The other thing to say about the EBacc is that it seems at least partly predicated on the notion that the definition of a good society will be when more working class children get to “good” universities, including Oxbridge. This is quite a narrowly-defined way of coming at the issue of social justice. For, by definition, only a minority of pupils of any kind can go to the “top” universities. Even if we do manage to get to a system which is fairer for “bright” children from disadvantaged homes on these terms, what about the rest? I worry that the emphasis on particular categories of children is not much of a strategy for the education system as a whole. And the EBacc's introduction, of course, follows no wider debate about what we want for that system as a whole. Consider, again, that narrowly-defined sense of what counts as succeeding at school. I was pondering these issues by email with a group of progressive-minded educationists yesterday. I wondered whether modern results-driven, test- and exam-focused accountability imposed on schools from afar was having an impact, in terms of some of this alienation. This was one response, from a university education academic: “Is there anything any of us can do to get politicians and those who perpetrate and perpetrated the testing culture to try to imagine what it feels like to enter secondary school below level 4, to be put in bottom sets…and to be expected to feel proud and behave well while getting failing grades again and again and again. “[Pupils are] being predicted to fail and then - amazingly - being set in a path to failure in groups with peers who are also lacking resources, lacking the cultural capital of the schooling system, and live impoverished lives. The reality of many classrooms is that students are told their predicted grades and labelled with their levels and go through secondary school, if they attend at all, being treated like, and taught to become, low attaining people. “I am not saying this is the sole or even main cause of the disturbances this week, but feeling worthless at school doesn't help, when school is one of the places which could institutionalise progress and worthwhileness.” Regular readers of this blog will be unsurprised to learn that I think we have lost something in the move to judge schools – and therefore to express what matters in education – in very narrow, statistically-framed and technocratic terms. I think it would be better if schools were not discouraged from thinking of their job more holistically: educating the whole person, and educating all pupils, not just those on which the latest set of performance indicators focus. Rowan Williams, the Archbishop of Canterbury, touched on this I think when he said in the House of Lords on Thursday that “our educational philosophy at every level has been more and more dominated by an instrumentalist model that is less and less concerned with the building of virtue, character and citizenship”. Education, he would argue, has a value in itself beyond the qualification generated at the end of the process. And this is partly moral. Education should be about encouraging children to be good contributors to society, yet this is not the way schools are judged. Indeed, I would argue that some actions undoubtedly and not surprisingly pursued in recent years by some schools in the race to generate better grade stats for the institution - such as children being told by their teachers what to write for GCSE coursework – have sent the wrong messages to young people, in a system where what we value in education seems to have to be expressed in a set of numbers. But I can't recently recall an Ofsted report, for example, raising questions about the morality of a particular school “gaming” the system to raise its numbers. The job of the head teacher, in the way success is defined, is simply to raise the statistics. I'm not sure how many teachers would agree with the archbishop's thoughts entirely, and I know, of course, that there is fantastic work going on in schools up and down the country every day. And certainly, some of the educationists in my discussion thought that statistical tracking systems within schools have brought some benefits. But I would say that a system that has become so obsessed with raising grades almost as ends in themselves** might be in danger of losing sight of the big picture of what, surely, really matters in the end: helping all children lead fulfilling, rewarding and, yes, law-abiding lives. *For a range of perspectives, see: http://bit.ly/pHnYsu , http://bit.ly/przfoy , http://bit.ly/1F73Nd. **As I've argued before, the imperative of improving pupils' results makes a lot of sense at the level of each school; the Rathbone investigation recognises the crucial centrality of qualifications to young people. But if, nationally, the system just becomes focused on grades as ends in themselves, the danger is that there is no long-term benefit for young people, for, as grades go up on average nationally, employers and universities simply raise their grade entry demands. Grades are largely, then, carry relative, rather than absolute, currency for young people. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=482 Sat, 13 Aug 2011 12:52:24 GMT 20110813125224 Sat, 13 Aug 2011 12:52:24 GMT 20110813125224 Sat, 13 Aug 2011 12:53:41 GMT 2011081312125341 Did exam boards break their industry's code of practice in allowing through a higher than usual number of errors in this year's GCSEs and A-levels? This question is being asked by England's qualifications regulator as part of its investigation into the blunders in question-setting which have been subject to much debate during this summer's assessments. Meanwhile, the Chartered Institute of Educational Assessors is calling for the errors to be a trigger for fundamental reform of the way GCSE and A-level papers are designed, with questions subject to extensive “pre-testing”. The errors – 10 of them are listed by Ofqual on its website – reportedly ranged from a printing mistake by AQA leading to some schools receiving GCSE maths papers which included questions from a previous version of the exam to an OCR maths AS level paper which featured an impossible question worth 11 per cent of the marks on the paper. This week, an Ofqual spokesman re-iterated to me that exam boards should have been checking all papers in detail before any candidate sat them. Perhaps more revealingly, he also drew my attention to paragraph 3.14 of the code of practice for GCSEs and A-levels, which states that question papers and mark schemes for all exams must be checked by a “scrutineer”. The boards, says the code, are required to check “the final drafts of the question papers…to ensure that the questions can be answered in the time allowed and that there are no errors or omissions”. The boards should also “work through” question papers “where appropriate” – in other words have someone working for them sit the paper or part of the paper in advance, and also double-check the mark schemes. Crucially, perhaps, the code also says the “scrutineer” should prepare a report on this process. “[This] report will be sent to the chief examiner, who must then approve any necessary changes to the question papers/tasks and provisional mark schemes,” says the code. If this process was followed thoroughly, it would appear that mistakes should not happen. So how did the errors get through? If reports are available to Ofqual, it would appear to be possible to find a paper trail and pin down responsibility. Certainly, Ofqual is suggesting it will be holding the boards to account for the extent to which they followed the code. The boards themselves have all said they are sorry for the mistakes : all three of England's GCSE and A-level boards, as well as Northern Ireland's Council for the Curriculum, Examinations and Assessment, had at least one reported error this summer. I asked England's three boards if they had followed the code of practice in all cases. AQA, which has five listed mistakes, said: “It is AQA's policy and intent to follow the Code of Practice. This is one of the areas that Ofqual will be focussing on as part of their inquiry and we are currently conducting our own internal inquiry to determine whether all procedures were followed correctly, including the role of the scrutineer.” OCR, with three errors, said the boards were “all carrying out our own internal investigations to determine whether all procedures were followed correctly and we will share this information”. It added: “OCR recognises that errors in exam papers are unacceptable. That's why we are working hard to ensure that no student suffers as a result of any errors identified in our papers.” Edexcel, with one error, said an internal review it had carried out had already established that its processes “were in line with the regulator's code of practice”. It added: “Regrettably, this year an error was missed in the checking and we are taking actions to make sure that no student will be disadvantaged as a result”. There will be many who will argue that the fact that mistakes can occur is a reflection of the pressure on the exams system, with the boards now jointly setting 60,000 questions across 5,000 papers. However, Ofqual's stance will be that the code of practice rules should have stopped errors getting through. The Chartered Institute of Educational Assessors is an organisation which has been around since January 2006 but which may become freer to speak its mind now that, since March this year, it no longer receives any funding from government. It has entered the debate with a call for all GCSEs and A-levels to be pre-tested. The comparison it draws is with our Sats tests. Sats, for all the criticisms levelled at them, have, as far as I can remember, remained free of the type of errors in questions seen this year in the secondary assessments The CIEA, which has 17,000 members, points to the pre-testing system for national tests, whereby every year's papers are subject not just to checks of the kind described above, but to pre-testing by samples of pupils not taking the assessments for “real”. This is meant both to help the test administrators in maintaining standards from year to year and to iron out any problems with questions. Some Sats writing questions in particular have, of course, been criticised for failing to engage pupils, but, to re-iterate, I cannot remember any that have been misprinted or have been technically impossible to answer. Graham Herbert, the CIEA's interim director, says: “The national curriculum tests are, if you want to use the phrase, stress-tested. General Qualifications [GCSEs and A-levels] tend not to be in this way. We think they ought to be.” Mr Herbert argues that exams systems in the United States, Australia and New Zealand include such pre-testing, and that it used to be more of a feature of our system until at least the mid-1990s. Pre-testing in this way would have stopped most if not all of this year's errors, he says. However, the boards appear unimpressed. None was able to point me towards pre-testing of this kind going on for GCSEs and A-levels.The Joint Council for Qualifications, the boards' umbrella body, responded with a statement. Dr Jim Sinclair, its director, said: “JCQ awarding bodies believe that the pre-testing of questions for use in GCSE and A level examinations would create unnecessary risks and have a negative impact on students, teachers and the reliability of the examination system. “It could have an adverse effect on student performance in actual high stakes exams, call into question the security of exam papers, and place a huge burden on schools and colleges.” The JCQ said that pre-testing through setting pupils trial papers could demotivate them when it came to the real assessments, by adding to the number of exams they had to take. There was a risk that questions set and tested in advance could be passed on to fellow students when the “real” exams came to be taken. It would be difficult to keep changing the questions to be pre-tested as examination specifications constantly changed. And the sheer number of exams – the 5,000 different papers at GCSE and A-level compared to a handful of different Sats papers every year in reading, writing and maths – meant pre-testing would create both those “huge burdens” on students and on the system as a whole. Awarding bodies were also working hard to minimise costs, and this idea “moves in the other direction”, said the JCQ. This position appears in line with that of Ofqual, a spokesman saying: "While Ofqual does not require pre-testing for General Qualifications, if an awarding organisation wanted to introduce it and had the means to deliver it while maintaining standards and protecting the security of the exams, it would be acceptable. "However, there are significant practical considerations [along the lines of those outlined by the JCQ above] that make this development unlikely.” Mr Herbert, however, disagrees. It would not be too onerous for schools, he says, since only three to four pupils a year in each would be required to sit an additional paper. Saying it would be too expensive was “not an option when the public pays for a valid and reliable service”, he said. He added: “In the professional test development community, pre-testing is the norm. No professional test developer would use a test as soon as it had been written. It is the school exam system of the UK that is out of step with the rest of the world and it is perhaps time that we made an effort to catch up with the US, New Zealand, Australia, Singapore and so on.” With Ofqual due to present an interim report on its investigation's findings by the end of October , and a further statement around A-level and GCSE results time, you should expect media interest around this subject to continue for a while yet. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=478 Fri, 29 Jul 2011 16:16:54 GMT 20110729161654 Fri, 29 Jul 2011 16:16:54 GMT 20110729161654 Fri, 29 Jul 2011 16:16:54 GMT 201107291641654 Yet more test data to be released, new targets for coasting schools, “zero tolerance” of public sector failure… …And “supporting dedicated public sector staff who want to make a difference” An important speech made by David Cameron on public service “reform” on Monday has struggled for attention this week, with Mr Cameron criticised for delivering it just as MPs were debating the latest tortuous, and riveting, developments in the phone hacking saga. Mr Cameron was launching the Government's “Open Public Services” white paper, which has reportedly been delayed for several months amid disagreements within the coalition over some of its more radical elements. The paper also got an underwhelmed write-up in the Guardian, in this report by the former education editor Polly Curtis: http://bit.ly/qrxlCa However, I think the report contains quite a few policy suggestions which will interest readers of this blog. I wanted to list a few, many of which are being proposed under the drive for more “transparency” about what goes on in the public sector, before commenting on some contradictions contained within the proposals. First, then, the proposals, some or all of which, I'm guessing, might be greeted with a sense of weariness among head teachers. So, in no particular order, here we go: -It appears the individual test and exam results of entire year groups of pupils in England are to be released, on an anonymous basis, to the public. This is the “national pupil database”, in which the achievements of more than 600,000 children in particular cohorts are collected, alongside many other background characteristics of each child, by the Government. So far, this database has been seen mainly by civil servants and by academic researchers, who have to sign agreements before accessing the data. The files on offer are potentially very powerful, allowing any individual to analyse the statistics in any way they want. But these datasets are also huge: data files on any one (anonymised) pupil run to hundreds of columns – and remember, there are more than half a million pupils in each year group – making them too complicated for conventional spreadsheets to handle. Making them available in raw form to the public directly, then, probably will be of little use. However, external organisations, including, probably, journalists, will want to use these files to present data in new ways. It appears these data will allow yet more school-by-school ranking information to be produced. I think this is not possible with the current national pupil database, because of fears that some pupils (in small schools) could be identified, so it will be interesting to see how that issue is tackled. Ministers were quoted in the Daily Telegraph (http://tgr.ph/pIYKjp) as viewing the national pupil database, reportedly to be made available from next June, as the “jewel in the crown” of their entire public sector data-release project. Mr Cameron said: “With our new plans, you'll be able to drill down into the performance of individual schools, checking their exam results by subject area, absence rates and the quality of teaching. Our aim is to provide similar information on performance right across our public services.” The report adds: “We will look to strengthen datasets in due course: from January 2012 we will bring together for the first time school spending data, school performance data, pupil cohort data and Ofsted judgements in a parent-friendly portal, searchable by postcode; from this date we will also publish data on the attainment of pupils eligible for the Pupil Premium.” -The report raises the possibility of a new set of targets for schools it describes as “coasting”. It says: “We will…explore how best to raise standards in coasting schools (eg introducing year-on-year improvement standards)”. I'm already getting into the contradictions now, but I wonder quite how this will match up to the proposals in the Bew report for schools to be judged on more than a single year's data. - Still on the data front, the white paper makes non service-specific mentions of “satisfaction surveys”, saying: “We will ensure that key data about public services, user satisfaction and the performance of all providers from all sectors is in the public domain.” This, then, would presumably mean, in the schools sector, parent – and pupil? – satisfaction survey results being published. A whole new set of league tables? - And a “payment by results” system could be introduced for early years providers. The paper says: “The forthcoming Foundation Years Policy Statement sets out plans to trial arrangements to pay Sure Start children's centres in part for the results they achieve”. I wonder how closely such a trial will look at possible unintended consequences. - Perhaps less controversially, the report also proposes that “Academy and foundation trust [the latter in the health sector] meetings be open to the public, to which my response was: “why weren't they already open?” A review of the “effectiveness and accountability of existing autonomous structures, for example academies and executive agencies”, is also proposed. - Potential changes in the background which could also affect schools and their staff include a suggestion that councils will be given the power to retain the taxes they collect from local businesses (will this be a potential boost to the funding of public services in some authorities, I wonder, or used to cut local taxes?) and an upcoming review of employment law which seems to be addressing, in part, claims from private providers that strict “TUPE” rules guaranteeing the pay and conditions of staff transferring from state to private providers are a “barrier” to private sector involvement. (Page 44 of the white paper, for those interested). Now, those contradictions. First, the white paper claims to be “replacing bureaucratic accountability with democratic accountability”. It is quite clear, I think, what this means; just unclear how to square it both with other aspects of the white paper itself and with what is going on in reality, at least in some schools. The intention is to replace “top-down accountability” where institutions (schools, in our case) mainly worry about performing to satisfy politicians, civil servants, inspectors and local authority advisers – for example, by implication, through performance targets - with “bottom-up accountability”, where institutions are focused mainly on meeting the needs of users of public services (pupils and parents). This is certainly defensible as an ideal, I think. Yet the coalition's floor targets, in which primary and secondary schools are set centralised baselines for test and exam performance and threatened with closure if they fail to improve, are a clear example of “bureaucratic”, or centrally imposed, accountability. Many will also see intervention by Ofsted as another form of “bureaucratic” accountability. And, indeed, the report itself says, quoting a TES survey from last year, that “42 per cent of heads in both primary and secondary schools named interference from local and central government and Ofsted as a barrier to them running their schools”. Yet, while schools with better results are having the requirements of both Ofsted and Labour's targets regime relaxed, there is no sense in the paper of this happening for those at the bottom of the league tables. Indeed, I would argue that, for these schools, “bureaucratic” accountability remains much more of a reality than “democratic”. The paper justifies the idea of intervening from the centre in these cases on the grounds that there is “no point in a parent having a choice of school if standards in those schools are low – that is no choice at all”. This is to substitute bureaucratic accountability, based on a judgement at the centre as to what counts as an underperforming school, for democratic accountability. Again, it is defensible – a speaker at an event I attended this morning suggested some parents in disadvantaged areas simply could not hold schools to account as effectively as others, so there was a need for someone at the centre to do that - but consideration of its effectiveness needs to include a detailed understanding of the effects on children's learning experiences of results pressures in the affected schools. And it does go against the philosophy that the way to promote improvement is, effectively, to make the relationship between the school and its community the key element of accountability. Second, the paper talks about the importance of allowing “individuals to exercise choice across a diverse range of providers”. Yet, in the education field, the Secretary of State is getting the power to force on “underperforming” schools just one type of status: becoming an academy. A truer commitment to this diversity would embrace a range of responses, with schools and local people given the chance to decide which solution works for them. The paper also talks about moves to “enhance the role of local councillors” and bolster the position of local authorities – “strong local government is at the heart of our reforms” though the academies policy downgrades the role of local government. There are other contentious points. The paper also says that “people with have more choice, especially in education”. But this just begs the question: choice in what sense? If I am a parent, will I see new schools opening in my area, broadening my choice? Perhaps so, given the policy of free schools, the government could reply, even though few are going to open in the early years. But then the question is: will free schools really be funded, in the long term, in addition to the existing schools in an area? If so, that looks an expensive policy. Or are free schools meant to replace existing maintained schools? If so, it will be a different kind of choice, rather than more choice. One could argue, of course, that the advent of more academies will open up a kind of choice, as each of them takes advantage of its freedom to modify its curriculum in different ways. But, in the secondary sector at least where most academies are being created, specialist schools under Labour already had the freedom to emphasise different aspects of their provision. And the accountability regime will limit teachers' freedom by encouraging teaching towards the performance metrics. Finally, the paper talks, in ways that will no doubt be welcomed in many schools, about reducing bureaucracy. It says: “The Government does not believe the centre should micro-manage public service delivery and we want to support all those dedicated public sector staff who want to make a difference.” Not only that, but it even mentions the Prime Minister's well-documented plans to try to measure personal well-being, and even the effect of its own policies on people's happiness – I'm not joking - is to be considered. It says: “For the first time, the Office for National Statistics will be measuring levels of national wellbeing and we are developing methods to better understand how wellbeing is affected by our policies.” Satirists will no doubt hereby rejoice. There is even a line promising that those “on the front line of public service delivery” are to be invited to “identify areas where central government can get out of the way”. “These suggestions should be online for everyone to see, and to select the best,” ventures the paper, a tad bravely in my view. I wonder, though, about how those floor targets will affect the “wellbeing” of school leaders and others working (and learning?) in schools at risk of not hitting their floor targets. The paper, while pledging to support “dedicated public sector staff”, also promises that its reforms will embody a “zero tolerance” approach to failure and “drive provider complacency out of the system”. That doesn't look very supportive to me. And, in education, any notion that over recent years schools have ever been allowed to be “complacent” about results, given the already multi-faceted pressures on them to raise grades and scores, is ridiculous. So what is it to be: a more trusting, less interventionist approach to public services, or one founded on the belief that “provider complacency” is, somehow, the greatest danger? The reality is that the way it feels on the frontline may depend on which part of the system you work in, but there is certainly a tension there. One way of reading this paper is that it is an attempt to enforce the ideology of the market on parts of the public sector which have so far proved resistant to it, with acknowledgements in places that this approach can have limitations. The paper, though having an ideological feel overall, is right to acknowledge, implicitly, that market systems have their strengths and weaknesses. But in general this document makes for a simplistic read, in places, and a confusing one in others. Given that the paper itself is to be followed by a period of consultation; many of the ideas are subject to further review; and the coalition is fast developing a track record for changing its mind, the real likely final impact of this document on public services must remain an open question. A consultation period is following the publication of this white paper, and the Government says it wants to hear the views of public servants. Go to www.openpublicservices.cabinetoffice.gov.uk for your chance to respond. The white paper itself is at: http://bit.ly/nFzcLV http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=475 Thu, 14 Jul 2011 15:07:56 GMT 20110714150756 Thu, 14 Jul 2011 15:07:56 GMT 20110714150756 Thu, 14 Jul 2011 15:09:20 GMT 201107141530920 I have a confession to make. Ten days ago, I made a decision that journalists sometimes make, but one I try to avoid if possible. Pressed for time, I read only to about two thirds of the way through a major report, on a subject of great interest to me and on which I knew I would, in time, be writing in detail. I reasoned that with the report's executive summary and much of its main text behind me, this would be enough, for now, especially as my main purpose at that point was simply to update people on twitter as to its central findings. Well, I think that I made a mistake now, and have learnt a lesson. For the report was the final verdict by Lord Bew and his committee on what Key Stage 2 assessment should look like in the future, and buried near the end – it is not flagged up in the executive summary – is a remarkable set of suggestions for new tests. It seems I was not the only journalist to miss it: nowhere has this been reported, as far as I am aware. Anyway, when I was alerted* to the need to read right to the end, I was staggered to discover what Bew was proposing. Having spent more than 50 pages up to this point discussing the merits of detailed changes to the current system of tests and teacher assessment, and advocating some rebalancing in favour of the latter, out of the blue, on page 66, the report offers a new idea for the “longer term”. This would appear to be a re-engineering of the Sats system. But it seems to have been put forward not in response to widespread criticisms of the backwash effects of test-based accountability, as you might suppose; indeed, this worry is barely considered here. No, in fact the reasons are likely to remain a mystery to any readers of this report, I fear, since they are offered with really very little explanation. What the Bew report proposes is that, in time, there should be two new sets of tests: one a pass/fail assessment of the “basics” of literacy and numeracy, which all pupils would be expected to pass, with the exception of some children with special needs. The test could be taken in either year 4, 5 or 6: whenever the child was ready. The second set, which look to be more akin to the current Sats, would be taken at the end of year 6 “to allow pupils to demonstrate the extent of their knowledge and therefore to measure pupils' progress during the key stage”. Teachers and heads who are critical of our current high-stakes testing system might think, here, that this proposal might not be the end of the world, for surely Bew would not think of loading accountability measures onto these new assessments without thinking, or explaining, very carefully the consequences, intended or otherwise, of using assessment data in this way? Well, in fact this appears to be exactly what has happened with this proposal. It appears the tests would be used in school-by-school accountability, or as a measure of the “quality” of the school, as this brief section of the report concludes as follows: “We feel the combination of these statutory assessments could ensure that all pupils reach a minimum standard of attainment while also allowing pupils to demonstrate the progress they have made – which would indicate the quality of the school's contribution to their education. It could provide a safety net in that all pupils should achieve a basic minimum, but would not impose a low ceiling on the able.” The report says it recognises that teaching to the test might be a problem under this new approach, but then seems to offer no suggestion as to how it would be avoided. (Indeed, the idea that these tests should act as a “safety net” suggests support for assessment-led education, or assessment having a large influence over teaching and the curriculum, a view which I think is implicit in the rest of the report). This is despite teaching to the test being probably the most influential criticism levelled at the Sats regime and one which any report, to have any credibility in seriously engaging with this subject, should be taking extremely seriously. The report also offers no analysis of previous attempts at introducing “mastery” tests assessing “basic” subject knowledge, which have a chequered history in English education. So there you have it: a new set of tests is proposed by Bew, if only not in the immediate future. (This idea is not a formal recommendation of the report). One wonders where the suggestion came from, while I also wonder, no doubt groundlessly, whether Lord Bew's committee is giving the public a sneak preview of a possible suggestion to come out of the national curriculum inquiry. If so, my guess is that the teaching profession will want to know exactly how this model would be an improvement on current arrangements. - I have already mentioned this on twitter, but it is just worth mentioning here that the Bew report features a punctuation mistake. This mistake occurs, believe it or not, in the section of the report which is putting forward a recommendation for a new test of punctuation, and, indeed, after the word “punctuation”. The full quote runs as follows: “We recognise there are some elements of writing (in particular spelling, punctuation grammar and vocabulary) where there are clear 'right or 'wrong' answers, which lend themselves to externally-marked testing.” Did you spot the missing comma? File under “you could not make this up”. *The tip-off came from contacts at the Cambridge Primary Review network. If you're interested to read my more general thoughts on the final Bew report, you can do so here: http://bit.ly/mstmBA http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=471 Tue, 05 Jul 2011 09:45:22 GMT 20110705094522 Tue, 05 Jul 2011 09:45:22 GMT 20110705094522 Tue, 05 Jul 2011 09:45:22 GMT 201107050994522 There are times, increasingly I find, when I come across the latest policy move by a government or one of its agencies with a sense of bemusement, verging on despair. How can the detail of what is being proposed possibly accord with the rhetoric of the politicians supposedly overseeing its formulation, I wonder? Why do people seem so unable to see the implications of what they are putting forward? Or do they see them, and simply have some deeper plan which they don't want to tell us about? The latest instance arrived on Friday, with a TES front page (http://bit.ly/jQVlLO) revealing, accurately I believe, the detail of how schools look set to be judged under Ofsted's new inspection framework. The story stated that “more than 5,300 schools with below-average test results will be failed by Ofsted from next year, unless they can show they are 'closing the gap', confidential documents reveal”. The basis for this was guidance which has been sent to inspectors carrying out pilot inspections which are trialling Ofsted's new inspection framework, to be introduced in January. This is the “evaluation schedule” under which inspectors are given ground rules against which to decide what grade they give schools. The guidance, as I understand it, advises inspectors to give schools an “inadequate” verdict on achievement if their results accord with the following formula. “Achievement”, says the guidance, “is likely to be inadequate if any of the following apply.” It then lists five possible triggers for that inadequate verdict on achievement. The first of these stipulates that: - The school has below average attainment (ie raw results) and “is showing little sign of improvement or is in decline”. The four other characteristics within the school's results, any of which would trigger an “inadequate” verdict, include underperformance affecting “key subjects” within a school; performance gaps between particular groups of pupils; a lack of progress made by special needs pupils; and “Pupils' communication skills indicate reading, writing and maths capabilities are not sufficient to prepare them for the next stage of education”. This picture of the type of results which would lead to an inadequate verdict on achievement is reinforced by how the guidelines define a satisfactory school. One whose raw scores were below the national average, it says, would have to have results which were “improving steadily and therefore closing the gap on the national average for all pupils”, to be satisfactory. By implication, then, one below national figures which was not closing the gap would be given that damning label: “inadequate”. Achievement is, of course, central to the overall verdict. The guidance advises that a school adjudged satisfactory overall should be at least satisfactory in all the four main inspection categories, of which achievement is one. So a school failing on achievement would fail overall. It is tempting now, to get into a debate about the merits of particular indicators, but instead I would just say, at this stage: “Hold on a minute. How does any of this square with a particular vision of how inspections should operate as set out by critics of Ofsted's system under Labour, especially the Conservatives and the new coalition government?” The immediate key to this has been statements, both by Michael Gove in opposition and by the coalition since it came to power, suggesting that inspections put too much emphasis on test and exam data, and therefore that the system needed to change. In 2008, Mr Gove reportedly told the TES that Ofsted would need more resources, to allow inspectors to spend more time in the classroom to make judgements, because: “One of the problems is that inspections have been too reliant on already published data”. (http://bit.ly/j2QXcL) Last November, the Government published a white paper setting out its thinking behind the new Ofsted framework. It said: “We will ask Ofsted to return to focusing its attention on the core of teaching and learning...[allowing] inspectors to get back to spending more of their time observing lessons, giving a more reliable assessment of the quality of education children are receiving”. This goal seems to accord with the view of Ofsted itself. I wrote an article for the Financial Times in February, in which I said that “Ofsted inspections have, in recent years, focused heavily on statistical indicators of school quality that are largely based on exam performance”. Ofsted's press office then rang me to say I'd got it wrong: inspections are now far more focused on lesson observation, rather than simply judging schools by spreadsheets. To which the obvious answer is: if this guidance continues to reflect the reality of how Ofsted judges schools, inspectors can be told as much as anyone likes about the importance of lesson observation, but in the end it is assessment results which will drive the overall outcome. Hypothetically, even if they spent a year in a school, building up a picture of its strengths and weaknesses, if the inspection criteria against which the school was judged in the end was framed almost entirely by statistical calculations based on test and exam data, we would still have a data-driven inspection system. This, then, remains inspection almost by computer formula, whatever the rhetoric. There are many implications. First, of course, this yet again underscores the view that what matters in schools is test results. I find it interesting that goals for our system which would appear to be benign, such as “ensuring that more pupils make good progress” are usually phrased in a way that omits the words with which that sentence logically should end: “as judged by test and exam results indicators”*.I say logically, because this is the reality of how pupil progress, as well as raw achievement, is judged. Therefore if we want to emphasise pupil progress through test data, we inevitably underscore the notion of narrowly-focused test-driven schooling. Do we want an education system which defines success and failure almost entirely on the basis of test results? Some would argue for this, some in particular alleging its benefits in safeguarding the interests of disadvantaged children. But what staggers me is that there has been almost no debate, at an official level, facilitated by those who preside over this system, about what the country wants to happen in its classrooms. Ofsted has certainly not sought to ask whether test-driven education is really what is needed, as far as I am aware. (Though it has made pronouncements about excessive test preparation in year six, for example, it has failed to link this to the logic of its own inspection regime; it is naive, given the above, simply to advise schools not to go in for excessive test preparation, because this is the incentive of its own structures.) If it believes that the goals of education can be expressed and captured almost entirely through test results, it should come out and say so, given its detailed guidance to inspectors. Second, of course, the detail of the way these indicators are constructed means that a certain level of failure at the school level comes built-in. That is, schools will fail under this system by definition because of the way the indicators have been constructed, no matter how good the system as a whole might become. Our schools could be far and away the best in the world, and still a minority would fail because the indicators used to define success and failure are relative. Consider, again, that definition of what it counts to be “satisfactory”- or not- on achievement. Schools are at risk of an “inadequate” judgement, first, if their raw results are below the national average. To state the obvious, on any indicator, a large proportion of schools will be in that position by definition, no matter how good the schools system is overall, because of the way averages work. They could escape that inadequate verdict, though, if they were “improving steadily” and therefore “closing the gap” on the national average. Again, though, this is a relative measure: if all schools were improving fast, any one school under threat would be at risk of being adjudged “inadequate” if it failed to match this pace. Thus this system has failure built into it by definition and the number of schools adjudged inadequate will say little about the overall quality of education in England. This means that Ofsted's overall pronouncements on the quality of education are compromised. If a large part of its rules as to which category to place schools in are based on relative indicators – how good a particular school's results are, compared to others', ie whether its performance/performance improvement is above or below the national average – it should not use its statistics of school failure rates to make claims on the overall quality of education. To put it another way, a country of tall people, such as the Netherlands, will always have a large proportion of its population of below national average height. But the presence of a large number of shorter-than-national-average people in its population does not mean that it is a nation of short people. I wrote about this several years ago, (http://bit.ly/jzBy0y) and the point remains. Third, for all the work currently going on on a new national curriculum, if accountability for schools continues to work according to test results formulae, this wider work will count for little: tests, exams and statistical calculations by the school as to how to avoid a “failing” label will continue to drive the curriculum. Probably the largest point, though, relates to trust. Mr Gove has sought to make political capital on rhetoric around trusting the profession. Teachers, rather than politicians, know best about what works in the classroom, he has argued. But the inspection framework shows up the reality. This system is relentlessly centralised: Ofsted, working to ministerial guidelines lays down the performance rules. Schools have to perform according to these data-driven metrics. If they do not perform, Ofsted and ministers intervene, with the government if necessary ordering the takeover of schools by private organisations. There is one important new development under the coalition, of course, in that schools defined as successful are freed from the more urgent demands of inspection. But, because the main judgement as to what counts as success is test and exam results, this could be seen as underscoring the centrality of assessment-driven performance metrics. This, then, is absolutely a continuation of Labour's set-up in which the route to success by those at the political centre is seen as defining the quality of our education system statistically and then monitoring schools intensively until, by implication, you force them to improve. This was reinforced this week with news that Mr Gove is to impose new GCSE targets on secondary schools. I would point out, here, that Mr Gove has, of course, repeatedly criticised the failings of England's education system. Yet here is a central component of that system, absolutely fundamental in driving what goes on in schools, whose underlying philosophy is identical to that which existed under Labour. I suppose there will be many who genuinely believe that this is the best route to improving education. The implication is that Ofsted believes that its statistically-orientated inspection systems represent the way to advance the public interest in improving our schools. Even the lingering unfairness of judging schools and teachers over results for which they can only ever have partial control – and in the process, in my view inevitably downgrading the role of the pupil in achieving those results - is over-ridden by the view at the centre that such a structure pushes all within it to ever greater feats of achievement. I don't share that position: do systems really get the best out of people by encouraging them to perform against a rigid set of numerical criteria? Would professionals often use their freedom from narrow statistical scrutiny to forget about the interests of their pupils, an assumption on which results-driven accountability rests? Does it, complete with its use of words such as “inadequate” to describe the work of professionals whose charges do not meet statistical expectations, create the kind of working environment that will attract talented young people to want to join (and stay in) this profession? It is management-by-numbers. Its rigidity narrows priorities and encourages statistical game-playing. (As witnessed by the school in the TES article said to be introducing a particular set of courses simply because of their likely impact on the school's data). One head suggested to me this week that the guarantee of failure contained within Ofsted's systems – ie a proportion of schools will always fail, because of the way the indicators are constructed - is meant by the coalition to “soften up” schools for takeovers by private organisations. I wondered about that, questioning internally whether actually it was more a product of a particular philosophy at Ofsted as to how success should be measured. Wednesday's Guardian front page, suggesting 88 “struggling” schools will be forced by Mr Gove to become academies, underscores the suspicion of the head, I believe, that privatisation of the schools sector is the big agenda here. In any case, whatever your view on whether Ofsted's systems are a force for good or ill, let's not kid ourselves about the nature of what – staggeringly, given the rhetoric about more freedom for the profession and less emphasis on data by inspectors - is still going on. *Ofsted might say, I guess, here, that teacher assessment judgements can play a role in inspectors verdicts. But test and exam scores remain central. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=467 Wed, 15 Jun 2011 11:21:20 GMT 20110615112120 Wed, 15 Jun 2011 11:21:20 GMT 20110615112120 Wed, 15 Jun 2011 11:21:20 GMT 2011061511112120 It is one of the central principles against which the current Government review of primary assessment has been framed. It is a statement which might, to the lay person outside of education, carry strong common-sense appeal. But how well-founded is one of the defining principles of the ongoing Bew review, that evidence from the Organisation for Economic Co-operation and Development shows that “external accountability is a key driver of improvement in education and particularly important for the least advantaged”? I should say at the start that, like the NAHT, I do not dispute the need for schools to be held to account in some form for the quality of education they provide. But I do want to question the evidence used to support a statement in Bew which, I think, carries very big implications for the way schools are organised and monitored in England. The full statement on this is set out in a paragraph explaining the Bew review's remit from Michael Gove, the Education Secretary, in Bew's interim report, published in April. It says there are two broad positions to which Mr Gove has asked the review to adhere. The Bew interim report says: “Firstly, we are mindful that the Organisation for Economic Co-operation and Development (OECD) concludes that external accountability is a key driver of improvement in education and particularly important for the least advantaged, and so the Government views a system of objectively measuring pupil progress and holding schools to account as vital. Secondly, the Government has made it clear that it wants schools and teachers to be free to set their own direction, trusted to exercise their professional discretion and be accountable for the progress of the children in their care. The Secretary of State has therefore been clear that school autonomy must be accompanied by robust accountability.” So, there we have it. A system of “robust” external accountability is needed, based on OECD evidence. Mr Gove's philosophy has been to talk about handing more freedoms to schools, but then holding them to account (statistically) for their performance, mainly through test and exam results through a mixture of market forces – “high-performing” schools expand, “under-performers” close – and centralised, statist measures – government intervention to close under-performers. Reading the Bew remit, then, one might think that an English-style system of league tables, tough inspections, closure threats to underperformers and so on, with which any school leader will have become nervously familiar over the past 25 years, had received a ringing endorsement from the OECD. This is surprising, given that Mr Gove has frequently highlighted England's disappointing performance in recent rounds of the OECD's tests. But, leaving that initial scepticism aside, looking at the detail of what the OECD says, where is the evidence for the statement in the interim Bew report? Well, turning to the OECD's source of policy conclusions, its well-known PISA reading, maths and science tests of 15-year-olds across the world, in fact I found only very limited backing for the statement used in Bew. It is true that the OECD's latest PISA review does end up backing a form of accountability, and that in some circumstances this can particularly benefit poorer pupils. But you have to question the depth and generalisability of its conclusions, based on the evidence it cites and, crucially, whether PISA really provides any evidence that English-style, high-stakes or “robust” accountability is demonstrably better, as measured by countries' test scores in PISA tests. It seems clear to me that there is no such evidence in the OECD's reports. The source for the OECD's evidence on the different forms of assessment and accountability is Volume IV (http://bit.ly/kxJBXH) of the latest PISA testing study report, which analyses school systems' results in the 2009 PISA reading tests. It was published under the enticing title: “What makes a school successful?” That remit statement in Bew is related to a number of OECD findings within this report, which I now deal with in turn. - First, the OECD's report finds that nations and schools operating “standards-based examinations” tend to do better in PISA than those without. So, this would be an endorsement of our Sats system, then? Well, no, actually. Consider how the OECD report actually defines “standards-based examinations”. These are those where students' performance is “defined relative to an external standard, not relative to other students in a classroom or school”, and where the results “usually have real consequences for the student's progress or certification in the education system”. That looks to me to be an endorsement of having a system – such as, actually, our GCSEs and A-levels – in which pupils are assessed against external standards, rather than relying on teachers conducting in-class assessment without moderation to judge pupils' progress and achievements, as happens in some other countries. But this definition makes no mention of how the results are actually used: for example, if they are used for school-by-school accountability in league tables or school targets. The education I went through in the late 1980s, concluding with A-levels, would satisfy the OECD's definition of a “standards-based” system, actually, even though results were not “high stakes” for schools, but only for pupils. But our Sats would not, I think, since they do not carry large “certification” consequences for pupils. An endorsement of “standards-based examinations” in this sense, then, is not an endorsement of the English high-stakes (for schools) accountability model, but simply could be backing for having externally-marked exams for pupils in secondary school. - Second, amazingly given the remit given to Bew, the OECD report also finds that: “Across school systems, there is no measurable relationship between [the] variable uses of assessment for accountability purposes and the performance of school systems.” To those following this debate, this should be dynamite. OECD seems to be saying that there is no clear advantage (or disadvantage, to be fair) to countries using test data for “high-stakes” purposes (ie publishing results, naming and shaming underperformers, moving in and closing persistent underperformers), in PISA reading results terms, over those that do not. Equally devastatingly, the report says: “Regarding the use of student assessments...high-performing school systems tend to use the data from these assessments differently.” In other words, there is no one model for accountability success, as the remit for Bew would seem to imply. It also highlights some interesting anomalies. Shanghai, which returned the best scores in PISA 2009 of any country or region taking part, sees few schools posting assessment results publicly. In Finland, another high performer, school principals rarely use pupils' assessment results to judge teachers. - Third, the OECD report says that, while the above may be the case, “performance differences between schools with students of different social backgrounds are, on average, lower in countries that use standardised tests”. So, is that saying that England's high-stakes system protects the interests of poorer students? Well, again, I think the answer is no, or not on this evidence, since, as argued above, “standardised tests” has not been defined by the OECD as a necessarily high stakes accountability system, and our Sats would not meet the OECD's definition of “standardised tests”. Given the limitations of these findings in explaining the statement in the remit given to Bew, what, then, is its basis? Remember, then, that the OECD report finds that there is no clear relationship between the different forms of accountability and nations' education performance as measured by PISA reading test scores. But it also includes the following conclusion (deep breath needed): “Within countries where schools are held to account for their results and through posting achievement data publicly, schools that enjoy greater autonomy in resource allocation tend to do better than those with less autonomy.” The OECD argues that giving schools greater control over the resources they spend is a good thing. If countries do that, then the data seems to show that – when comparing schools with less such autonomy within a country against those with greater autonomy – it is better to have an external accountability system which includes public posting of schools' assessment results. But what is the extent of that benefit? Well, according to the OECD's report, it seems to carry a net benefit to pupils, after those pupils' socio-economic backgrounds are taken into account, of 3.2 points on the PISA reading test; ie a gain for a school given autonomy over resource spending and held to account in this way, over one not held to account in this way, of 3.2 points. To put this in context, pupils can score from 0 to more than 700 in the PISA tests, and the international average score is 500. So 3.2 points, in this very carefully-defined scenario, if I have interpreted this finding from the report correctly, looks a staggeringly low figure, given the policy weight which could be placed on this finding. In fact, this looks to be the finding that allowed Mr Gove, in an article for the Times Educational Supplement last December, to write: “PISA tells us that countries do better when they allow schools greater autonomy over how budgets are spent and pupils are taught, and that these freedoms should be combined with transparent assessment and accountability.” Yet look again at what the OECD report said. It concluded that: “The PISA results suggest that some features of autonomy and accountability are associated with better performance. However, this is not a simple relationship under which any policy to increase autonomy, accountability or choice will improve student outcomes.” Not mentioned in detail in the PISA report, either, is another paper from the OECD, a literature review from last November (http://bit.ly/eX1wKJ), which documents downsides of high-stakes test-based accountability to weigh against the gain claimed here, including that it “often creates unintended strategic behaviour”. “There is little evidence of a positive relationship between performance tables and increased student performance,” that report added. We are, then, as far as I can see, talking about a very limited conclusion here. Arguably, in fact, the dominant conclusion from PISA – which backs up what must be self-evident to anyone who has looked at the various accountability systems in detail – is that countries with very different forms of accountability have been successful, as measured by the PISA results. This, in fact, accords with research I have carried out in recent months for the NAHT. This looked at the accountability systems of 10 nations (South Korea, Finland, Japan, the Netherlands, Canada, Australia, New Zealand, France, Germany and the United States), and sought to draw conclusions. It found that only two of these countries, the US and South Korea, had truly “high-stakes” – or what Mr Gove might characterise as “rigorous” - test-based accountability systems, with punishments for underperforming schools, and the latter has only introduced such a system very recently. All others combined various forms of testing with a less punitive form of accountability than England's. There is, then, no inevitability to the English model based on what happens abroad. In conclusion, I do not think the OECD's report allows the easily-generalisable and easily-misinterpreted statement used in the Bew remit. I understand that OECD is now undertaking some more detailed work in this field, which is interesting. (See http://www.oecd.org/dataoecd/43/25/46927511.pdf) But the problem with the OECD's PISA study is also that the complexity and sheer number of results and interpretations contained within its reports allows politicians with particular reform ideas themselves to cherry-pick particular findings, confident that the public will find it difficult to query their interpretation. The detailed content and methodology behind the OECD's reports really does need to be subject to greater discussion and debate, given their influence, but perhaps I state the obvious when I say that this is not an easy job. - Bew's final report is expected by the end of June. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=464 Fri, 03 Jun 2011 09:54:21 GMT 20110603095421 Fri, 03 Jun 2011 09:54:21 GMT 20110603095421 Fri, 03 Jun 2011 09:55:35 GMT 201106030995535 Am I alone in seeing last week's Government response to the Wolf review of vocational education as a bit of a non-event? The two major aspects of the document (http://bit.ly/inhkz0) which earned media coverage both seemed to me to be underwhelming, and I am sceptical about them having long-term influence on what actually goes on in schools and colleges*. Taking these two elements in turn, the media gave some space, first, to yet another performance indicator proposed in the Government response. Introduced into GCSE league tables next year, then, will be a new measure showing, for each school, “variation in performance of low attaining pupils, high attaining pupils and those performing as expected”. In other words, league tables are to be reformed to place more emphasis on low- and high-attainers, as well as the C/D borderline candidates around which the current headline rankings revolve. In addition, average GCSE points score figures, for each school, which do not encourage a focus on the C/D borderline, will continue to be published. This sounds sensible enough, when put in such terms. The Government is right to acknowledge the side-effects of the current system, which it says can incentivise schools to neglect the academically less- and more-successful. As the paper rightly points out: “It is vital that performance indicators do not inadvertently cause schools to concentrate on particular groups of pupils at the expense of others”. “Threshold measures”, centring around a particular level of performance such as a C grade, can incentivise schools to neglect groups such as high-achievers, says the paper. To which the obvious response to ministers and civil servants, when surveying the way performance indicators are shaping up under the coalition, is: if you really think this, why are you pursuing other policies which are running in exactly the direction which you establish as problematic in this paper? Consider, first, that this new indicator will not be a stand-alone measure of school effectiveness, but one among many. As the paper admits, the average points score measure already exists – and did for many years under Labour – and yet the behaviour documented in the paper, which sees many schools focusing on middle-achievers, has continued. And the new measures for high- and low-attainers must find their place amongst a host of others. Under Labour, I've just counted 10 main GCSE indicators used to measure school effectiveness in 2009, from the proportion of pupils achieving five or more A*-Cs at GCSE or vocational equivalent to average points scores and the percentage of pupils gaining a C or better in a language, or two separate sciences. On top of this, Labour published contextual value added and “progress” measures. The coalition has now added to this: January's rankings featured 12 main indicators, including the new English Baccalaureate. Since then, the coalition has published yet more school-by-school data, including individual subject information on each school such as the number of pupils gaining A*-A in each subject in each school. The ability, then, for any new measure really to affect school-by-school decision-making, unless given heavy emphasis either by the media or by politicians in this ecosystem of competing metrics, will be limited. In reality, of course, Labour's measure of the proportion of pupils in each school gaining five or more GCSE A*-Cs (which itself was said by schools minister Nick Gibb, during a confusing appearance before the Education Select Committee last month**, to be the main “accountability” measure used in schools), and which incentivises schools to focus extra attention on C/D borderline pupils, continues to dominate. It is now, under the coalition, the indicator used to set a “floor standard” for secondary school performance. If institutions fall below ministers' statistical goals on this measure, they are liable for intervention and even closure. Arguably the only high-profile rival for schools' attention, given the emphasis given to it by the Government in recent months, has been the English Baccalaureate. And this, remember, is also a “threshold” measure, ranking schools on the proportion of pupils achieving C grades in five subject areas stipulated by ministers. So, here we have a paper supposedly worrying about the dangers of threshold indicators while other policies lay heavy weight on them. Confusing, isn't it? The second aspect of the response which commanded attention was the eye-catching idea that pupils who do not gain C grades at GCSE in English and maths by 16 should persist with the subjects to 18. This is interesting stuff, and, of course, highly worthy of debate. But, leaving aside any arguments as to its merits as an ideal, the immediate thing that strikes me is how it would be policed. It seems, from the paper, to be an intention of ministers that young people should persist with the subjects in this way. But the paper sets out only limited measures, as far as I can see, to make it happen. I have asked the Department for Education for a reaction on this, and had a response saying that businesses value maths and English achievement, so pupils should persist with the subjects. More practically, in terms of a policy change which could influence take-up, it said a review of funding for post-16 programmes was being launched. This would allow only “coherent” programmes of study to receive public cash. These would include English and maths “as appropriate”. Hmm. One other possible policing mechanism is set forward in the paper. And it is...wait for it...another performance measure, this time assessing schools' and colleges' ability to get good English and maths grades out of their post-16 students. Overall, then, this response to Wolf's attempt to bolster the place of vocational education in English education looks to me to be shaping up to have only limited impact. The really powerful potential change, put forward in November's white paper for consideration by the Wolf report and then sent back by Professor Wolf for consideration by ministers, is still yet to be set out in detail. This is how the “equivalence” system – under which non-GCSE qualifications such as General National Vocational Qualifications, BTEC Firsts and OCR Nationals were said for school results purposes to be worth up to four GCSEs – is to be reformed. Coalition ministers have constantly criticised this structure – rightly, given that multiple-GCSE equivalences were a historically curious anomaly which have certainly distorted decision-making in some schools - but we are still waiting to find out how it will change. The latest response says only that ministers are to consult over the summer on which non-GCSE qualifications are still to count in performance tables, and how. So no decision has yet been taken. Among their stipulations are that such courses must include an element of external assessment. This is presumably why Edexcel, which offers the popular BTEC Firsts which are currently entirely assessed through coursework, has already said it wants to include external assessment as an element from now on. (http://bit.ly/fmyEwY) How the equivalences change will have a big effect on numbers taking non-GCSE courses, I think. Overall, this Government response has a lightweight feel, especially given the seriousness of this subject. Clearly contestable implicit claims within it – such as the view that, since pupils with better maths and English GCSEs tend to do better in the workplace, and in further and higher education , so increasing the supply of people with such qualifications will mean that this effect continues – being cited without caveat probably do not help. Nor do hyperbolic assertions such as that it will “transform the lives of young people” (“Really?” I wondered “How many of them? And by how much?”). Lurking behind it, too, I have two other observations. First, Wolf has avoided trying to improve vocational education by getting the Government to launch another major qualification of its own. The Tomlinson review, which led to Labour's diploma, was the high water mark of this approach. Wolf's view – that government has a poor record in this field – may be fair enough but one should be under no illusions that what is being put in its place is less ambitious than that attempted under the previous government. The scale of that ambition, perhaps, is illustrated by the length of this response paper: 22 pages compared to the 93 pages of Labour's 2005 response to Tomlinson. Second is the implicit view that the answer to so many problems in education, as far as this and the previous government is concerned, is to publish another performance indicator. There is further evidence of this in the response paper, which highlights how the Government's recent green paper on special education aims to improve provision. The response paper again emphasises performance metrics as a possible solution to the problems facing SEN education, revealing “plans for new indicators in the performance tables relating to the lowest attaining pupils between Key Stage 1 and 2, and Key Stage 2 and 4 in English and maths”. Reading this stuff, it is difficult to know whether to laugh or cry. Another few columns on a league table, then, and all will be right with the world, is the implication. This may fit with a quasi-religious view of public service reform which says that the answer to any problem is more measurement and more transparency, but it is to say the least highly simplistic. For all the occasionally welcome acknowledgements here of the side-effects of specific indicators within the current system – including an admission that league tables should never force institutions to put their own interests ahead of those they educate – the underlying philosophy that says that ranking and measurement systems of some kind are always to be pursued as a centrepiece around which the rest of education policy must revolve remains intact. Yet just how serious some of the implications of the current set-up can be was underscored, for me, by another report last week, by the Advisory Committee on Mathematics Education. Staggeringly, this quoted research which found that in around a quarter of schools which entered pupils early for maths GCSE, candidates were stopped from studying the subject further if they obtained a C grade in early exams. With results indicators, for the school, hingeing around C grade success in English and maths, this looks to be a clear case of accountability pressures pushing school needs above those of their pupils. The report highlights the pressures on National Challenge schools, at the bottom of GCSE league tables, to pursue such an approach. This is, if one steps back from the logic of school-by-school ranking for a second, scandalous stuff: if I were a parent or student affected by this policy, I would be outraged to find a pupil prevented from continuing with a course because they were found to have achieved all they needed to, from the school's perspective, in gaining a C grade. Dame Julia Higgins, chair of ACME, said: “It's no longer a case of careful selection of the brightest students being pushed through early – it's whole cohorts now, whether or not it's in their long term interests as individuals. “The pressure on schools to improve their standing in the league tables provides an incentive to act in the school's best interests rather than those of the individual students.” Parts of the curriculum not needed for a C grade may be “skipped” in some schools for the reasons above, says the report. This very strongly-worded document can be read at http://bit.ly/mCEONK . Readers of this blog will need no reminding that the underlying philosophy behind holding schools to account in this way, as implicitly supported by the Government's response to Wolf, is a huge problem for England's education system. * I am concentrating, here, mainly on school provision. The paper also includes sections on apprenticeships, which I haven't written extensively in the past. ** See: http://bit.ly/j1rRKV http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=459 Fri, 20 May 2011 10:17:01 GMT 20110520101701 Fri, 20 May 2011 10:17:01 GMT 20110520101701 Fri, 20 May 2011 10:17:01 GMT 2011052010101701 It is, perhaps, the biggest dispute underlying the government's current review of Key Stage 2 assessment. Yet it remains, I think, largely hidden below the surface of last month's interim report from Lord Bew, which summarised the evidence so far. The dispute centres on a question which runs as follows. Is England's system of centralised monitoring, pressure and support for schools, which uses test and exam data as its informational bedrock, a force for good or bad in our education system? Many people, including me, observe the effects of this system and end up heavily sceptical. The interim report states that 62 per cent of the nearly 4,000 respondents to the review's online consultation exercise had “concerns about the way national curriculum test data is used”, and concluded that “few would disagree that the current system of testing, assessment and accountability has led to undesired consequences”. Many were concerned about the extent of test preparation, especially in year six. Yet supporters of test-based monitoring, which reached its apotheosis under New Labour, were also referenced in the report. Their argument is usually sketched out in the following terms. The system of collecting data on pupils' performance in nationally-designed or regulated assessments has been positive, since it ensures that policy-makers can keep tabs on the “best-performing”, and crucially, the “worst performing”, schools. Armed with this information, policy-makers can then supervise a system which intervenes in those schools whose results suggest they need more attention, either through carrots – more support for “underperforming” schools – or sticks – closure threats and “notices to improve” – or a mixture of both carrot and stick – programmes couched in the language of support which nevertheless will be perceived as containing an implicit threat. In evidence summarised in the interim report, Professor Pam Sammons, of Oxford University, argued that the attention placed on schools through test results had led to a “combination of policy attention, resources and considerable support and professional development” being directed at schools causing concern. This had helped to raise standards and to reduce the number of “poorly performing” schools in England. In other words, the centralised performance monitoring system had been, at least in part, a force for good. Sir Michael Barber, the former head of Labour's school Standards and Effectiveness Unit and Tony Blair's public service “Delivery Unit” who is now a senior figure at the management consultants McKinsey, was also supportive of the idea that centrally-collected data collection was valuable, says the Bew report. He “argued that if schools (and government) are to make decisions based on evidence, they need regular assessment and data which are comparable over time. This enables analysis of the performance of different groups of pupils”, it said. Sir Michael has been, in fact, an arch-centraliser. His 2007 book, “Instruction to Deliver”, is essentially an argument that public services can be improved by those at the centre focusing relentlessly on statistical measures of performance. In the book, he writes of the benefits of having "restless, sleepness nights worrying about where the next percentage point [in test result improvements] was coming from"; cites the gains to be made by the school advisers working for the national literacy and numeracy strategies being “single issue fanatics driven by a mission to improve test scores”; and laments that it is “the road back to the 1970s to say each teacher knows best in their own classroom”. Incidentally, Sir Michael has also confessed to “loving” league tables and also, in Instruction to Deliver, bemoans the departure of Chris Woodhead as chief inspector in 2000, writing that “a key lever in the strategy for school improvement – Ofsted – had been weakened”. Conor Ryan, former education adviser to David Blunkett as education secretary and Tony Blair, wrote in 2009: “The only way that we can know whether individual primary schools are doing their job is through independently set and marked tests.” The belief that Whitehall can improve schools thanks to its statistical monitoring system – arguably a form of surveillance through data – was encapsulated a few years ago when it emerged that civil servants had created a system called “the Bridge”, whereby the results of any local authority or school in the country could be called up on a large TV screen at the Department for Children, Schools and Families, and their scores compared to others on many indicators. (See http://bit.ly/jYKF82) I understand that “the Bridge” is no longer in existence. This embodies a view which says that ultimately, nationally-elected politicians and the civil servants working for them are the guardians of the public interest in education. I point out these viewpoints not to rubbish them but to argue that there is a profound difference of opinion as to the benefits of centralised performance monitoring in schools between some influential policy-makers and, I would guess, the majority of school leaders and teachers. Some centralised policies clearly can work. Evidence put forward to the Cambridge Primary Review, for example, suggests that the National Numeracy Strategy – centralised instructions to maths teachers – has been widely perceived as beneficial. There have been powerful arguments in favour of some kind of national curriculum. I also believe that central government has potentially a huge amount to offer if it sought to provide a supportive structure for improving teaching. But there have undoubtedly been major problems with the use of test data for central monitoring. A recent Organisation for Economic Co-operation and Development report on the UK pointed out that “the use of [centralised test- and exam-based] benchmarking in England is more widespread than in virtually any other OECD country.” (http://bit.ly/gHMTRe) England is certainly highly unusual in the degree of this kind of centralisation: few, if any, other large countries see central government exerting such direct pressure on schools through statistical monitoring. The OECD's report continued: “Transparent and accurate benchmarking procedures are crucial for measuring student and school performance, but 'high-stake' tests can produce perverse incentives.” The report argues that while England's system is unusually developed in the amount of data it collects, its results on the OECD PISA's tests have not been improving – and the overall evidence for England from international tests is mixed. The OECD's report adds that England's focus on test scores incentivises teaching to the test and “strategic behaviour”, including “negligence of non-cognitive skill formation”. Sir Michael's claim that English education benefits from policy-makers having sleepness nights over single percentage point improvements in test scores always was absurd, when looked at from anything other than a political point of view, since the way national test results are calculated* means they are almost certainly not reliable enough to be sure that, when national scores climb by a percentage point or two, there has been any meaningful change in underlying standards. And the one-dimensionality of this focus, which is no doubt seen as a strength in a policy world desperate for simply measurable improvements to take back to a sceptical public, is undoubtedly a weakness when it gets realised in schools which become one-dimensional in the education they offer to pupils. This, of course, is the pressure created by such a system and it is interesting that Sir Michael's book offers no insights into the overall effects of statistical demands at the level of the classroom. For me, there is another concern underlying England's system. At its worst, it has been a form of institutionalised bullying. Under the targets system, politicians and civil servants set the education system as a whole improvement goals in terms of test and exam results. They then cajoled – bullied, in the worst cases – local authorities into improving on those measures. Local authorities then pressurised and, in some cases I am sure, bullied schools into seeking improvement. This system did produce test score gains, especially in its early phase in the late 1990s, and GCSE results have continued to improve. But I have heard too many stories about the human cost in terms of the working atmosphere in individual schools to be anything other than sceptical about this regime. Certainly, if I were a parent I would not want my child educated in a school under the pressure some of them have been placed for short-term results improvements above all else. Ultimately, of course, many heads have been sceptical about this system, in part because they do not believe the basic premise of the arguments above – that the true source of wisdom about how individual schools should improve resides in a set of spreadsheets overseen from Whitehall. The coalition is retaining some of Labour's apparatus of centralised control, for example through the use of “floor standards”, statistical test and exam targets below which schools will be eligible for intervention including closure if they fail to make gains. It is, though, placing a greater rhetorical emphasis on using test and exam scores to put pressure on schools to improve through parental choice. But this latter approach still retains the idea that true power to set the goals for schools remains outside of their control: in the hands of those designing the tests, exams and statistical metrics by which institutions will be measured; and in the hands of parents choosing schools. Teachers, though, will surely know that the most powerful spur for improvement usually comes from within. My guess is that what is in essence an externally-imposed improvement regime will continue to alienate large sections of the profession. I wonder if the Bew review, impressive in my view so far in the scope of its consideration of different points of view, is going to take these issues on board. *Changing the number of marks needed for a level four at Key Stage 2, for example, by a single mark can change the proportion of pupils achieving level four by a couple of percentage points. Yet those deciding the number of marks needed for particular national curriculum levels have always faced uncertainty in their decisions: a government report on the reliability of the testing system by the former senior Ofsted inspector and government adviser Sir Jim Rose concluded that: “Where such small margins are involved, it becomes obvious that testing is not an exact science. The justification for choosing one 'pass mark' over another can be barely discernible”. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=454 Wed, 04 May 2011 13:42:36 GMT 20110504134236 Wed, 04 May 2011 13:42:36 GMT 20110504134236 Wed, 04 May 2011 13:44:15 GMT 201105041314415 It is one of the paradoxes of national curriculum assessment. More than 20 years after the initial foundations were laid for our current testing system, serious questions are still being asked about what the point of it is. This is not a flippant suggestion: debate about the purposes of high-stakes assessments which dominate many pupils' lives for months if not years is pervasive and, as two recent papers illustrate, ongoing. Lord Bew's interim report on Key Stage 2 assessment, published this month, says it received 4,000 responses to an online consultation exercise; took verbal evidence from 50 stakeholders and also received “many” written submissions. It also says: “Almost all respondents have at some point questioned the purposes of statutory assessment.” It adds: “There seems to be widespread concern that there are too many purposes, which can often conflict with one another. 71 per cent of online consultation respondents [that would be at least 2,800 respondents, then] believe strongly that the current system does not achieve effectively what they perceive to be the most important purpose.” The report then cites evidence that statutory assessments are used for at least 16 purposes, for some of which they are better suited than others. It reports that in 2008, the backbench Children, Schools and Families select committee recommended that the use of national testing for multiple purposes, including for measuring pupil attainment, school and teacher accountability, should stop. All of this sounds unsurprising, to me. The enduring central mystery, for many teachers and parents, I think, about national curriculum tests has been whether they are supposed to be mainly a test of the pupil, or of the school. The largely unstated answer must be that their main use is to evaluate schools and teachers, rather than children, since one could certainly think of assessments which would be of more use, in terms of genuinely helping a child progress, than a one-off series of tests occurring just as the pupil is about to move from one school , and one set of teachers, to another. But this has never, quite, been said officially, as far as I am aware. For secondary exams, the position looks different, with GCSEs, A-levels and other certificates meant (and originally, only meant) to assess individual pupils' mastery of a subject. Does that mean they can generate truly useful data on, for example, national education performance? I am not alone as a sceptic. Now another voice has been added to those urging ministers to define the purposes of all assessments, from Sats tests to secondary exams. In a report discussed at a conference held last month by the exams regulator Ofqual, a group of assessment experts commissioned by Ofqual to investigate assessment reliability had some interesting thoughts. Among the report's 11 recommendations, their first was: “Ofqual should outline the primary purpose of each qualification and Ofqual should regulate against that purpose”. This is very interesting. Backing its view, the report cites a 2010 draft of the “European Framework for Educational Assessment”. This says the goal of an assessment should be specified, and include: “what are the assessment measures”; “what inferences can be drawn from the results”; “who the intended users are”; and “who the intended candidates are”. Professor Jo-Anne Baird, of the University of Bristol, who chaired the group reporting to Ofqual, told the conference that, while the use of secondary exam data as measures of students' achievement was broadly appropriate (both largely valid and largely reliable, to use the jargon), their use in school league table results was neither valid nor reliable. She added: “Measuring school or teacher effectiveness reliably is very different from measuring student achievement reliably. You would collect different data, analyse it differently and so on [if the prime aim of secondary examinations were to measure other things than student achievement]. “Regulation would also involve different activities if these were the purposes of assessment.” Why does this matter? Well, I think over-interpretation of data, often based on assessment results, is a huge problem in the English education system. Clarity from assessment designers, in particular, about what inferences data generated through results can support, and, most importantly, what they cannot, is therefore potentially powerful. I would offer two concrete examples of what can go wrong with the way assessment data has been used recently. The first concerns new school-by-school financial data published by the government in January. This sees column after column of information on the funding and spending of each school followed, ludicrously I think, at the end by a single statistic setting out the school's “raw” test/exam results. The inference the public is being asked to draw, I would suggest, is nonsensical. It is that the “best” schools are those achieving the highest results for the lowest expenditure. But this is ludicrously simplistic: those in better-off areas may well be lower funded because they receive less funding for deprivation, but may well also achieve better results because of their catchment. This provides absolutely no evidence that they are the “best” schools. Assessment data do not support the obvious inference that a member of the public might make.* While it would be difficult for it to be stated in advance that assessment data should not be used for the purpose to which it is apparently intended to be put here, I wonder whether government data releases of this kind might benefit from some external and independent expert scrutiny before publication, to guard against this kind of problem. The second example is more directly related to assessment. I spoke at Ofqual's conference, and talked about a graphical illustration of pupils' progress across key stages, as measured by test results, which was popular among education officials and advisers during the latter years of the Labour government. This considered the test results of pupils at key stage 3 (while these still existed), compared them to those the same children at key stage 2, and used this to generate figures representing the varying degrees of “progress” made by children over the three years between the two sets of tests. Similar exercises were carried out comparing key stage 2 test results with KS1 teacher assessments. Children achieving the same level in both key stages were said, in a presentation I watched by the government's standards adviser Sue Hackman, to “be stuck”. They were represented graphically by a “stick man” figure coloured pink in this case to represent lack of progress. The more stick men in this colour for each school, or across the nation as a whole, the more “stuck” children there were. I argued in my speech that, being conscious of the views of people expert in assessment design, I did not think that test results supported the certainty with which this interpretation was being made. This was because of the degree of error around pupils' performance in both sets of assessments: for many reasons, including a child simply over- or under-performing on the day, we cannot be certain on either occasion that the test score represented their “true” ability. Thus, we should not use test data to make categoric assertions such as that a child with the same set of test results on two occasions had made “no progress” in a particular subject as a whole over a number of years. Having made this comment, the only complaint I got afterwards was from an assessment expert who argued it did not go far enough: national curriculum level judgements at different key stages were defined in different ways, so were not comparable at all, she suggested. A warning as to the comparability of test data over different key stages would have been very beneficial, then, I think, to stop this over-interpretation. So how is this to be resolved? Well, the interim Bew report concludes that: “One of the key early decisions for this review will be to define clearly the purposes of statutory assessment in primary education and what we want to achieve it in the future. The system can then be designed to serve these purposes”. That suggests a complete redesign for our primary assessment system, almost from first principles. That is a very ambitious agenda for change. Whether it can be pulled off, in a time of very tight budgets, and with the review due to produce a final report by June in advance of the associated national curriculum inquiry, must be a question of some doubt. *One could also argue that spending money efficiently is of little direct, primary interest to a member of the public in choosing a school for their child. Why would a parent faced with a choice between an expensive, but effective school and a much less expensive but slightly less effective school choose the latter over the former, I wonder? But that is another subject...) http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=449 Tue, 26 Apr 2011 16:35:59 GMT 20110426163559 Tue, 26 Apr 2011 16:35:59 GMT 20110426163559 Tue, 26 Apr 2011 16:36:22 GMT 201104261643622 Launching the publication of yet another set of league tables last week, Michael Gove said that its advent would help move power over schools away from government and towards those who use the data, or “parents, local authorities and all those who are deeply and intimately involved in education”, as he put it. The idea, he suggested, was a democratisation of information, with ministers no longer holding back statistics. Lots of new figures – in this case, on the performance of each school in individual GCSE subjects – would decentralise the system, with parents free to choose how they ranked schools. I wonder, though, just how powerful that notion is, and how it will fare against the more centralising instincts also in evidence under the coalition. Another way of asking the question above is to say: how much power will these new tables genuinely hand to parents? And how does this compare to the centralisation drive in other aspects of policy? The first, and most obvious, thing to say about the new GCSE information as presented last Thursday is that it is hardly being put forward in a manner which will maximise its usefulness to the average person in the street. That's probably the polite way of putting it. People used to viewing and manipulating school statistics – including me - have been confused, initially at least, by this new information, so what use the average parent unfamiliar with this subject can make of it is anyone's guess. There are several ways in which the data as presented could be seen as confusing, if parents are really meant to be the main audience. First, it is not obvious where to go to access the data. At the moment, there is a press release on the Department for Education's home page, (http://bit.ly/ihaRP4) under the headline “parents get the full facts about every secondary school”. “Parents can from today see how every single secondary school in England performs in each GCSE subject,” begins the release. Click on the second link on the right hand side, and you come to a page full of national statistical information. But it is not immediately clear, on this page, where the subject-level data alluded to in the press release is. Hands up, here, I could not find it, on first looking, and had to be helped by a press officer and a former TES colleague. In fact, the information is available if you scroll down to “publication's underlying data” and then click on “archive:zip”. This is hardly the easiest route in. Second, once you get this far, you need Excel spreadsheet manipulation skills to make much sense of the data. Members of the public will come across a set of data files, set out over several spreadsheets. Every school in England is listed in this national statistical set, against which are then charted the number of entries each school had in each of a vast array of GCSE subjects, followed by the numbers of pupils passing each of those subjects at various levels. Other data, including that related to the new English Baccalaureate, also feature. Anyone wanting to get the percentages of pupils successful in each subject, for a particular school, will have to cut and paste data from two of the spreadsheets and then do some number manipulation, as far as I can see. Any sense of how entry numbers in each subject compare to the cohort number for that school is also something which can be worked out from the data, but, again, you need spreadsheet skills. Third, while this new data purports to “present the full facts about every secondary school”, in fact it is incomplete. The reason is that many of the qualifications now taken in English secondaries – such as BTEC Firsts, OCR Nationals and the ASDAN awards – are not included. Schools with pupils taking these courses, plus one exam board's version of International GCSE courses which were not accredited by the government in time for this year's tables' publication, will therefore not have any record of entries or performance in these subjects. There has been a lot of controversy about these “GCSE equivalent” courses, with ministers arguing that some schools put pupils in for them to gain credit under Labour's league table measures. I argue this myself, and there seems wide acceptance now that the existing equivalence formula overvalued these courses. However, not to include them at all, in data billed as providing a “full picture”, really risks being very misleading to parents. It suggests that the only qualifications ministers believe to be of value for this age group are GCSEs and those IGCSEs that are accredited. Remember that secondaries were encouraged, for several years under Labour, to broaden the curriculum they offer for 14- to 19-year-olds. If they followed that message and did so, GCSEs will now make up only a part of what they offer to pupils. Yet publishing data in relation only to GCSEs means parents – encouraged to make overall judgements on schools on this basis - are being presented with information only on a part of the curriculum on offer. A school offering, for example, New Labour's flagship diploma qualification which was meant to occupy pupils for three days a week, would find none of its charges' achievements in this course recognised through this information. How, then, could full comparisons be possible for parents? To come back to one of the original questions, is it really true that these spreadsheets will hand power to parents? Will they influence what happens in schools, or “drive schools to improve standards across the board”, as the press release puts it? Well, only if you assume that most or even many parents are able to wade through this stuff and make sense of it. That may sound patronising towards the average parent, but I would suggest it is the truth: this information is simply not presented in a user-friendly format. Michael Gove has said in the past, I think, that publishing data like this will not just be of use to parents directly. Instead, all sorts of organisations will be able to access the data files and thus use them to present information in whatever manner they choose. It might be, then, that another organisation will make it more immediately palatable. To which, one could respond: fair enough, but at least one organisation has been doing this for years. The Good Schools Guide has had access to data underlying the rankings and has been using it to produce reports on individual schools which, though they reflect aspects the editors of the guide have chosen to highlight in individual school results, are certainly more accessible than that presented in these spreadsheets. The notion of opening up more data and thus “handing power to parents”, then, makes a good soundbite and political campaigning point. But a government really committed to making a difference in this area would be making more of an effort to offer data in a more digestible form. I could be accused of churlishness here. Information such as this can be used by journalists and others to build up a more detailed picture of what has been going on in schools than has been possible – at least from centrally published information - in the past, and I have written that simply relying on the headline “five or more GCSE A*-C passes, with our without English and maths” figures used under the latter years on Labour risks masking the detail of school curricula. There are, clearly, potentially interesting articles to be written on the basis of these new data. And if accountability really were being decentralised to become more of a direct relationship between schools and parents, rather than being routed through Whitehall as has tended to happen for the past 20 years, potentially that would be very interesting and, probably, welcome. However, for the reasons argued above, I don't think, first, what has happened here has been the opening up of a more direct, deeper form of accountability between schools and parents. Indeed, it could be argued that what is on offer still does not provide the clarity of subject-by-subject information which was on offer to parents a few years ago. Until 2005, when Labour replaced this with “school profiles”, as I understand it, performance by schools in individual GCSE subjects had to be published by each institution as part of their prospectuses. In this way, parents were given information on a school-by-school basis in what almost certainly amounted to a much more digestible form than occurred last week. If they wanted information on another local school, this would not be available through Whitehall, but by looking at this second school's prospectus. This solution, though not without its problems, at least had the advantage of simplicity and supporting a more localised version of accountability. The second issue with this notion of a decentralisation of accountability is to compare it with other, centralising, powers which remain. There is not space to go into all of the detail on this here, but suffice it to say, I think central control of education through performance monitoring has not gone away. Last month, for example, Mr Gove wrote (http://bit.ly/gWNSWq) to all local authorities (and academy sponsors) asking them to draw up plans, to be submitted to him within seven weeks, on how to improve schools which were performing below certain “floor standards”, or targets, of GCSE and key stage 2 performance according to centrally-designed metrics. This has come with the ultimate sanction of schools which fail to improve being closed and turned into academies. Although some “outstanding” schools are being freed from Ofsted visits, for others the monitoring through Ofsted is going to become more intense. It can be argued that Ofsted itself, in publishing a consultation document last month promising to concentrate more tightly on aspects of education which match the priorities set out by ministers, has underscored again how it is guided by central government priorities, and thus part of the centralising/controlling policy apparatus emanating from the Secretary of State. The national curriculum review will mean that schools are likely to get more freedom in some aspects of what they teach. But a central “core” looks like being more closely prescribed. Finally, Mr Gove's introduction, without much consultation, of the English Baccalaureate GCSE measure underscores the impact of central direction to shape schools' priorities. Summing up, and looking at the balance of centralising versus decentralising tendencies, it may be that some more successful schools will have more freedom under the new regime. But those with average or below average raw results are unlikely to see that democratisation of accountability, with power transferring from ministers towards the direct relationship between schools and parents. I may be proved wrong, but it seems to me unlikely that last week's GCSE publication, though fitting very neatly with a rhetorical and ideological commitment to this agenda and valued by journalists as a potential source of more stories, will be powerful enough actually to make much difference either to how most parents weigh schools' qualities, or to affect greatly the balance of accountability in schools. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=442 Tue, 05 Apr 2011 13:41:36 GMT 20110405134136 Tue, 05 Apr 2011 13:41:36 GMT 20110405134136 Tue, 05 Apr 2011 13:41:36 GMT 201104051314136 I wonder what accountability and school management will look like in 10 years' time, if we are truly on the brink of a major movement away from local authority co-ordination of schools. Further, will head teachers be in control of what goes on in their schools to a greater, or lesser, degree than happens now? These questions were in the forefront of my mind, on reading a fascinating and combative document submitted to the committee of MPs now scrutinising the Education Bill. The evidence submission (http://bit.ly/elWSpO ) came from John Burn, the former principal of Emmanuel City Technology College, a school in Gateshead with an interesting history. Aside from once being at the centre of national controversy over alleged teaching of creationism, the school has experienced a transition from its origins as a CTC to one clustered with a small group of schools, to a CTC in a much larger group of academies, according to this document. Mr Burn's central claim is that its movement to become one element of that larger federation diminished its autonomy. Moreover, schools making such moves arguably end up with less freedom than local authority institutions, this long-term seeming advocate of non-LA schools suggests. And organisations making such changes are worryingly remote and unaccountable. In detail, Mr Burn says that after retiring as principal of Emmanuel, he worked with Sir Peter Vardy, its sponsor, to advise on the setting up of three academies in the north of England, also sponsored by Sir Peter. He adds: “After a number of years this sponsor felt unable to sustain a continuing family interest in this work and in effect gave away Emmanuel College and its three sister schools to another federation.” However, says the submission, this had been carried out as a “fait accompli”. “Everything was done behind closed doors without the knowledge of the school principals, the governors, the parents or the local communities which these schools serve.” The new organisation taking control was the United Learning Trust, which is now the largest sponsor of academies. This sponsor, writes Mr Burn, has “introduced a large and, in my view, unnecessary and unaffordable system of central services based in an office in Durham”. The central management of academy chains such as this is “apparently uninspected”, and unaccountable to its own school principals, he suggests. Mr Burn adds: “In many ways, the principals of these institutions have less freedom to manage their affairs than their counterparts in maintained schools. The governing bodies are effectively stripped of all powers and responsibilities, which are transferred to ULT's national board hundreds of miles away in Northamptonshire.” There was nothing, he added, to stop central academy bureaucracies deciding schools' spending priorities and then “paying for them from the unilateral top-slicing of their schools' funds”. Mr Burn proposes a change to the legislation so that “successful academies within federations” retain their self-governing status, rather than having to follow central directives. “Unless this change is made,” he writes, “schools which join federations seeking the freedoms promised them as academies may opt out of their local authority only to find themselves ultimately within a structure that is more restrictive, less accountable and from which there is no escape.” At time of writing, there was no sign of any reaction from ministers or the committee. I've also asked for a response from the ULT, but have yet to receive one. The United Learning Trust website says the four schools mentioned by Mr Burn are about to join the ULT, but that this will happen in the “near future”. This would take its organisation to 21 state-funded schools. All of this reminded me strongly of reading a document written in 2009 which was raising similar concerns about groups of charter schools – privately-run, largely state-funded institutions on which the UK government is keen - in the United States. The report, “Sweating the Big Stuff: A Progress Report on the Movement to Scale Up the Nation's Best Charter Schools”, was written by a charter school supporter, Thomas Toch, who nevertheless wanted to document the challenges facing the movement. * (http://bit.ly/9rAZtR) One of its most telling conclusions, for this reader, was that although charter organisations were set up partly in a bid to reduce the perceived bureaucracy of local government, in fact they had struggled to do so. They had needed to run central offices to support their schools, and the tendency seemed to be a drift towards standardised management practice. The report lists five well-known charter school operators which it suggests were spending 16-18 per cent of their revenues on “central office” functions. One group of 11 institutions, called Uncommon Schools, appeared to have incredibly centralised strategies. Its schools, he wrote, “adhere to a nearly 300-page instructional manual...It details a wide range of techniques that teachers should use in their classrooms, including six different types of questions they're expected to ask their students”. Charter school organisations, writes Toch, have ended up “taking on the complexion of the traditional public school systems that charter schools were founded to counter”, because they believed bureaucracy and standardisation were cost-effective ways of pursuing higher standards. Why does any of this matter? Well, it seems to me that, with the Government here seeking to encourage schools to become academies, two distinct models are emerging. To use an analogy from the private sector, which seems apt, they could be characterised as the independent high street shop model, and the chain store branch model. The high street shop model would see a school leaving a local authority to operate on its own, with ministers claiming at least that this would leave it more in charge of its own affairs. The chain store branch model would see large groups of academies set up across England under rival sponsoring organisations, competing with each other for the “business” of parents. I wonder which model will win, or whether local authority schools will persist. Arguably, the coalition's move to change Labour's scheme by allowing many more schools to become academies easily, without sponsorship, handed power to the high street shop model, since many schools would go it alone. Yet two developments in recent days seem to suggest a possible bigger future for the academy chain version. First, the TES (http://bit.ly/dTJnNM) reported an ambition at E-Act, an academies group now running 11 schools, to operate 250 academies within five years. Second, changes to the rules on applications by groups to run free schools, which technically are types of academies, seem to have strengthened the hand of large private academy sponsors, rather than individual groups of parents. At least, that is the view from this blog (http://bit.ly/eXyrtn) by Toby Young, the founder of the West London Free School. The practice of schools opting away from local authorities is often sold as promoting autonomy, and also innovation. But to what extent is this really the case? I think this, at least, is a very open question. I remember, at this point, hearing a lecture last autumn by Hugh Lauder of Bath University on how the modern global economy works now, based on detailed analyses and interviews with those working in the private sector. Among his arguments were that much modern corporate practice is incredibly standardised. Local bank managers, for example, are given little freedom on whether or not to hand out a loan to a local company or individual, but simply following a centralised computer program, he said. (Details of the book on this co-written by Prof Lauder are here: http://amzn.to/hXgHJa). If corporate management of education is really about to take off, and Mr Burn's fears are realised, the danger might be that we go from one form of bureaucracy, which at least in theory is locally grounded, reasonably transparent and accountable, to another, which is not. This could be based on a centralised model of head office control which is surely popular in at least large parts of the private sector. If academy chains really are going to be the future of a good part of England's schools system, Mr Burn's claims deserve some serious scrutiny. *“Sweating the Big Stuff...” was actually only a draft paper, whose contents were heavily edited before publication. This has its own back story. See here if interested: http://bit.ly/anTBkO http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=438 Thu, 24 Mar 2011 14:51:55 GMT 20110324145155 Thu, 24 Mar 2011 14:51:55 GMT 20110324145155 Thu, 24 Mar 2011 14:51:55 GMT 201103241425155 Alison Wolf, a professor of public sector management at King's College, London, is an impressively independent-minded academic, who has seemed to win respect from people from across the political spectrum. Her report for the government on vocational education, published last week, comes with both geographical breadth, in its frequent references to policy traditions and developments abroad, and historical depth, in its allusions to more than a century of efforts to improve the quality and standing of work-related learning. It also impressively seeks to look in some detail at the reality of the labour market facing today's school leavers. The report, with its 27 broad-ranging recommendations, will also strike many as refreshing, in bringing an outsider's perspective to a policy sector which, she suggests, has been beset by often well-meaning, but frequently dysfunctional, reform attempts by previous governments of both political persuasions. The detail of what she has to say on the effects of current league table indicators is also welcome, in my view. But ultimately, there is a sense of disappointment that the report fails to follow up on its observations by asking more fundamental, and I think unavoidable, questions about rankings systems of all kinds. In fact, its treatment of questions as to the wider effects of the league table mentality in schools looks heavily skewed towards the politically circumscribed status quo. So, first, the detail. It is worth quoting verbatim what Wolf says about the current effects of league tables, specifically with respect to the impact she says they have had on the number of vocational qualifications offered to 14- to 16-year-olds in recent years. The contribution of what are badged vocational awards – including BTEC Firsts, OCR Nationals and Certificates in Adult Literacy and Numeracy – to annual national figures listing the proportions of pupils gaining five or more A*-C passes in GCSE or vocational equivalent has soared from seven per cent in 2005 to 23 per cent last year. The report says the league table equivalence system, which has seen these courses deemed to be worth up to four GCSEs, is the cause. The problem is, says the report, that they can be worth more to the school than to the pupil in the long run. The report says: “Young people are being entered for 'vocational' awards at the end of KS4 for reasons which have nothing to do with their own long-term interests, within education or the labour market. They can and do find that they are unable, as a result, to progress to the courses they want and have been led to expect they will enter.” Arguably the most devastating paragraph in the whole report says, on page 108, that: “Many education sector respondents also agreed that recent curricular changes at Key Stage 4 had been made largely in response to performance table pressures, not students' best interests.” Both Professor Wolf, at the conference where the review was launched, and Michael Gove, in his forward to the review, attacked as “immoral” any move by schools to push pupils towards courses which might not be useful to their futures because of the pressures of league tables. Mr Gove writes, using an ugly word beloved of Ofsted, which I wouldn't use here: “Performance tables incentivise schools to offer...inadequate qualifications”. It seems to me that results pressures are clearly not the only factor at play in the move in recent years to offer a broader curriculum at key stage 4. The Labour government actively encouraged this, and even the report says that there was “overwhelming consensus”, among respondents to the review, for vocational courses to be available to all 14-16 year-olds. However, the high weighting given to non-GCSE qualifications was always going to create a risk, in my view, that some institutions would push students towards them mainly because of the worth the courses had to school and college statistics, even when this was out of step with their “real world” value. There is too much pressure within the system for institutions to raise their results for this not to happen, in some cases. The comments in and around the report confirm this and are, in fact, the strongest attack on the effects of particular league table indicators that I can recall in any government-commissioned investigation in recent years. The great pity is that the report then seeks to shut down discussion as to the extent to which this “immoral” behaviour reflects a likely effect of all systems of this kind, rather than just being a product of individual measures. The implication, then, is that there is a need to change the indicators against which the rankings are compiled – towards an average points score system with the impact of non-GCSE courses dramatically reduced – rather than looking at more fundamental reform. The report says: “It would be naive (and incorrect) to believe that simply abolishing the current set of incentives would deliver improvements in quality and efficiency. Students and their families need to know whether an institution is performing well. Governments need to ensure that information is not only provided, but is accurate and appropriate.” In other words, there must be league tables, even though their effects in the current context have been shown to be so pernicious. Later in the report, there is an attempt to weigh the effects of league tables in a rounder sense. But it amounts to two paragraphs. It says: “Performance management systems, especially league tables, have proven to be extremely powerful tools for controlling institutions' behaviour, used widely by successive governments. “This is for good and bad: performance tables have created perverse incentives, as we have seen, although we also have evidence that such external measurements raise performance.” The final part of that sentence carries a link to a note referencing just one study on this much-debated and bitterly contentious subject. When I read this, my jaw dropped. The research was an analysis carried out last year by academics at Bristol University, which looked at secondary exam performance in England, which has retained league tables, and Wales, which dropped them in 2001. It found that results rose faster in England, and thus concluded that league tables, and naming and shaming schools, “work”. I have blogged at length on this study here. (http://bit.ly/dqWLc0). I believe it is fundamentally flawed, because the central assumption on which it is based – that the only meaningful difference between the two countries over the period, other than funding, was the existence of league tables in England and not in Wales – is incorrect, or at least very far from proven in the Bristol study. It also seems to me that part of a possible explanation for England's greater results success, the move towards the very vocational qualifications that Wolf decries, was too quickly discounted by the Bristol research. I was thus astonished that it seemed to be being used as the sole evidence as to whether or not league tables actually do “raise performance”. Although the Bristol work is cited, many studies questioning the value of league tables by the statistician Harvey Goldstein, who has also been based at Bristol, are not. I wonder whether a truly balanced evaluation of the full evidence on whether league tables have actually helped or hindered English education was simply ruled out as politically impossible. But it does limit this review. Instead, we have a move to change the indicators again. Since the tables' introduction 20 years ago, we have moved from them providing simply “raw scores” information, to value added data, since the raw scores were seen, rightly, to say more about the characteristics of pupils than of schools. Then we moved to contextual value-added, because even attempts to measure pupil progress, through a value added system, would disadvantage some schools by failing fully to take into account different levels of support in their home backgrounds. There was the change to measuring five or more GCSE A*-Cs including English and maths, because of concerns that schools could boost the five or more A*-Cs indicator through non-core qualifications. In the last years of the Labour government, a new “report card” scheme was proposed, incorporating wider measures of school performance than exam results, because of worries that results pressures meant assessment grades were overemphasised. We also had moves, under this and the previous government, to go back to a more “raw” method of measuring pupil progress, since contextual value-added was seen to be too complicated. Under the coalition, we have had the English baccalaureate, which came about amid ministers' fears that existing GCSE measures encouraged schools to push pupils away from traditional academic subjects. Now Wolf seems to be proposing a new average points score measure – although the precise details are yet to be decided - to stop schools neglecting lower-achieving teenagers in their drive to secure the C grades around which the current rankings revolve. Somewhere along the line, perhaps someone might conclude that there are deeper problems with league tables of all kinds, alongside the issues with the precise nature of the indicator under discussion. For me, the lesson is simple. Wolf has said that the current system encourages schools to put their own league table interests ahead of their pupils', in pushing them towards vocational options, and that this is “immoral”. But it is precisely the function of results-driven accountability to give schools self-interested reasons to perform, as measured by the indicators that those at the political centre deem the most important. In other words, schools cannot simply be trusted to do the right things by their pupils. They have to be regulated through mechanisms which seek to give them self-interested incentives to do so. Self-interested decisions, on the part of the institution, are a natural consequence of any such structure. Indeed, arguably they are its function. When the measurement mechanisms are badly designed so that schools' interests appear to be dramatically out of kilter with those of their pupils – as is claimed in the report – the response of those at the political centre is always that the mechanisms must be reformed to correct this, to bring pupil interests in line with institutional self-interest. Doing so raises the prospect of benefits, in the short term. But in my view, the system overall has proved itself dysfunctional because of its essentially negative view of human nature: the desire to do the right thing by their pupils, surely present in many professionals, is simply written off, in a regime implicitly based on a belief that self-interest is, in the end, all we can rely on. True reform won't come until someone in power realises the underlying limitations of this approach. Change must recognise that some form of accountability and data production is important, and it cannot involve a move towards absolute and unconditional trust. But it must create a better balance between mistrust and professionalism than we have now. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=433 Tue, 08 Mar 2011 13:56:54 GMT 20110308135654 Tue, 08 Mar 2011 13:56:54 GMT 20110308135654 Tue, 08 Mar 2011 13:58:17 GMT 201103081315817 What are David Cameron's plans for public sector reform, and how could schools be affected? Many will have noticed reports of an article Mr Cameron wrote for the Daily Telegraph (http://bit.ly/eqgH1u) on Monday, in which he put forward proposals for a “revolution” in the public realm, with a much greater role for the private and voluntary sectors in the provision of services. However, I found another piece, which received much less attention but appeared in the Guardian (http://bit.ly/fmfq6m) on the same day, to be at least as interesting. This provided a much more detailed glimpse into what Mr Cameron's forthcoming white paper on reform, expected in the next fortnight, could contain. And its contents were potentially hugely controversial, for many people working in the public sector. The Government wants to introduce a system of true payment by results across public services, the Guardian article said, with boundaries between public, private and third sector organisations “melting away”. Conceding that this agenda would face resistance, it would simply have to be “forced” on the public sector in the early stages, said the report. The Guardian article was based on what it said were the views of Paul Kirby, Number 10's new head of policy development, who it said was “one of the main minds” behind the forthcoming white paper. The rest of the Guardian's story was then based on a document on public sector reform (http://bit.ly/fs69o5) which Mr Kirby co-wrote with two former colleagues at the management consultancy KPMG while he was working there last year. The central argument of the 24-page document is that the public sector needs to be made to resemble much more closely a market, with providers of services given stronger incentives to improve their quality by tying the money they receive directly to the results they achieve, as it says happens in the private and voluntary sectors. In school education, what is known as a “quasi-market” has been in operation since the 1988 Education Reform Act. This has seen funding follow pupils, with schools which attract more through the door receiving more cash, and vice versa. The KPMG paper's arguments and conclusions are entirely generic – they are meant to apply across almost the entire range of public services, from schools and hospitals to social services and prisons – and only supplemented for particular services with anecdotes. So it is hard to tie down the specifics of what it thinks about the effectiveness of the current quasi-market schools system. But its belief seems to be that this is not a sufficiently true market. Therefore, some kind of more commercial set-up is envisaged. The paper implies that providers – including, one imagines, schools – would gain all of their funding “per customer”. In other words, in the case of schools, there would be no core budget funding reflecting fixed costs, and all money, including capital and deprivation allocations, would be sent to each institution on a per-pupil basis. Crucially, this would have to be structured so that poor performers would face a true battle for survival, while the more successful could expand. I have read the document twice now, and I still am not sure whether payment by results, or “payment for success”, as it calls it, means, first, that ministers would specify centrally achievement outcomes that schools would have to reach – the obvious is exam results, although they could be set out more broadly than this – and reward them for achieving these goals. Or it could simply mean that the consumer of the services – parents, choosing for their children – get to decide what matters and thus that “payment by results” means money following parental choices, so that schools attracting more pupils gain a relatively bigger benefit financial than now. If this truly does break with what happens now, it seems to me to be very radical stuff. But I think there are major problems with this document. These are illustrated powerfully, on occasion, by the few anecdotes it does come up with in respect of education. The document implies that the public sector has not changed fundamentally in the past 20 years, because previous governments had failed to push through this idea of a market-based system, instead imposing too many top-down programmes which simply told professionals what to do, rather than creating a structure which would encourage them to find their own ways of improving. Many billions were invested under Labour, but without sufficient reform, it suggests. So the public ended up footing a larger bill for services which essentially stayed the same. Yet the paper fails to mention, in relation to education, the structural factors within the schools system which constrain its ability to change in the way the authors wish. The most obvious is that for a true system of choice to operate in schools, there needs to be a ready supply of spare places within a local area. If not, there will be no room for all parents to get their first choice school, as choice is never evenly spread between schools (indeed, the authors would presumably not want this to be the case, since it would not create the “winning” and “losing” schools which they would argue force up standards through competition). So, in the absence of spare place capacity, parents will be forced into schools they would not have chosen and therefore the payment by results system envisaged, whereby money always follows the child to the school their parent actually wanted – as opposed to the one they had to go to, because there was no space elsewhere - will not work. Yet providing spare places costs money, when the document is predicated on a belief that the public sector needs to become more efficient. There are other problems with the choice argument, too, such as the fact that there are downsides associated with schools which are unpopular with parents simply closing, most notably that this carries a human cost for the pupils they educate. (Unlike, say, a supermarket closing having failed to attract customers). And in my experience, successful schools are often loathe to expand for fear of losing what makes them special. The argument that Labour's investment had the effect of pushing up salaries and thus that it may have made the profession less unattractive to those considering working in public services such as teaching, thus raising quality, and thus that this could be seen as a return on the public's investment, is also not mentioned. Neither is the question of the degree to which the truer market system advocated really would be popular with professionals – and the knock-on effects on recruitment and quality depending on the answer to that question. Detailed anecdotes in this report which come across as unconvincing are a suggestion that public sector professionals need to be “forced” into change because, for example, relatively few schools have opted to go grant-maintained or become academies in the past 20 years. This suggests a belief that this always would have been better for the school concerned and that not to make the move was simple unthinking conservatism, rather than reflecting the truth that rational concerns are at play in decision-making. The report also includes a suggestion that library services no longer need to be run publicly, but could be staffed by volunteers. This is coming across quite a bit of scepticism as the “Big Society” rolls out. The document is also predicated on a belief that public sector “productivity” has not kept pace with investment. But this is in itself based unquestioningly on a concept of productivity which, for at least some aspects of the public sector, is notoriously difficult to quantify. Using exam results to measure the “productivity” of education, for example, has never really worked, because they are not designed for this purpose. The deeper problem with this document is that, while it claims that the system it advocates will liberate public sector professionals to come up with solutions for “customers” which work, and thus that it is enhancing “ground-up” problem-solving, in fact this is an incredibly top-down, centralised model. The paper offers no meaningful detail of some obvious potential pitfalls, within each service – or at least among those I can spot in education - which must surely have to at least be taken seriously before progress can be made towards what it wants. Instead, it simply sets up a monolithic theoretical framework into which all public services must be made to fit, seemingly founded on the belief that not to go along with this simply signals irrational conservatism. Having re-read the Guardian piece, I am unsure to what degree these ideas are going to find expression in the white paper. However, Mr Cameron has certainly signalled his intention to think radical thoughts on the future of public services. If this really is the depth of the coalition's thoughts on this subject, I think they- and we - should be very worried. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=428 Thu, 24 Feb 2011 10:15:41 GMT 20110224101541 Thu, 24 Feb 2011 10:15:41 GMT 20110224101541 Thu, 24 Feb 2011 10:15:41 GMT 2011022410101541 I challenge anyone to follow England's education policy-making process at close quarters for any length of time and not to get profoundly disheartened. Probably the biggest problem is the degree to which important national questions get framed as subordinate to the points-scoring needs of party politicians. The launch last week of another review of the national curriculum would, in a more considered and rational world, be the occasion to open a process of deep reflection about what counts to be educated in the modern world, and about the balance between central direction and local discretion over what should be taught, among other things. To be sure, plenty of vigorous arguments would be expected. But we might also expect people whose views clashed to treat the positions of their opponents with respect. These things may well be going on behind the scenes as part of this new review. But they do tend to get obscured by the predictable Punch-and-Judy fight between “traditionalists” (caricatured as in favour only, it seems, of children endlessly reciting names of countries, cities, rivers and great figures from Britain's history) and “trendies” (written off as believing children need learn no facts at school). It is possible, I think, for governments to try to rise above this fray: to pronounce on sensible areas of, if not common ground over what should be taught, then at least a deeply argued and fair-minded handling of what are important issues. But not this time, it seems, or at least not so far in some of the public pronouncements. In fact, I was dismayed to see the review – which, I believe, will itself attempt to take a considered approach to the subject - launched with a degree of politicisation and on occasion selective approach to evidence which I do not think will help anyone in this debate: not schools, not pupils, not the review team itself and not even the person who unveiled it, Michael Gove. It really was depressing to read, on the day the review was launched, an article in the Daily Telegraph under Mr Gove's name under the derisive headline “National curriculum review: children failed by Labour's education reforms, says Gove.” OK, Mr Gove is unlikely to have written the headline. But underneath, the piece continued: “Labour are guilty of many things – wrecking our economy, piling debt on the young, breaking faith in politics [I don't think Labour bears the sole responsibility here, somehow...] – but their failure to equip our children for an ever more competitive 21st century must count as their greatest crime.” His speech to an audience at Twyford Church of England high school, in Ealing, west London, to launch the review, was slightly less partisan. But it also quoted selectively from international evidence: the UK has indeed been slipping down the international rankings in one major testing study – though fresh questions were asked about the validity of that argument last week (http://bit.ly/eVo3lA ) – but England's performance is far more impressive in another international study, which was mentioned in the speech without reference to this country's results. (See my 2008 TES story on this study: http://bit.ly/eWtsDo). Mr Gove might also be guilty of misrepresentation. In his Twyford speech, he said Labour's curriculum has become “bloated with prescriptive detail about how to teach”. This may be a reference to claims, in particular in a paper by Tim Oates of Cambridge Assessment*, who is leading the review's expert panel, that the current secondary curriculum wrongly promotes a particular type of teaching (cross-subject). Mr Gove's language, however, is needlessly polarising, in my view, and it is not clear that the word “bloated” applies to the curriculum in this sense at all. Consider, now, the following quotations. In 2005, Labour published a white paper on schools reform which promised to “review the Key Stage 3 curriculum, to improve its coherence in subjects where there are problems, to review the overall level of prescription and allow more scope to schools to stretch their pupils and to help those who fall behind expected standards to catch up”. In 2008, Ed Balls, schools secretary, wrote a remit letter for the review of the primary curriculum to be led by Sir Jim Rose which said: “While most primary schools are providing their pupils with an inspiring and engaging curriculum, some tell us that the number of subjects and the amount of prescription in some of the current programmes of study restrict their flexibility.” Last week, Michael Gove's curriculum review said: “It is the Government's intention that the national curriculum be slimmed down so that it properly reflects the body of essential knowledge which all children should learn and does not absorb the overwhelming majority of teaching time in schools.” There are, to be sure, differences of approach behind the two parties' version of the nature and level of control over what teachers should teach. But the dominant criticism around the curriculum in recent years - that it is overloaded – has actually been shared by both parties. In other words, the common view has been that there is too much prescription of what must be taught. At the same time, there seems widespread agreement that there should be some prescription, or that there should be a national curriculum of some sort. That, at least, was my conclusion having read through papers submitted to an inquiry on this issue by the Children Schools and Families select committee in 2008, where virtually everyone welcomed the principle of some kind of national curriculum. The difference, then, comes down to how and where to strip back content. I turn now specifically to the secondary curriculum, for this is a debate I have reported on for several years now. The 2007 review by Mick Waters, at the time the director of curriculum at the Qualifications and Curriculum Authority, responded to the call for greater flexibility for teachers to develop their own approaches to teaching by reducing content dramatically. For example, the content of the Key Stage 3 science curriculum was reduced from 94 bullet points of required scientific knowledge in the 1999 curriculum to 14 points in 2007. In KS3 maths, the corresponding reduction was from 134 to 58 bullet points. It is the contention of some, including Mr Oates, that the reduction in prescribed content occurred in the wrong way. The fact that photosynthesis, for example, is not now mentioned in the secondary curriculum, Mr Oates argues, runs the risk that it will not be taught at all to some pupils. Thus, the original thinking behind the curriculum – that it guarantees central topics are taught to children everywhere – has been subverted, runs the argument. Mr Gove made what - for those who back Mr Oates's carefully-argued view - will have been powerful points in his speech at Twyford school when he highlighted mathematical and scientific concepts which are specified in the curricula of high-performing Hong Kong and Singapore but not in England. I think he was on far less solid ground, however, with headline-chasing comments on history and geography suggesting that children were losing out because the names of historical figures, or other countries than the UK, were not on the respective history and geography curricula. The notion that World War II could be taught without mentioning wartime leaders such as Churchill is absurd, while if Mr Gove truly believes that geography should focus much more on familiarity with the names of different countries, or history should focus on historical biography, he should say so. Having looked at the 2007 secondary curriculum again, I believe this review is looking to increase central prescription of what should be taught, at least in the subjects that it deems to be “core”. But the bigger point is that both points of view on the secondary curriculum – that the flexibility created by the 2007 review has given more space for teachers to develop their own approaches, and, alternatively, that this carries major risks – are worthy of respect, rather than ridicule. If many teachers favour the former view, then these teachers are those Mr Gove is going to have to work with. So being quite so political and confrontational looks counter-productive to me, unless the main purpose is to make party-political points. This is especially the case given that the Education Secretary himself appears to be making the case for curriculum flexibility, in allowing academies and free schools not to be covered by the “national” curriculum which results from this review. Confusingly, they will be able to use the curriculum which results only as a “benchmark”, against which to compare curricula of their own. If, as Mr Gove intends, all schools will be academies/free schools in the long run, it is difficult to see how, in that case, we could have a national curriculum at all. It is to Mr Gove's credit that he has – perhaps unusually for a major government inquiry these days, and certainly conspicuously missing under Labour – managed to assemble a team of genuinely expert academics as advisers, with Mr Oates leading a group also comprising Mary James, Andrew Pollard and Dylan Wiliam. That team deserves better than the review's politicised opening. It is strange, too, that their expertise is not being complemented by having some current teachers on the wider review. There is one final point. With the abolition of the Qualifications and Curriculum Development Agency, curriculum development is now entirely “in-house” at the Department for Education, under Mr Gove's overall leadership. Because there is now no longer any official, semi-independent, national voice on the curriculum, what is surely needed is someone to at least try to put the national interest – remember that phrase, used a lot after the coalition was formed? – above narrow party-political concerns. While I would expect those I know within the review team to do that, the minister overseeing their work has a long way to go. *Tim Oates's paper is here: http://bit.ly/ajbCp2? http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=422 Wed, 26 Jan 2011 09:35:08 GMT 20110126093508 Wed, 26 Jan 2011 09:35:08 GMT 20110126093508 Tue, 01 Feb 2011 11:30:53 GMT 2011020111113053 Changes in league table performance measures seem only to underscore the huge impact the rankings have on what goes on in schools. This week, with the introduction of the “English Baccalaureate”, has been no exception. In researching an article on the subject for Education Guardian, (http://bit.ly/hx7Kqk), I found anecdotal and unsurprising evidence that some schools are going to react to the new measure, which ranks them on the proportion of pupils achieving GCSE A*-C grades in English, maths, two sciences, a language and history, ancient history or geography, by making curricular changes. A member of the management team at one school told me that it was considering “forcing” all pupils to do either history or geography. The head of another said the “top half” of next year's year 10 would all be taking English Baccalaureate subjects. It was widely claimed that many schools would be pursuing curricular changes. Not all schools will react in this way, while others under pressure to improve above central floor targets for performance on the old/current five GCSEs including English and maths measure may well choose to focus on these. But it seems likely that many will change curricular provision because of the English Bacc. Beyond that observation, the controversy has illustrated the perhaps-not-unconnected point that league table mechanisms, controlled from the political centre, are absolutely pivotal to the way policy is now enacted in England. That makes us very different from most, if not all, other countries of which I am aware, and also helps to explain the degree of complexity and contradiction surrounding this week's events. To explain, many other countries do indeed push children towards certain academic subjects in secondary school. This was acknowledged in November's white paper, which said that “in most European countries, school students are expected to pursue a broad and rounded range of academic subjects until the age of 16.” The difference, though, with England, is that most countries or states tend simply to specify, centrally, what subjects children should study and where they have options. None as far as I am aware uses league tables to – to use a fashionable term – “nudge” schools in a particular direction favoured by government, as happens here. A recent Nuffield Foundation-funded study (http://bit.ly/fSNxIJ; see table 3)on pupils' participation in maths pointed out that many countries require students on both academic and vocational routes to continue with study in subjects usually including maths, a second language and a science until 17 or 18. Ministers here, who have been particularly concerned about the falls in the number of pupils taking languages since Labour made the subjects optional in 2004, could – logically if not without facing practical difficulties - have responded by simply making them compulsory again. Similar arguments could be made for geography and history. I am not seeking to advocate this approach here, but simply pointing out the difference. Alternatively, they could have had a completely “free market” approach, leaving pupils to choose exams to take post-14 and returned to a previous policy of opening up data for all schools in all subjects, leaving parents to decide which subjects were important to their children. What is particularly English about the latest move is that ministers' main mechanism for reform is a league table system which paradoxically both suggests parents are free to decide which subjects are important to them but also involves a centralised view of what count as important subjects. The new indicator, says the white paper, will “provide a powerful incentive for schools to drive the take-up of individual science subjects, humanities such as history and, especially, foreign languages”. So, we have neither compulsion – which presumably would have needed a full curriculum review, and therefore, time – nor complete freedom, but a half-way house. In a response to me last week, the Department for Education said it did not want compulsion because the “full range of EBacc will not be suitable for all pupils”. But this compromise does create questions. The first set surrounds the contradictions, as has been widely pointed-out: the notion of freedom of school choice versus centralised stipulation, without consultation, of what a “broad and rounded academic curriculum” means; how this fits with Government moves to set up technical schools for 14- to 19-year-olds where non-academic subjects will be the focus; and how to reconcile ministerial claims that academies improve pupils' results faster than other schools with the news that nearly a third of open academies with results to publish have zero per cent of pupils achieving the Government's favoured measure. It also should be remembered that the reform has created a performance indicator which is not yet a qualification*, and a baccalaureate which is at odds with how the word has traditionally been understood, in that it usually comes at the end of school, at 18, and involves combining subjects at more than one level. There is another question. Do ministers believe schools should force pupils to take the subjects of the English Bacc, or not? Questioned fiercely on this subject by a parent on Radio 5 Live on Wednesday, Michael Gove, the Education Secretary, said no school would be forced to offer the English Baccalaureate. But the white paper (see the quotation above) suggests he clearly wants to use the league table mechanism not just to provide information to parents, but to push schools to push pupils towards these subjects. To analyse that incentive, it means that schools are now being given self-interested reasons to direct pupils towards particular subjects, through the need to look good on the published indicators. In a case where a child is believed capable of doing well in an English Baccalaureate subject, but prefers not to take that subject, the incentive on the school is to overrule them in order to do well in the rankings. The school may not choose to follow that incentive, but it is there, and it has been created by the Government. Some have argued that it is better for schools to be incentivised into pushing pupils towards the English Baccalaureate subjects than towards vocational options, which have carried far higher weight in the government performance formulae in the past than employers and universities give them in the “real world”. Hands up, I have been reporting on the second part of that sentence for years now, and have argued that the heavy weighting given to non-GCSE courses can act against pupils' interests, because of the pressures on schools to look good in published results. But it is revealing that the answer to this question is not seen by ministers to be a careful, detailed and thorough consultation on what the country as a whole should be encouraging in its young people, or even what counts to be educated today. Nor is there felt the need to take a truly sceptical and open-minded view of the overall impact of league tables as a whole. Instead, the move to react by simply changing the preferred centralised indicator, to make it more in line with what the Government believes parents want, shows how hooked the policy-makers are on our league table system. A particular type of league table system – for the character of the information put out by the Department for Education was very similar to that under Labour, with only preferred indicators changing - seems to be the immovable object of schools reform, around which policy must be built. However, the changing of league table indicators will not eradicate injustices – identified in the white paper - of pupils being pushed towards subjects which might not be in their best long-term interests, but simply serve their schools' needs. It will simply change the nature of those subjects, towards ones which the new Government favours. One suspects policy-makers love league tables in part because change can be introduced quickly, cheaply, according to a centralised design without the need for the extensive consultation which would at least seem to be de rigeur for curriculum inquiries and with ministers able to claim that they are not “forcing” anyone to do anything. For a very different approach, I would recommend the following document. “School Education in France”, (http://bit.ly/fizxyz) published in English by the French education ministry, sets out admirably clearly the subjects and choice offered to all French pupils, on principles which it says are linked to the French constitution. There is no mention of league tables, although they do exist there, in a minor way. But then, of course, England is not France. *I may be being overly sceptical here, but I do wonder whether this qualification will be awarded to individual pupils, and if so, how. In 2005, Labour published a white paper (http://bit.ly/gw1GgR) which introduced the five or more GCSE A*-Cs including English and maths into league tables. It also proposed to recognise this for individual students, saying: “We will expect more teenagers to achieve 5 A*-C grade GCSEs including English and maths and we will introduce a general (GCSE) Diploma to recognise those who achieve this standard. It never happened. When I asked the Department for Education last week who would be producing certificates in the English Baccalaureate, given that it will presumably be achieved by pupils taking exams through different boards, it said: “We are currently developing options for issuing such certificates. We will ensure that the EBacc recognises achievement in GCSEs from more than one awarding organisation.” http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=419 Thu, 13 Jan 2011 16:36:41 GMT 20110113163641 Thu, 13 Jan 2011 16:36:41 GMT 20110113163641 Thu, 13 Jan 2011 16:36:41 GMT 201101131643641 I woke up this morning to Radio 4's Today programme announcing ominously that “thousands of boys start secondary school only able to reach the reading standards of seven-year-olds or below”. Leaving aside my initial irritation that the programme claimed national figures showing nine per cent of boys fell into this category were being revealed for the first time – they were not, since these data are published annually – I started to wonder about what it actually meant to have the reading age of a seven-year-old. Or rather, I knew what it meant, and wondered if the test data on which the lead story on the nation's leading political broadcast were being hung could really support the interpretation being put on them. So let us deconstruct what it means to be, as Today said, a boy who is “only able to reach the reading standards of seven-year-olds or below”. I want to look, first, at what that means in terms of national curriculum levels. And, in this sense, the story reminded me of concerns, suggested a few weeks ago in a paper on the national curriculum by Tim Oates of Cambridge Assessment, about the levels system as a whole as a way of reaching judgements on children. Anyway, in level terms that statement – a boy reaching “reading standards of seven-year-olds or below” - translates as a boy who scores below level 3 in key stage 2 reading tests. Now we need a bit of history. In 1987, when the Task Group on Assessment and Testing (TGAT) produced a report for Kenneth Baker as education secretary, it set out the “levels” system which produced certain proficiencies expected of the average child at certain ages. The average child at seven would achieve at level two, said the report, while his or her counterpart at the age of 11 would be at level four. This system basically still applies. So the Today programme was technically completely correct to base its story on the fact that a boy achieving level 2 or below in the reading test is reading at or below the proficiency expected of a seven-year-old. However, if you actually look at how levels are calculated, you begin to have questions. Whether a child reached the expected level of a seven-year-old, or of an 11-year-old, came down this year to eight marks on one 50-mark, 45-minute, national curriculum test. This summer, a child who scored 10 marks out of 50 would have gained an “N” mark, failing to gain a level 3. As level 2 is the level below 3, and level 2 is the expected level for a seven-year-old, any child coming into that category could be said, fairly by a journalist taking these figures at face value, to have the reading age of a seven-year-old. Between 11 and 17 marks out of 50 would gain the child a level 3. At 18 marks, they would achieve a level four, and so could be described to have the reading age of an 11-year-old. At 31 marks, they would have a level five, which is officially defined as the reading age of a 14-year-old. (To counterbalance the story, what was not reported was that 45 per cent of children achieved level five in tests this year, and thus technically were at the level expected of a 14-year-old). In other words, the difference between a child being said to have the reading age of a seven-year-old, and that of a 14-year-old, can come down to just 21 marks on a 50-mark 45 minute test. That's the difference between 20 per cent and 62 per cent. Consider again the doom-laden tones of the Today programme, which would seem to imply that thousands of children are really years behind their classmates in learning, effectively only having reached the level they should have been at four years, or more, ago. Years at school have simply been wasted when it comes to reading, is the implication. But there is another reading of these numbers. If only, to take these test figures literally, these boys had all managed to get eight more marks on these tests. Then they would all be at level four, and all reading at the level expected of their age. Instead, they have fallen behind. If taken completely literally, therefore, the entire progress expected of a child, if these tests are any guide, between the ages of seven and 11 comes down to two marks a year on a 50-mark test. This measurement system is, on that reading, ridiculous. I am not trying to minimise this problem. Clearly, if boys are not able to read properly when they reach secondary school, this is a serious issue. It is of concern that the Radio 4 reporter said she had spoken to secondary heads and that they were concerned about the literacy levels of boys reaching to them, which meant they were struggling to access the curriculum. It may well be, then, that another measurement system would reach similar conclusions. It's just that I'm afraid I'm not sure how much light the current one, and these level threshold scores in particular, truly shed on this problem. There is another angle here. Consider, again, how these figures were being interpreted on the radio. John Humphrys, the presenter, talked about “illiteracy”. The lead headline on the programme's website now reads “11-year-old illiteracy 'unacceptable'”. And, if around one in 10 boys are indeed “illiterate” when they leave primary school, this is truly a huge national problem. But are they? And do these data tell us that they are? Well, I don't think so. Strictly, they tell us that these boys cannot score – or were not entered for the test because their teacher did not believe they would be able to score – 11 marks on a 50 mark reading comprehension test. That is, certainly, a cause for concern. But is these boys' problem a) illiteracy (not being able to read at all), b) a fundamental weakness in reading comprehension, or c) a lack of confidence in test-taking? Despite what Mr Humphrys said, it would seem fairly clear that the problem is not a), at least for the vast majority of children, since the fact that almost all can score some marks shows that they can read – ie decode words on a page - to some degree. But if this is not the fundamental problem for many children, why is the focus, now, on introducing a simple decoding or phonics test for all six-year-olds, rather than focusing on reading for understanding? Of course, the most likely answer is that it is mainly b), but that c) cannot be discounted in at least a minority of these cases. But this lack of clarity about what the test data actually mean is very serious. Of course the fact that many boys struggle with reading – which is a phenomenon in education in many countries – is a big problem. Unfortunately, these data don't give us nearly enough information about what the problem actually is, or indeed what to do about it. In terms of alternatives to the current testing regime, the easy answer is simply to say teacher assessment, but I am tempted to add to that by saying that any assessment system has to look at pupils' reading ability and progress over a long period of time, rather than in a one-off test. We want to know in detail how or whether a child's facility with texts improves, and where the gaps are. And, though I am not a teacher, I think there must be merits to making use of technology. Adaptive computerised assessments which really try to diagnose what skills a child has mastered over a period of weeks, or months, or years, and where there are gaps, surely must be the way forward at some stage, if only to take some of the administrative assessment load off teachers, and of course these tests are used in many classrooms. As it is, though, the dominant 20-year-old testing and levels model which equates, implicitly, pupils' key ambition in primary reading as picking up a few marks here or there in a one-off test, is beginning to look very dated indeed. This, then, seems to be a serious problem, so please let us take it seriously. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=412 Fri, 17 Dec 2010 15:20:00 GMT 20101217152000 Fri, 17 Dec 2010 15:20:00 GMT 20101217152000 Fri, 17 Dec 2010 16:55:28 GMT 201012171645528 Much has been written already about the UK's disappointing set of results in the latest Organisation for Economic Co-operation and Development PISA* tests, which assess 15-year-olds around the world in reading, maths and science. But for me, just one set of hugely contrasting figures gets to the heart of the debate about the extent to which education policy in this country has succeeded in the last 15 years, and the nature of that success. Consider the following statistics. And I apologise if what follows is somewhat number-heavy; it is more or less unavoidable, in this context. In 2001, 50 per cent of young people (mainly 16-year-olds) in England achieved five or more A*-C grades at GCSE. By 2010, that figure had been transformed, rising by a half to a new high of 75 per cent . Now compare the test results of those same two cohorts of pupils in the PISA tests. In 2000, a more-or-less representative sample across the UK of that same 2001 GCSE cohort scored, in PISA, 523 points in reading, 529 in maths and 532 in science, where 500 points represented average performance across all OECD countries. Now, in 2009, a sample of the UK pupils who would go on and take GCSEs in 2010 scored 494 for reading, 492 for maths and 514 for science. In other words, while results at GCSE soared, those in PISA have gone backwards. The same picture is discernible when PISA results are compared to key stage 2 test scores. The 2000 PISA cohort took KS2 tests in 1996, before domestic test scores began a rapid ascent in the late 1990s. In 1996, the proportion of pupils achieving the “expected” level in the three tested subjects ranged from 54 per cent in maths to 62 per cent in science. By the 2000s, scores had risen such that, in 2005, when the 2009 PISA cohort sat their year six tests, the corresponding figures were more than 20 percentage points higher, varying from 75 per cent in maths to 86 per cent in science. The key question from PISA, then, is why this improved performance in our domestic exams has not been reflected in the PISA assessments. Regular readers of this blog should need no reminding of the dangers of over-interpreting test statistics, so there are some caveats to make before going further. First, the 2000 PISA results are classed as “unofficial”, since the UK did not technically provide sufficient pupils that year to meet the OECD's sampling requirements. However, being slightly charitable and cautious with the figures because of this, suffice it to say that when you compare the unofficial results of the 2000 and 2003 PISA tests with the PISA scores from 2006 and 2009, there is no evidence of the improvements registered particularly through the GCSE scores. And the England's PISA results are static between 2006 and 2009, while GCSE scores climbed. As a second caveat, PISA is not the only set of international tests available. The rival Trends in International Mathematics and Science Study (TIMSS), which operates every four years, does present a very different picture, showing test scores have risen strongly in primary maths in particular in England since the mid-1990s, with England's performance in the four tested subjects of maths and science in primary and secondary consistently now well above the international average. So, as ever with data, different verdicts can be reached depending on which statistics are used. However, the PISA scores do lend further support to the hypothesis that, while England's education system has become very accomplished, in recent years, at targeting pupil performance at particular indicators of success – domestic test and exam measures – if you change the test, or the measure, the picture can change dramatically, even to the extent that “progress” evaporates. This is vital, of course, since pupils who have been taught to do well in the context of particular exams, but are then unable to transfer that to contexts away from that particular set of tests arguably will have received little deeper benefit from the results drive, in terms of understanding. England's accountability system, then, in forcing schools to focus so overwhelmingly on domestic test and exam success, has not been successful at all in raising standards as measured by the PISA tests, which do stress the need for pupils to apply their understanding. Has the narrow focus of accountability – implicitly prized as useful in achieving measurable results at least by policy-makers in the New Labour era - got in the way of teaching for more general understanding which can be picked up through other tests? There is no doubt, too, that the PISA figures cast doubt on Labour's literacy and numeracy strategies, introduced in 1998. If these strategies had truly raised pupils' long-term understanding, progress ought to be seen in all international testing studies, for both primary and secondary pupils, not just in some of them. I would say that, having carried out some research on a number of countries' education systems in recent weeks, England's stands out for a few reasons, including the bewildering number of types of schools; the large and sometimes competing number of reforms made to its system; the richness of the data the government and schools now hold on pupil performance; and, to a large extent although England is not completely alone in this position, the punishment meted out to schools through its accountability system. If those characteristics set it apart among countries ranked by PISA, its results appear not to. This should give policy-makers pause for thought, particularly when they talk about accelerating some of these reforms. If all this sounds overly pessimistic, I should perhaps end with another cautionary note. The UK is not the only country to fail to see a huge advance in its test scores, as measured by PISA, in recent years, although its scores' decline has been pronounced. In fact, of the 20 top-performing nations on PISA's reading tests in 2000, only two had improved their scores, on average, by 2009, with the rest either declining or staying roughly the same. In maths, only four of the top 20 performers in 2000 have raised their scores, while in science, only seven have. In fact, across the OECD, test scores are slightly down as a whole on average in each of the three tested subjects, although the fall is technically too small for this to indicate standards doing anything other than staying broadly unchanged. 2009 scores are directly comparable with the 2000 figures, partly because some test questions are retained between rounds of the tests, so the general lack of progress seems statistically robust. This is despite the OECD having presented figures this week suggesting that many countries have invested heavily in education over the period. Perhaps all schools are struggling against factors in children's lives which can militate against learning; maybe children have so many more potential distractions away from education these days that registering progress in tests is tougher. In any case, raising pupil performance, as any teacher must know, is not an easy business. *Programme for International Student Assessment http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=409 Thu, 09 Dec 2010 11:25:35 GMT 20101209112535 Thu, 09 Dec 2010 11:25:35 GMT 20101209112535 Thu, 09 Dec 2010 11:26:30 GMT 2010120911112630 Where to start with the white paper? The 95-page document heralds such a blizzard of further reform that it is almost difficult to know where to begin. But perhaps it is best to highlight, very selectively, a few detailed aspects, concentrating mainly on topics with which this blog has been concerned, before pulling back for some more general thoughts. In no particular order, then, here is what the white paper says on: -The national curriculum. This is going to be slimmed down drastically, with the paper talking about its reduction to a “core”, around which teachers have freedom to choose what and how to teach. But I think the white paper leaves it unclear whether ministers envisage a reduction just to today's core subjects: English, maths, science and perhaps ICT (although technology hardly gets a mention in the paper), or whether it will extend to others. Margaret Thatcher, who wanted a “basics” curriculum, and Kenneth Baker, who wanted a broader entitlement, famously fought a battle over this in the late 1980s and we may be about to see it rejoined. The enthusiasm of Michael Gove, the education secretary, for history, and high-profile involvement by celebrity historians, makes it hard to see how this subject would not be included, however. There also seems to me to be a lack of clarity about the level of curricular prescription expected in academies, which will have greater (complete?) curricular freedoms. More details will follow in a curriculum review which is about to get under way, with the new curriculum in schools by 2013. -Testing. A new test of reading, emphasising phonics, will be introduced for six-year-olds. The agency which is to replace the Qualifications and Curriculum Development Agency has also been charged with developing tests for 14-year-olds, in unspecified subjects, which schools can use if they want to. While the key stage 2 tests are under review, the fate of early years and key stage 1 assessment other than the reading tests is not specified, the paper saying only that “we will look closely at the assessment and reporting burdens in the early years and at key stage one”. -League tables and secondary exams. There are some changes here that I would welcome, including the acknowledgement in a few places in the paper of some of the side-effects of current accountability, including “excessive test preparation”. The paper proposes measuring school performance according to a new “English baccalaureate” GCSE measure, based on the proportion of pupils gaining C grades or better in English, maths, a science, a language and a humanities subject. However, I wonder how much effect this will have on schools at the bottom of the league tables, where the main accountability mechanism driving intervention in school management structures(see below) will remain the five-plus A*-C including English and maths measure. It may be that higher achieving schools will be able to focus on subjects such as languages and history, with those at the bottom of the rankings concentrating on the “basics”. It also appears that vocational exams are going to lose their “equivalence” with GCSEs and that ministers will try to reduce the number of pupils taking GCSEs and A-levels through modules, rather than end-of-course assessment. The detail suggests this may not be as straightforward as some media reports have indicated, however. Contextual value added measures are to be scrapped, with schools being monitored on “raw scores”, the progress pupils make and the achievements of particular groups. New data requirements are likely to see the public able to get information on pupil performance in individual (secondary) subjects, as well as spending information. It remains unclear whether the new reading test is to add a new league table, as on page 11, the paper lists the test in a paragraph about providing information to parents on their children's progress and “the effectiveness of schools”. -Inspections. Schools previously adjudged “outstanding” will no longer be inspected unless there are indications, including from results data, that standards have slipped. They will, though, be able to request an inspection from Ofsted from 2011, though they will have to pay. Any likely takers among readers? The paper confirms that schools will no longer have to complete a self-evaluation form, and says “neither the Government nor Ofsted [will] require written lesson plans, let alone in a particular format”. -On funding, heads appear to be being given more latitude, with fewer ring-fenced programmes and also no requirement for local authorities to have plans to “claw back” reserves schools have in their bank balances. The paper wants more flexibility in how teachers' pay and conditions are handled. It looks as if school sixth form funding will be cut to the level of that in colleges. Right, that is a lot of detail, and there is space now for only a couple of more general points. First, among several contradictions within the white paper, the biggest is the tension between its repeatedly stated – and no doubt very welcome to many- commitment to handing more power to professionals and local communities, and elements of top-down control which in some places go beyond the experience under Labour. The paper says: “We believe that public services will improve most when professionals feel free to do what they believe is right, and are accountable for the results.” This is classic ideology from New Public Management, the worldwide movement, dating from the 1980s, which is designed to make the public sector operate more like the private, with a diminishing apparent role for central government but with front-line workers then held to account according to their achievement of performance indicators. But there is a tension in the statement above, as true freedom will be clearly limited, at least for some schools, so long as the accountability mechanism is clearly controlled by the centre, with strict penalties for non-achievers. The white paper includes these sanctions, including radical restructuring plans for schools which are found to be underperforming. These culminate, amazingly and shockingly I think, in the ability of the Secretary of State to simply order a school to become an academy, seemingly without any reference to what its staff or local community thinks about this plan. Academies are by definition a good thing, is the implication of the paper. It also says that school sixth form provision which falls below a certain standard may no longer be funded. My final thought is on the scale of this reform plan. The paper makes much of the need for further change to England's education system. The Government argues in a companion document with detailed evidence – although its highly spun use of evidence itself is in several places disturbing – that other countries have reformed their systems and thus that further change is vital. However, in recent investigations into change in 10 different education systems, I have not come across another where national politicians seem to feel the need to put forward plans which attempt to reform, largely unchecked, seemingly all aspects of education, virtually constantly, year after year. This politically-minded tinkering is now reflected in the number of types of schools we have: the paper makes much of new teaching schools, university technical colleges, academies and free schools, to go alongside the city technology colleges, community schools, voluntary-aided and voluntary controlled schools, specialist schools, special schools, trust schools, grammar schools.... I wonder what visitors from other countries make of this complexity. The politicians say that we have no choice but to change things, since education is so important. But, ultimately, if change becomes an end in itself, it produces a kind of insanity. Either that, or the need for “reform” is actually code for greater privatisation, which has been promoted by politicians of both sides. I end up thinking of a football analogy: English education is like a mid- to higher-level Premiership team (for this may be our true position in international rankings), where the board feels the need constantly to at least present itself as overhauling everything about the club, from the playing staff to the colour of the seats, year after year, in search of success. Is this permanent revolution likely to make for greater progress, or make it more difficult? Eventually someone, somewhere, is going to have to see that slowing down a bit – or at least proceeding a bit more deliberately - has its merits. The white paper is here: http://bit.ly/fZzwwx. The companion document is here: http://bit.ly/hTZeNp. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=403 Thu, 25 Nov 2010 15:20:16 GMT 20101125152016 Thu, 25 Nov 2010 15:20:16 GMT 20101125152016 Thu, 25 Nov 2010 15:20:48 GMT 201011251532048 What are the coalition's plans for the new national curriculum heading English schools' way over the next three years? Anyone looking for clues should study in detail the paper just published by Cambridge Assessment, the examining body. The paper, “Could do better: using international comparisons to refine the National Curriculum in England”, drew headlines when it was released on Thursday for its criticism of what were said to be constant changes under Labour which had promoted a “tick list” approach to what gets taught. But what does not come across in the news reports I've seen, I think, is that this study represents the sketching out of a possible alternative curriculum model which will carry a lot of weight in government circles over the coming months. Michael Gove, the education secretary, wrote the forward to the report and Tim Oates, its author who is head of research at Cambridge Assessment and once held that post at the Qualifications and Curriculum Authority, is going to be a central figure in the forthcoming curriculum review for the government. Fundamental to the paper is a view on recent attempts to tackle a particular criticism. This has probably been the one most frequently made of the curriculum throughout its 21-year history: it is simply overloaded, with teachers asked to cover too much material. The most recent curricular change to take effect in English schools was the revised secondary curriculum, completed in 2007, which did indeed strip down content dramatically in response to this widely-alleged problem. For example, the number of bullet points setting out the scientific knowledge expected of pupils at key stage 3 was cut from 94 to 14, while in maths it was reduced from 134 to 58. Other subjects followed a similar pattern, with the argument being that teachers needed more freedom to build understanding; to help pupils make links between subjects; and to allow more time for pupils who needed it to be taught the “basics” of English and maths. Mr Oates has said in the past that reduction of content was indeed needed. But the paper argues that what has replaced detailed specification has been too “vague”, meaning that there can be no guarantee that pupils anywhere in the country are being taught the same essential concepts. This idea of entitlement – that any child should be entitled to be taught nationally-agreed central aspects of what is to be taught - was the central argument behind the introduction of the original curriculum in 1989. As an example of what it argues is wrong with the current arrangements, the paper cites a change to the KS3 curriculum in chemistry which now, it says, states that pupils must understand “that there are patterns in the reactions between substances”. “This statement essentially describes all of chemistry. So what should teachers actually teach? What are the key concepts which children should know and apply? The concept of entitlement becomes seriously eroded”, it argues, because the current curriculum is not specific enough about what is important. The paper implies, then, that in future content be specified in detail, but also suggests that the number of key concepts be kept manageable, so that teaching can build around them in depth. It also, provocatively, suggests criticism of the notion that governments should create curricula which will motivate children to learn. Content – such as ratio or photosynthesis – it suggests, should not be seen as “motivating” or “demotivating” –but just topics to be understood. The job of teachers should be to make the teaching motivating. The paper also says ministers have in the past been too keen to appease lobby groups who want their own particular interest put on the curriculum; that topics likely to be of current but passing interest should be kept off the curriculum (one wonders if Mr Oates has in mind a draft of the Rose primary curriculum referring – albeit in very small print - to the social networking site twitter here) because they just necessitate too-frequent updates; and frequently endorses concerns about assessment dominating the curriculum through teaching to the test. Arguably the most radical stuff is on the use of national curriculum levels, which were introduced alongside the original curriculum. Mr Oates is scathing, arguing that there is no reason that the same eight-level format be used across different subjects, and that the levels themselves when achieved in national curriculum tests offer no guarantee that a child has mastered key concepts. The logic of the report is that levels should be scrapped altogether. The report argues throughout that “high performing” education systems do it differently, laying down core concepts in detail and then expecting teachers to build teaching designed to promote deep understanding around them. A white paper expected in the coming days will set out plans for a review of the curriculum, covering primary and secondary education (ages five to 16). It must report in time for a new curriculum to be with schools by 2013. There is much in Mr Oates's paper that will be controversial, of course. First, expect lots of argument about the detailed content expected within particular subjects. Second, the emphasis on the importance of individual subjects is itself contentious. Third, this is, of course, yet more change, although Mr Oates argues that reducing a curriculum to core, to some extent unchanging, concepts will reduce the need for future revisions for long stretches. And finally, of course, how the proposed curriculum interacts with assessment – which is, far from ideally, perhaps, subject to a separate review – will be critical. However, this paper represents, I think, a likely starting point in the eyes of ministers as the coalition embarks on an aspect of its education reform plans which is likely to be as fiercely contested as any other. It is therefore worthy of serious attention. Anyone with thoughts on any of this, please either comment below or email me at warwickmansell@gmail.com Mr Oates's paper can be viewed here: http://bit.ly/cqt0ud? http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=400 Fri, 19 Nov 2010 15:43:45 GMT 20101119154345 Fri, 19 Nov 2010 15:43:45 GMT 20101119154345 Fri, 19 Nov 2010 15:44:28 GMT 201011191534428 Anyone looking for a landmark day in the history of England's system for holding public services to account might consider what was announced this Monday, November 8th, and think “here's a candidate”. For the announcement (details here: http://bit.ly/cC9QWv) of departmental “business plans” by the Government marks a break with Labour's system of trying to exert control over what goes on in schools and other public services. Whether the numerous changes being put forward by the coalition, when considered in their entirety, will represent improvement on what went before is another question, with some aspects of the announcement already provoking grave disquiet. The headline significance of Monday's announcement is that it would seem to signal the death of the targets regime, the system by which each department gave itself a set of numerical indicators to achieve, and then sought to pressurise public services into doing so. There are no statistical targets in the Department for Education's business plan. It says: “We will...not use top-down targets to drive performance.” That looks to me as though it should – if ministers are true to what they say here - spell the end of the department putting pressure on local authorities, for example, to raise test results to achieve targets and, by extension, of schools being set improvement targets either by their local authorities or by central government. Results pressures on schools, though, seem unlikely to go away. They will continue, I think, in at least three forms. First, Ofsted inspections will remain, and there have been no signs as yet of any move away from the current pre-occupation with holding schools to account at inspection for pupils' test and exam performance, particularly in English and maths. This, of course, is subject to the current key stage 2 assessment review in terms of the type of indicators used. Second, the philosophy of the new government is that parents should be able to use even more data than is currently available to hold schools to account. The business plan talked about a new school league table indicator setting out pupils' “readiness to progress” to the next phase of education at age five and 11. This looks as if it does not require a new set of tests, as the plan says the first set of these indicators were actually published, on a national basis, in August, seemingly based on the proportion of pupils achieving level four in English and maths Sats. There will be, though, trials of a new reading test for six-year-olds. Other indicators to be introduced include ones setting out the progress of pupils with particular characteristics – such as those eligible for free school meals, or children with special needs – in each school and reportedly and controversially, the qualifications that teachers hold, what they earn, how many are full- and part-time and teacher absentee rates. The thinking behind this is that more “transparency” about schools tends to drive up improvement, as parents get better information through which to make choices about where to educate their child. But changes to what is published, and what is not, inevitably risk unintended consequences, not least of which is adding to bureaucracy for schools. The NAHT said this week that publishing teacher salaries, to take one example, also risked inflating pay bills. Third, direct government intervention in school management – which one would have thought would be anathema to an administration which has talked about new freedoms for professionals – remains within the system and is, in relation to some schools at least, even being stepped up. In a speech last week (details here: http://bit.ly/9jyKnw) , Michael Gove, education secretary, said he would be prepared to use new powers to force a school to become an academy, in cases of underperformance. There was no reference to anyone other than the secretary of state making such a decision. In some cases, Mr Gove said, a school which was adjudged “satisfactory” by Ofsted could be eligible for this intervention, if agreement could not be reached with the local authority on the best way forward. He set out three other criteria for such a move, including low attainment. It looks as if secondary schools at least are also going to be held to account through a system of “floor standards”, or minimum results requirements. Replace the word “standard” with “target” and you are, in reality, not far away from Labour's target-setting system, in this aspect of school performance at least. Does this all add up to a coherent plan for promoting genuine school improvement? Ducking the question a bit, we are going to have to wait for more details in the schools white paper due from the end of this month. The plan itself also mentions the introduction of a new national curriculum, which Mr Gove has said will be “simpler, more focused” to allow schools more freedom over what they teach, for first teaching in 2013. Anyone trying to look at this without considering as-yet-unresolved questions about the level of funding coming to schools and the support services they use is going to be in trouble, too. However, I do worry that the government is pushing ahead too quickly as a result of ideological views – such as that giving parents yet more data on schools will drive up performance – when a calmer look at what actually impacts on schools' actions might be of benefit. Lurking behind all of this, I think, is the need for a really detailed exposition based on a genuine search for understanding of what the government – central and local – can do well, and what it tends to do less well. I have yet to read this from either this government or from the last. One final point: ministers need to be held to account for their selective use of evidence on the performance of the English education system. The business plan includes a section seeming to justify the case for reform by setting out how the performance of our schools in one main international testing study has fallen in recent years. It does not mention that, in the other main testing study, results have actually risen, with England registering the largest gain in primary maths of any nation against which results can be compared since the mid-1990s. This is not a Parliamentary debating contest, but a plan which will affect schools throughout England. Teachers, and the public, should be treated with more respect. - I have written a long blog as a riposte to a study from Bristol University last week on the effects of league tables in England and Wales. It is viewable here: http://bit.ly/dqWLc0 http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=395 Wed, 10 Nov 2010 09:30:51 GMT 20101110093051 Wed, 10 Nov 2010 09:30:51 GMT 20101110093051 Wed, 10 Nov 2010 09:32:42 GMT 201011100993242 What is the proper role of the state in education? And how great should be the influence of the politicians who stand at the centre of the stage in this debate? I find myself asking these questions again and again. For while democratic influence over state-funded schooling is vital, evidence of what appears to be dysfunctionality in how it often currently works seems to come up so frequently. The backdrop for thinking this through again was Cambridge Assessment's annual conference in Cambridge last week. This is an occasion which can focus on the technical aspects of this part of education – the keynote was a talk by the influential academic Professor Paul Black on problems with formative and summative assessment – but where wider topics such as the influence of the state have also been a common theme in recent years. A presentation by Professor Jo-Anne Baird of the University of Bristol provided some interesting insights. Professor Baird has been part of a research team looking at how assessment reforms for 14- to 19-year-olds have taken shape in schools and colleges. The project, funded by the Qualifications and Curriculum Development Agency, has used interviews and questionnaire surveys to test the views of school leaders, chairs of governors, teachers and students in 52 institutions. Recent years, of course, have been incredibly busy for reform in this area, with changes including the Labour government's diplomas and functional skills, new GCSEs and A-levels and in the related area of a new key stage 3 curriculum. The major finding to emerge from the research with school leaders may not come as a surprise to many of you in this position. Faced with the scale of this change, schools had reacted by taking a pragmatic view, only fully implementing reforms which they believed were in the interests of their students.* “We found that schools and colleges were being led strategically by the savvy education manager who had political nous,” said Professor Baird Part of that “nous”, she said, was realising that many had seen through several “waves” of education reform, and that they had to take a decision as to which ones would be likely to endure, taking seriously only those that they thought would last and which could benefit their students. Professor Baird acknowledged that in one sense this was unsurprising: the research was carried out just before this year's general election, which seemed likely to bring about a change of government, and Labour's diplomas seemed particularly vulnerable. But the stance taken by school leaders was more profound than being simply a product of the timing of the research, she suggested. “These institutions, and their leaders, were going to be around for far longer than many of these policies, so they had to take a longer term view than any of the politicians,” she said. There was some discussion of the implication of this point – that policy developments could be taken come without real consideration of their long-term impact, at least by those at the centre - afterwards. Dr Geoff Hayward, of Oxford University, said civil servants were moved around so often to and from the Department for Education (or its predecessor bodies; its frequent name changes are indicative in themselves, I think) that there was virtually no “policy memory” within government, such that initiatives would seem new to officials even when they were repeating past reforms. There are other examples of dysfunctionality in the policy-making process, I thought. I was shocked, in the run-up to the general election, when years of planning for primary curriculum change in the form of the Rose Review were scrapped suddenly because the Conservatives and the Liberal Democrats would not allow legislative changes through Parliament. This is a policy failure – in terms of the waste of time and resources, I thought - not of one party or another, but of England's system. How can schools plan properly, when there might be a lurch from one policy to another when political circumstances change? Speaking to open the event, Simon Lebus, chief executive of Cambridge Assessment, had cited an example of David Miliband, schools minister at the time, answering a question at another conference in the wake of the 2002 A-level marking crisis. The questioner asked: should ministers not wait until initiatives had been fully trialled before introducing them? Mr Lebus said: “The minister responded by saying that when politicians had good ideas for change, their instinct was to implement them immediately. This was because if they were good ideas and they weren't implemented straight away, it was a lost opportunity”. Tony Blair writes, in his recently-published autobiography, that effective politics boils down to taking a stance on simple questions, including: “Are you for reform or status quo in public services?” It's the wrong question, I thought. In a more rational world, the questions would be: “Reform to what end?” and “How best can we promote improvements in the public services, without damaging what already works well?” The conference also heard a suggestion that other countries did things differently. “In France, education is regarded as so important that it should not be left to the various manoeuvrings of the political classes. “In Germany, they took 10 years to decide whether modularisation [splitting exams into smaller units] was a good idea or not,” came a comment from the floor. This idea that politicians, if left to their own devices in their desperation to be viewed as changing things for the better, can get in the way of coherent reform designed to support what goes on in classrooms, is powerful. I wonder whether we will ever get to a situation in England where all major parties might come to an agreement on a better policy-making process. In any case, with another set of “reforms” to be announced next month with a white paper which will set out changes to the accountability system including the introduction of a new “English baccalaureate” qualification and curriculum proposals, it is time to ask serious questions of the way policy-making is done. More details of the research study are here: http://bit.ly/aM4Tuo *My own view, though, is that unfortunately, because of the accountability regime, decisions are not always taken with students' interests as the only consideration: results pressures mean that the needs of the institution to convince outsiders of success have to be kept in mind. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=387 Wed, 27 Oct 2010 14:38:04 GMT 20101027143804 Wed, 27 Oct 2010 14:38:04 GMT 20101027143804 Wed, 27 Oct 2010 14:40:18 GMT 201010271424018 I spoke at a conference on assessment this week, during which a common theme began to emerge. I must sound like a stuck record writing this, but genuinely, the theme, expressed throughout the day, was the frequent conflict between assessment and accountability, and the damage often being done by high-stakes testing and examining. There were many interesting insights and suggestions. One speaker, who has spoken in defence of national tests in the past, put forward the idea that they should not be used to judge teachers and schools. Another, from an employers' group, bemoaned the fact that so much emphasis was placed on qualifications in schools, when companies often regarded them as only a relatively small part of the job application process. There were several comments that high-stakes assessment made schools, and qualifications developers, reluctant to innovate. Well so far, so unsurprising, you might think. But one aspect that I did find interesting was that the man chairing the event, Lord Knight of Weymouth, better known as the former schools minister Jim Knight, seemed not to be disagreeing with the general thrust. This is the same Lord Knight, you will no doubt recall, who in his former position had to defend the very system many of us have been criticising for years. At the end of the event, however, Lord Knight sketched out a position which seemed to be a bid to rethink his party's approach to education, based on an analysis of what he thought had gone right, and what had not, during Labour's years in power. He seemed to have no difficulty conceding that high-stakes accountability was part of the problem. And what was needed was some fundamentally rethinking of the way education now operates. The last government had “thrown everything we could at the existing paradigm of a schools system in this country”, he said, and “driven it as hard as I think it is possible to drive it”. There had been a 50 per cent real terms increase in resources directed at schools, he said, while capital investment had risen six-fold. Teacher numbers had increased by close to 40,000, while the support staff roll had swelled by around 130,000. The literacy and numeracy hours had been introduced, and “lots [else] that I am really proud of”, he said. He then assessed what had been achieved in return for this investment and reform programme. “We got some pretty good improvements in results...in test score results, if that's your definition of a good education,” he said. Reading scores for 11-year-olds had soared, he said, before plateauing, and the proportion of pupils achieving five good GCSEs, including English and maths, had risen from 36 or 37 per cent to 50 per cent, while the numbers going to university had also climbed markedly. But this was just the precursor to a list of concessions. Lord Knight added: “If you measure the success of our education system on the basis of those not in education, employment or training between 16 and 18, [that number] did not really shift. It remained on about 10 per cent, and that is shocking. “If you measure it on tackling truancy, those figures remained about the same, as an indication that those kids were not enjoying school. “If you measure it on the gap between the attainment of those children on free school meals, and those not, that did not really shift: one or two per cent was all we managed.” Radical change, then, might be needed. He said: “We have got to look fundamentally at the system itself, and the paradigm around which we think about schools.” Children needed to want to come to school, and one of the areas that policy-makers would need to look at was accountability. He said: “We have too many pupils who are not having a great time. They are getting really good at passing tests, but not enjoying education as a result. “It's too high stakes...and we have got to find different forms of accountability. “We need to think about how we can get proper school-level accountability that lowers the stakes around testing so that we can genuinely engage every pupil, and also meet the needs of the labour market.” Well, intrigued, for want of a better word, having read so many justifications of this high-stakes system from Labour in the years leading up to May, I had a brief chat with Lord Knight afterwards. And it seems these were not just off-the-cuff remarks prompted by the discussions of the day. He had made a similar speech at a fringe meeting at Labour's party conference. It is, perhaps, predictable that soul-searching will be going on in the party after the general election result, and it should be borne in mind that, even if Lord Knight, with his experience and involvement in education, has influence on the party's leadership on these matters - Labour is unlikely to see power for a while yet so these ruminations may have little real-world implications. Many will also ruefully wonder why the former minister did not express these thoughts publicly while in government. But I guess you could take this as a sign that the high-stakes accountability system we have grown so familiar with is very much still open for debate. Alternatively, you might think that it is easier for politicians to speak out after they move away from power. In either case, these were very interesting comments. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=382 Fri, 15 Oct 2010 10:10:47 GMT 20101015101047 Fri, 15 Oct 2010 10:10:47 GMT 20101015101047 Fri, 15 Oct 2010 10:11:44 GMT 2010101510101144 Some very interesting research came out on testing last week. Maybe I would say that, as I worked with the Wellcome Trust charity to write the document which summarised its findings. However, I think that it would be worth chewing over the conclusions of the trust's investigation, which surveyed the views of nearly 1,000 pupils and more than 200 parents in England and Wales on science assessment at key stage 2. The research is significant because – amazing though this sounds after getting on for 20 years of Sats testing now, and despite many studies by the teacher associations – in-depth analyses by the Government and its agencies of what pupils and parents actually think of the tests, and the learning experiences of children as they prepare for them, have been hard to come by. The investigation, led by academics at Queen's University, Belfast, produced some very detailed findings, which accordingly drew on the face of it contradictory interpretations in different newspapers. The Daily Mail emphasised the fact that the surveyed children generally felt that Sats had a negative impact on their friendships and home lives: in England, 62 per cent said Sats impacted badly on their friendships, or home lives, or both. The Mail also quoted a series of opinions from children on how stressful some find the whole experience, including worries about feeling pressurised by their parents. However, the TES highlighted another finding, which revealed that more than half of the surveyed pupils in England were actually disappointed with the Labour Government's decision last year to scrap the Key Stage 2 science tests. Labour ministers made that move after accepting a recommendation from the former government's “expert group” of advisers, who said that the focus on the tests could take attention away from practical work in the subject, reducing some children's enthusiasm for science as a result. The decision was announced just before the surveyed pupils took the 2009 tests, meaning they were the last to sit them in England. Some 56 per cent of children said they disagreed with the move, compared to 26 per cent agreeing. The most frequent reasons given by the children in opposing the move to scrap Sats were that children would not learn as much about science, and that they would not know their levels in science as a result. Although they thought that assessment in science was important, however, they were not sure that Sats were the best way to do it. The pupils' perspective is clearly important. However, two findings which were not mentioned in the media coverage are also worth discussing. First, the report provides strong evidence that teaching to the test – at least in relation to science – is pervasive in English primaries. Although this may sound obvious, I am occasionally challenged over this issue, as some people argue that extensive test preparation does not go on. Yet in this study, some 53 per cent of pupils said they completed practice Sats science papers “very often” in lessons, compared to only 14 per cent in Wales. So it did take place in most primaries. Was it the best use of pupils' time? I remain sceptical, and this was borne out by the views of the surveyed children, only 10 per cent of whom viewed practice Sats as the best method of finding out how they were doing in the subject. Second, the researchers also asked what parents in both countries thought about the decisions in England and Wales to scrap science Sats. Wales, of course, dispensed with English-style end of key stage 2 testing in 2004. The researchers found that, in England, some 36 per cent of parents thought that the decision would be a change for the worse, compared to 23 per cent believing science lessons would improve. However, in Wales, nearly six times as many parents believed science lessons actually had improved as a result of the decision to scrap Sats (45 per cent), as thought they had been harmed (eight per cent). In addition, while 21 per cent of English parents worried that children would learn less science as a result of the decision to scrap Sats, only three per cent of parents in Wales said this had actually happened. By contrast, some 27 per cent of Welsh parents said children there now learnt more. It could be argued that the sample of Welsh parents was relatively small (74 parents), and that perhaps some did not have children going through the system before 2004, when Sats were still around. The experience in Wales is also different from that in England, in that Sats for all three tested subjects were dispensed with at the same time. In England, of course, tests in English and maths – together with a hyper-active accountability regime based upon them - survive. However, this finding, together with 2008 research from the Wellcome Trust which revealed that some 62 per cent of Welsh heads surveyed believed teaching standards had improved since Sats were scrapped (with only 17 per cent disagreeing), is very interesting, in providing a perspective from people who had already lived with change, rather than those who were only preparing for it. Will the end of Sats spell a decline in science teaching, as some parents and pupils in England clearly fear? And would all subjects suffer if the current Sats-based accountability regime were reformed on the Welsh model? The perceptions of parents and teachers in Wales, as monitored by these reports, would suggest not. - The media release and full reports can be viewed here: http://bit.ly/bN9eIE http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=370 Tue, 28 Sep 2010 10:37:36 GMT 20100928103736 Tue, 28 Sep 2010 10:37:36 GMT 20100928103736 Tue, 28 Sep 2010 10:38:46 GMT 2010092810103846 It's just a pity it took you so long to spot this One comment by Tony Blair in recent coverage of his book had my jaw dropping so much that I resolved to buy the 718-page doorstopper. So job done from his point of view, I suppose, and another donation to the Royal British Legion. I cannot confess to having ploughed through all of “A Journey” yet. But the section I am talking about, on Freedom of Information legislation, is certainly worth analysing on its own merits. On page 516 of the book, Mr Blair describes his decision in the late 1990s to open up some of the workings of government to greater scrutiny, by introducing a Freedom of Information act, as an enormous blunder. He uses two arguments in explaining how he ended up arriving at this view. First, although Freedom of Information legislation, passed in 2000 and finally made available from 2005, is presented as serving “the people's” right to know what is being done in their name, in fact it is not generally used by “the people”. It is used by journalists, as a weapon against politicians, he argues. It also handed “gigantic” power to reporters and broadcasters, he writes. Second, its existence inhibits decision-makers from committing to paper their honest thoughts on the likely effect of a particular policy, for fear that this will eventually be reported in the media. This, he says, makes for a “thoroughly bad way of analysing complex issues”. To address the second point first, I find this view surprising, as I have experience, as a journalist, of how the thinking and discussions behind policy decisions can be protected from wider discussion, even when an application for that advice is submitted under the Freedom of Information Act. Under section 35 of the Act, discussions can be - and in my experience usually are - exempt from release if they relate to discussions around the formulation of government policy, and if it is felt by the government that releasing them would be so detrimental to the policy-forming process, for the reason Mr Blair states, that this outweighs the public interest in transparency. I have had several FOI requests blocked in this way. While it is true that journalists are enthusiastic users of the FOI act, interested and particularly campaigning members of the public –and often those who are unhappy with a policy being implemented by central or local government – also clearly use it extensively. But the reason for my shock in the first place was more substantive. Mr Blair obviously has now decided he does not like the effect of transparency, from his own point of view. Yet transparency has been one of the central planks of his public sector reforms. Arguments for it, indeed, have helped to justify the league table system of publishing performance information with which every school leader has had to become very familiar. In the book, Mr Blair praises the work of Sir Michael Barber, the former teacher and National Union of Teachers official who went on to head the Standards and Effectiveness Unit at the Department for Education and Employment, and then Mr Blair's “Delivery Unit”, which oversaw public service performance. Sir Michael's 2007 book on his career in Whitehall, “Instruction to Deliver”, can be read as a hymn to the power of league tables, which he sets out as a way not just of monitoring the performance of public sector institutions such as schools, but of ranking the performance of individual government departments. Sir Michael writes: “Not everyone in the public services likes league tables, but I love them. I have spent much o f the last decade advocating them, usually in front of sceptical or even hostile audiences of head teachers.” The first advantage of league tables, he writes, is that “they make the evidence about performance public”. Sir Michael has also advocated “devolution and transparency” – handing some control of decision-making power to professionals but, crucially, using data to hold them to public account – as a possible model for public sector reform. In “A Journey”, Mr Blair writes that under Sir Michael's regime “those charged with delivering knew they were being monitored”. All of this is coherent, if highly debatable, of course. But the staggering thing about Mr Blair's remarks is that, taken alongside his public sector reform policy, he seems to be saying: transparency is great, so long as I'm not on the end of it. Of course, he is right to argue that public information can be used as a weapon, but again there is no self-awareness in the sense of acknowledging that this can happen to just the sorts of organisations he has argued should be made more transparent. Advocates of putting more information in the public domain through Government-endorsed performance tables, such as Sir Michael, can sometimes come across as suggesting there are no downsides: that the public have a right to know and that the effects of giving them data will have only benign effects. Yet, clearly this is not the case. As no reader of this blog will need telling, league tables can stigmatise schools which feature at the bottom of the rankings. This is often unfair, given that raw score rankings suggest huge differences in the quality of schools when mountains of research says that much of the difference comes from the characteristics of the pupils, not the teaching they receive. I have also argued that the publication several times a year of results statistics for English schools as a whole gives a sceptical media the chance to suggest, several times a year, that our education system is failing, on the basis that x percentage of pupils have failed to meet Government expectations, whatever the percentage is. That is, data, seemingly put out in a neutral sense of encouraging transparency, is being used as a weapon in this sense. All of this is not to argue, necessarily, for a reduction in transparency. But it is to call for greater recognition that with that transparency can come big consequences. Some people have long argued that, for example, league tables should at the very least come with caveats or health warnings, spelling out to the public the limits of the data, that all statistics are subject to measurement error and how accepting these figures as categoric evidence of school quality would be a mistake. This has never happened, though, with ministers appearing less than interested in the side-effects of transparency and the problem of over-interpretation. One of the reasons politicians and civil servants love league tables and are thus reluctant to view them as the problematic instruments they are, I think, is because of the power they give to those overseeing this system. It is great, then, that Mr Blair has realised that increasing the information flow can be, at least, double-edged. It is just a shame he appears not to recognise that others, on the end of his government's own push on “transparency”, may see it in exactly the same way. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=366 Thu, 16 Sep 2010 15:26:53 GMT 20100916152653 Thu, 16 Sep 2010 15:26:53 GMT 20100916152653 Thu, 16 Sep 2010 15:26:53 GMT 201009161532653 What was the effect of the new A* grade on the school backgrounds of England's top performers at A-level? Predictions of its possible consequences have been the subject of intense debate for years, with many people worrying that the new grade's advent could be a set-back for social mobility. With pupils from independent schools likely to dominate the higher echelons of the mark scale, it was said, universities would find it harder to reject their claims and those educated privately would win more places at the most selective institutions. I have written about this a fair bit myself, reporting as long ago as 2004 on a study from the exam board OCR which looked at existing A-level results and then modelled what would have happened if two new grades above the A grade had been introduced. The proportions of students from the independent sector – as opposed to those from state schools and colleges – getting the top grade would have jumped by more than 10 percentage points in several subjects, the study reported. Other studies from exam boards since have also pointed to the A* being likely to benefit students from the fee-charging sector, who would garner a greater share of the top grade than they had done when the A grade was the highest possible. I even sat in on a session of the House of Commons Children, Schools and Families select committee when the chairman, Barry Sheerman, expressed passionate concerns about the A* for this reason. However, a close analysis of this year's data suggests its effect has actually been modest. The key statistic to look for is not the relative success rates of private versus state schools – which are widely reported and worried over– but the share of the overall number of top grades awarded which go to the state, and to the independent, sectors. By my calculations based on data provided by the Joint Council for Qualifications, independent school pupils garnered 27 per cent of the A grades awarded in 2009, compared to 31 per cent awarded to comprehensives. Among the other categories of institution, 23 per cent of A grades went last year to those educated in further education and sixth form colleges; 19 per cent went to state grammar school pupils and fewer than one per cent to students from secondary moderns. In 2010, some 30 per cent of all A*s awarded went to private school pupils. The same percentage was awarded to those from comprehensives, while the percentages to colleges, to selective state schools and to moderns were 20, 19 and one per cent respectively. In other words, the effect of the A* has been to increase the independent sector's share of the top grade from 27 to 30 per cent. Comprehensives have been largely unaffected and the largest “loser” from the change has been colleges, whose share has fallen from 23 to 20 per cent. Some will see these figures and argue that any increase in “market share” for the private sector is a reason for great concern. I haven't seen a subject-by-subject breakdown of results by sector, which might show, as is sometimes claimed, independent schools dominating subjects such as languages and sciences, which do carry currency in university admissions. But my feeling overall is that these statistics are underwhelming as evidence of any momentous shift. The independent schools' “market share” statistics are also more modest than an analysis by the Independent Schools Council (ISC) suggested they would be in June. This said that ISC schools – which are, it is true, only a subset of independent schools as a whole - would win 36.5 per cent of A*s if the award had been made last year, and also cited evidence from the AQA board predicting a 33 per cent figure for the independent sector in total. One final thought on the results season. Two major changes at A-level - the advent of the A* and the reduction of courses from six modules to four - presented a big challenge for the boards and Ofqual, in the technical exercise of trying to ensure that grades from this year remained of a comparable standard to those of 2009. And, while there were worries beforehand about a grading shambles of the type which dominated education headlines for months in autumn 2002 the last time there was a major structural reform, this never happened. All in all, touch wood, I think the A-level reforms have passed off relatively smoothly in the end. That's probably bad news for education reporters, but good news for everyone else. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=357 Tue, 07 Sep 2010 16:47:40 GMT 20100907164740 Tue, 07 Sep 2010 16:47:40 GMT 20100907164740 Tue, 07 Sep 2010 16:53:45 GMT 201009071645345 A change in the way science tests for 11-year-olds were carried out this year, and the consequent effect on results, has led to national headlines that standards are now at their lowest level for more than 10 years. I am really struggling to get my head around how such negative stories can be generated when the data on which they are based clearly do not support this inference, even though I warned in the spring on this blog that this misinterpretation could happen. The basis for the reporting was that the proportion of pupils achieving level 4 in science dropped from 88 per cent last year to 81 per cent this summer. The proportion reaching level 5 fell even more steeply, from 43 to 28 per cent. The reporting is fair enough, then, you might think. Except that the two years' results were generated under hugely different conditions. Trying, then, to use the scores created by the two systems as evidence that standards have slipped is just not valid. Of course, last year's science tests were taken by the whole 600,00-pupil cohort in England. The results were used to generate league tables, as well as being important to Ofsted judgements, targets and the panoply of mechanisms which is England's school-by-school accountability system. Last year, the last government wisely accepted a recommendation from its “expert group” on assessment which highlighted the downsides of this approach, including the negative effect on science teaching of over-extensive test preparation. It is also the case that, perhaps paradoxically, more insightful judgements on overall standards can be achieved by testing fewer pupils. So this year we have a new system. Only five per cent of year six children nationally were tested in science. And, crucially, the results were “low-stakes” for their schools: league tables would not be constructed according to them and they would form no other part in the accountability system for individual institutions. It should surprise no-one, then, that results fell back in these circumstances. Schools which were encouraged by the previous system to spend months preparing children for these tests, including extensive teaching to the test, no longer had any incentive to do so. Also, pupils may well have felt less of a need to try hard at the new tests, since their schools would have had no reason to tell them that they were important. To state what should be obvious, then, the change in results would appear to say much more about the alteration in the testing system than about any underlying change in standards of science education. To be fair, I cannot be sure about this. It is just a hunch. But clearly, neither are the results evidence of changing underlying educational standards, when the measurement system has also changed. It is, surely, basic science that if you want to measure change, you do not change the measurement mechanism. To use a comparable example from economics, you can imagine the scenario if the official means of measuring inflation was changed, leading to a rise in the number which was generated at the end of the process. Would we really expect to be reading that inflation was at its highest level for years, based only on a comparison with numbers generated under the old measurement system? This would just not be credible. The error here is not the Government's, however: the lack of comparability of the figures is made clear in its official statement on the results. It is ridiculous, then, to say that this is evidence that standards of science education are lower now than they were in previous years. That does not mean, though, that the new figures are not revealing in their own way. While they do not show that standards have slipped, they may offer some sign, as other reports said, that the previous figures generated under Sats were inflated by teaching to the test. There is much evidence, of course, that teaching to the test is going on in a lot of schools. The gap between the new figures and the old would appear to suggest that when incentives for schools to go in for close test coaching are taken away, we get a more realistic picture of performance. I would, though, be a bit cautious even before reaching that conclusion. The absence of teaching to the test in science, and perhaps schools' decision to focus even more attention on the remaining “high-stakes” Sats subjects of English and maths will, I think, explain a good part of the “fall” in results this year. But it may also be the case that children who are no longer being told by their schools that these results are important to them have simply not tried as hard as their predecessors did. That does not mean they have learnt less science as a result, by the way. It just means they might have put in less effort for the tests themselves. There may be quite a large element of silver lining in these stories, though, for heads who would like to see the back of Sats. If the media do come to believe or imply – rightly, in a sense, although I am not sure there will ever be any one “true” measure of national education standards – that sample tests are revealing a truth about underlying standards which have remained hidden by Sats and their tendency to promote teaching to the test, perhaps the move towards sample tests will have been strengthened. Sample tests, as I have written earlier on this blog and many others have also argued for a while now, are potentially a much better way of measuring national education standards, without educational side-effects, than Sats. But let us wait until they have been in place for more than one year before we start trying to use them to judge trends in national education performance. Supporters of teacher assessment will have been cheered by the following paragraph on the “research and statistics” section of the Department for Education's website. It reads: “From 2010, Key stage 2 science at school level will therefore be assessed by teacher assessment only, which takes greater account of pupils' practical grasp of the subject and is based on their attainment throughout the academic year across the full programme of study.” If this is true of science, many would say, is it not also true of English and maths? http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=351 Thu, 12 Aug 2010 08:59:52 GMT 20100812085952 Thu, 12 Aug 2010 08:59:52 GMT 20100812085952 Thu, 12 Aug 2010 09:01:31 GMT 201008120990131 How reliable was the marking of last year's key stage 2 tests? If you read the official review of the quality of the 2009 marking process by Ofqual, published earlier this month, you are likely to come out confused. This is because, in my view, the central questions - did pupils and schools get the results they deserved? - are not answered in this report. This is despite Ofqual's homepage listing as among its priorities ensuring that “all learners get the results they deserve”. The findings of Ofqual's monitoring report are broken into three, under “key successes of the 2009 cycle”; “key concerns...”; and “conclusions”. Under “key successes”, the report says that the tests yielded results which were consistent across the country and that “the low number of changes to results after schools had requested reviews could be interpreted as evidence of the reliability of the tests”. Yet under “key concerns”, many difficulties are listed with the system by which markers were trained in the use of the mark scheme, particularly in the English tests, and with the content of that mark scheme. In English, the report says, the amount of time spent training markers on the interpretation of the mark scheme for writing was less than that for reading, and generally was “rushed”. Some questions were framed with a “lack of precision”, which forced the mark scheme into becoming complex – in one case 23 bullet points were needed to convey instructions helping examiners mark a two-mark question - and “ambiguous”. Extra guidance on interpreting the mark scheme had to be sent out mid-way through the marking process. Some marking personnel were not trained in how to mark all the questions, and overall there were inconsistencies in the way markers were trained, which “may have led to dissemination of incomplete, confusing and inconsistent messages about the application of the mark scheme – particularly for new markers”. The report also identifies some “bias” in some of the questions, going into particular detail about the notorious longer writing task in which pupils were asked to write about a pair of trainers. This was biased towards those with an interest in sport, said the report. Some 30 per cent of teachers complained about it after the test was trialled with them, reveals the report, and pupils' average scores on it during the real tests were the lowest for any task set over the past four years. Overall, the report provides evidence to back up what a marking source told me last year about the difficulties marking colleagues had in interpreting the mark scheme and so awarding consistent scores to pupils, although the marker said it only “skimmed the surface” of these problems. In conclusion, then, under “reliability”, Ofqual's report says that, in the reading test, it was “difficult for consistency to be achieved”. In other words, it was not clear that a paper marked by one marker, using the mark scheme as best they could, would receive the same mark as if graded by another marker. But problems of inconsistency were not confined to English; difficulties with the training of maths markers were also mentioned. “The extensive use of additional guidance to supplement and even correct the mark schemes inevitably leads to inconsistency in marking,” says the report. It concludes that the Qualifications and Curriculum Development Agency, the soon-to-be-defunct organisation which has overseen the tests, should ensure that any late changes to the mark scheme are kept to a minimum, and that, when needed, changes are communicated consistently to markers. To be fair to Ofqual, it is useful to have an independent authority looking at how the marking process worked, and pronouncing on any failings publicly. This report may well lead to improvements in future. However, overall I can understand why heads have been reported in the TES as being frustrated by this report. For it fails to follow up on its conclusion that there were serious problems with the marking process with the logical next questions: did pupils and schools lose out as a result? And were the marks generally accurate at the end of this process, or not? Reading the report, it is unclear whether the problems with the mark scheme and training are likely to have led to a few wrong marks here and there for children, or wider inaccuracy. And the seeming acceptance that the low number of successful reviews in itself was evidence that marks were generally acceptable* seems remarkably uncurious: did Ofqual do anything to investigate some schools' suspicions that the marking agency was institutionally reluctant to admit that it had got marking decisions wrong? If there were such systematic problems with the mark scheme, why was this not reflected in many more markers' judgements being over-ruled at appeal? And you can only conclude that if they are reading the report, those conducting the reviews process might now have an incentive to let fewer appeals through in future, if this will be taken as evidence by the regulator that the original marking was accurate. Implicit in the report is a sense, I think, that problems with the marking process are just that – problems of process to be sorted out the next time the tests are sat. The difficulty is that the test results carry real consequences, in this case particularly for schools and potentially for the careers of those who lead them in light of one bad set of scores. If the 2009 tests did result in inaccuracies, Ofqual should be doing more to try to identify the scale of any potential injustice, and ensuring more is done not just to improve the consistency of marking in future, but to protect pupils (for whom the results are medium-stakes) and particularly schools (very high stakes) when things go wrong, as they seem to virtually every year. In passing, I wonder if the current system would stand up if it were ever challenged legally by a head who lost his or her job because of a poor set of test results they thought had been inaccurately marked. I personally think there may well be a case now for an extra stage in the appeals process, with schools given the chance to take their case to a third party, if they are unsuccessful at initial review stage. Otherwise, the suspicion must remain that, while this system is used to hold schools to account with huge consequences when results slip, meaningful counter-balances protecting them when it is the marking system, rather than the school's teaching, which is at fault are less than adequate. - I have written another blog about the possible effect of the boycott on what can be read into next week's national KS2 test results. You can view it at: http://www.educationbynumbers.org.uk/2010/07/26/sats-boycott-how-were-national-results-affected/ *Interestingly, in its own report on the marking reviews process last year, the QCDA itself said: “QCDA does not believe that the outcomes of marking reviews are a reflection of the quality of marking in any particular year because many other factors can influence the number of reviews and the outcome.” http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=346 Tue, 27 Jul 2010 11:31:41 GMT 20100727113141 Tue, 27 Jul 2010 11:31:41 GMT 20100727113141 Tue, 27 Jul 2010 11:31:53 GMT 2010072711113153 How did support for the Sats boycott vary across the country? I have carried out an analysis of government information provided last week on the schools which decided to follow the advice of the NAHT and the National Union of Teachers and refuse to take part in the administration of the tests this year. It reveals variations in the take-up of the boycott in different areas of England. Scrutiny of this information by local authority shows that in 24* local council areas, at least 50 per cent of schools joined the boycott. This represents almost one in six local authorities overall, and will mean that league table results, to be published early next year, will be almost unusable if based on test data alone in these areas. Performance tables, then, will rely especially heavily in these areas on teacher assessment judgements. The North East appears to have shown the strongest support for the boycott, having four of the top 10 authorities ranked on the percentage of their schools backing the action. Hartlepool tops the list overall, with all of its schools taking part in the boycott, and with North Tyneside; Redcar and Cleveland; and Middlesbrough all having more than three quarters of their schools boycotting the Sats. Schools in this region tend to have higher-than-average numbers of children eligible for free school meals, government figures show. However, more prosperous areas are also represented in the higher echelons of this table. Rutland, for example, which official data suggest has the smallest proportion of primary children eligible for free meals in England, had 15 of its 17 schools taking part in the boycott. At 88 per cent, this was the second highest level of support for the action of any authority. At the bottom of the table, 10 local authorities appear, from the government's list, to have had none of their schools joining the boycott. I find the contrast between local authorities in nearby areas particularly interesting. In Islington, in north London, for example, none of the 44 schools is listed as boycotting the tests, though in each of nearby Haringey, Tower Hamlets and Waltham Forest, more than half of schools joined the action. Other reasonably large authorities with no schools joining the boycott included the Wirral, Knowsley and St Helens. If there is a pattern, it is one of a slight North-South contrast. The North East; Yorkshire and the Humber; and the North West had the highest percentages of schools taking part in the boycott, while the East Midlands; South East; and East of England, had the fewest. Also, despite the exceptions in individual authorities described above, the regions of the country with relatively high numbers of pupils eligible for free school meals tended to have more schools joining the boycott, while the four districts with the lowest proportions of free school meals pupils – the South East, the East of England, the South West and the East Midlands – saw the lowest take-up of the boycott. Overall, the national proportion of 26 per cent of schools taking part in the boycott represented, I think, a strong vote of no-confidence in the current high-stakes testing regime from a constituency which, as you know, takes industrial action very reluctantly. I wrote a reaction to the national boycott figures last week. It can be viewed here I would also be interested in comments from anyone with any observations, perhaps from a local perspective, about these figures. Please either comment below or email me at warwickmansell@gmail.com Local authorities taking part in the boycott, with the percentage of schools joining the NAHT/NUT action: Top 10: Hartlepool (100 per cent); Rutland (88 per cent); North Tyneside (84 per cent); Sefton (82 per cent); Middlesbrough (81 per cent); Redcar and Cleveland (78 per cent); Stoke-on-Trent (70 per cent); Torbay (68 per cent); Dudley (67 per cent); Calderdale (67 per cent). Bottom 10: Islington; Halton; Knowsley; Wirral; St Helens; Isle of Wight; Bournemouth; Thurrock; Kingston upon Thames; Bracknell Forest, all of which had no schools boycotting the tests. Government regions, with percentage of schools joining the boycott: North East (39 per cent); Yorks and Humber (35); North West (31); Outer London (28); Inner London (27); West Midlands (26); South West (20); East Midlands (18); South East (15); East of England (12) *The analysis is based on information stating which schools boycotted the Sats from the Department for Education. I've not included data from England's two smallest local authorities – City of London and the Scilly Isles – which only have one primary school each. I compared the total number of schools taking part in the boycott for each LA with the department's records of the number of primary schools in each local authority. This may slightly underestimate the support for the boycott in some authorities, as the department's total for the number of primary schools in each local authority is slightly higher than the number it said last week should have administered the tests. This may be because of the presence of infants' schools in the department's figures for total number of of “primary” schools in each authority. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=342 Tue, 13 Jul 2010 10:28:16 GMT 20100713102816 Tue, 13 Jul 2010 10:28:16 GMT 20100713102816 Tue, 13 Jul 2010 10:29:32 GMT 2010071310102932 It was one of the most dispiriting examples of how the political process can get in the way of carefully-considered decisions on education policy that I have witnessed. In early 2005, with an election looming, Tony Blair decided to overrule the advice of the Tomlinson committee, which had investigated qualifications reform, by deciding that GCSEs and A-levels would be staying more or less as they were, without becoming part of a larger, overarching qualification called the diploma. Fair enough, some might say. Except that Mr Blair's government had given Sir Mike Tomlinson, the former chief schools inspector, two years to come up with a blueprint for reform which, while far from perfect and potentially very complicated, had managed to win support from most within education. Why, if the intention had always been to keep the system they had, had the inquiry ever started? These thoughts came flooding back this morning as I read that David Miliband, the former foreign secretary and now Labour leadership front-runner, had called for another rethink of the entire 14-19 qualifications system. Pupils, he said, were spending too much of their latter years of secondary school locked in to an “obstacle course” of exams. “Success is often being achieved despite the testing system and not because of it,” he told the Guardian newspaper. Education, he added, still had to be Labour's top priority. David Miliband is no newcomer to qualifications reform. Way back in 1990, as a young research fellow at the Institute for Public Policy Research (IPPR), he was one of six authors of an IPPR pamphlet called “A British 'Baccalaureate'”, which put forward the idea – which was never taken up but appears to have retained some attraction among more radically-inclined thinkers on qualifications reform - of replacing A-levels with a continental-style qualification offering a broader framework for study than the traditional three-subject A-level experience. Then as schools minister from 2002, he joined his boss, the education secretary Charles Clarke, in seemingly supporting the Tomlinson vision in behind-the-scenes negotiations with Downing Street. However, with the Government about to respond to Tomlinson, both ministers were moved away from education by Mr Blair, with Ruth Kelly becoming secretary of state. Miliband now says: “I had been working for two years on this historic English problem of the 14-19 curriculum. I think it was a historic error really that we did not follow through on the vision of a unified world class academic and vocational framework for curriculum and testing.” Like all four former Labour ministers who are now contesting the party's leadership, Mr Miliband will have to face the question as to why he did not speak out on this when in office. But the more important point, for schools, is the substance of what he is saying. He is right, of course, to criticise the “obstacle course” of exams. It is extraordinary that, towards the end of secondary school, teenagers can now find themselves faced with four years, from year 10 onwards, dominated by preparation for public exams. You would struggle to find any other country where this happens. Perhaps more pertinently, is this what anyone connected with education would have come up with, if they were designing a system from scratch? I find that hard to believe. It has come about because individual reforms such as the Curriculum 2000 changes, which split the A-level into two, and the move towards modular GCSEs, have happened without anyone taking an overarching look at what this would mean, in educational terms, from a student's viewpoint. Tomlinson was an attempt at just such an approach. The only debate is the extent to which his reforms are more or less completely buried now, with the more vocationally-orientated diploma which eventually emerged under Labour gradually being marginalised by the Tories. A fresh look at this problem, which should also not ignore the central issue of how exam results are used to hold schools to account, is urgently needed. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=331 Tue, 29 Jun 2010 16:43:23 GMT 20100629164323 Tue, 29 Jun 2010 16:43:23 GMT 20100629164323 Tue, 29 Jun 2010 16:43:55 GMT 201006291644355 Do those responsible for co-ordinating the marking of Sats tests have a death wish for these assessments? The thought does occasionally come to mind as I contemplate yet more confusion surrounding the marking of the key stage 2 tests and what looks to this observer like a brewing public relations problem connected to the release of information to schools. As I reported in Friday's TES, this year's KS2 English tests have been affected by behind-the-scenes difficulties with the way examiners tot up the marks they award to pupils. For reasons seemingly best known to itself, the Edexcel exam board, which runs the marking of Sats for the Qualifications and Curriculum Development Agency (QCDA), changed the rules this year governing the system by which markers transfer the marks they award from individual scripts onto a summary document to be sent to the board. The new system makes this process more complicated, meaning that “double-counting” errors, in which pupils' marks in one section of the paper may be counted twice, appear to have crept in, in relation to some pupils' results. The board has been so concerned that it has asked marking “team leaders” to telephone markers to double-check their work. But perhaps a bigger furore may be about to break with the frankly quite puzzling news that the schools whose pupils completed the Sats this year will, from later this week, start getting marked scripts back without full information on how their charges have done. In the past, as most readers of this blog will probably need no telling, schools have received marked scripts back complete with a marksheet providing a level for each pupil on each section of the test, provided by the marker. They would, then, have full – if provisional – results for each pupil. Teachers would also use this information to calculate the overall results for the school. This will not happen this year, it seems. In 2010, for the first time, markers were not given any information on the level thresholds – the minimum number of marks needed to gain a level in each part of the test. They simply marked the work, sent each pupil's results in each section of the test on to Edexcel and, from this Wednesday, June 23rd, will be sending marked papers back to schools. But, because the markers this year did not have to assign a level for each child, (Edexcel now does this bit by computer), the scripts are going back to schools with the pupil's numerical marks for each section included, but with no indication of the overall level the child has achieved. Schools will not find out the overall provisional level of each child, and of the school itself, until results are published online on Tuesday July 6th. This is the same date on which the national “level threshold” information – the number of marks pupils needed to reach each level – is released, which is two weeks later than last year, when it came out on June 22nd. How will this go down in schools? Well, I am not a fortune teller or mind reader. But I would not mind betting that there will be a fair amount of unhappiness. If I were a child, I would be frustrated to learn from my teacher, for example, that I could be told the number of marks I had scored on a test, but I would have to wait perhaps 10 days to be told the overall grade or outcome. The QCDA told me that schools are being informed, via the June test “circular” which is now available on its website and includes a single sentence on this aspect, that mark sheets with level information are not being included with scripts this year. But I feel that, perhaps, more communication was needed setting out this fact and why the change is being introduced. I am still seeking more information from the agency as to the thinking behind it. Where these tests are concerned, it seems, controversy is never far away. This must be exasperating, even for the supporters of high-stakes testing. Postscript: Philip Blaker, director of test delivery operations for the Qualifications and Curriculum Development Agency, told me on Tuesday that the changes had come about to “reduce the administrative load on markers” and to increase consistency for schools. The fact that markers no longer have to work out pupils' levels had cut the demand on them, he said. And schools now had just one source of information on their pupils' levels, in the form of the online results site, rather than the two they had had in the past (one from the marked scripts; the other when their results were released by the marking agency). He said the change to this system had been put to a “reference group” of head teachers, local authorities and other stakeholders. “They do not see a problem with what we are proposing,” he said. “Fundamentally, we are trying to improve the quality of the processes and make sure that schools get the right result,” he added. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=326 Mon, 21 Jun 2010 13:02:52 GMT 20100621130252 Mon, 21 Jun 2010 13:02:52 GMT 20100621130252 Wed, 23 Jun 2010 11:35:59 GMT 2010062311113559 Last week, the new Department for Education published a notice on its website which included what looked like an uncompromising statement on the future of key stage 2 testing. The message was posted mainly to update teachers on the state of the primary curriculum, following the government's rejection of the Rose Review. Ministers wanted, it suggested, to ensure that the curriculum would promote a “relentless focus on the basics and give teachers more flexibility than the proposed new [Rose] primary curriculum offered”. I will leave aside, at this stage, the debate about this, or the worrying implications of using a word such as “relentless” to describe anything educational. My point is to bring to your attention a question posed at the end of the notice, and the response. The question was: “Will this also mean the end of Key Stage 2 tests?” The answer: “No, Key Stage 2 results are a robust and consistent source of information for parents at a crucial transition point for their child as they move on to secondary school. Tests at 11 mark the end of primary school for each pupil, and it is right that we have a consistent and externally validated view of individual pupils' progress at that time.” That looks to me like a more hardline position on testing than any of the parties were taking in the run-up to the general election. Last year, the former government's “expert group” on assessment recommended that ministers should “continue to invest in, strengthen and monitor the reliability of teacher assessment to judge whether a move away from externally marked national tests might be viable at a future date”. Labour accepted all of the recommendations, although it continued to defend key stage 2 testing and clearly this arguably ambivalent and non-committal position was not enough to persuade the NAHT and the NUT that ministers were serious about testing reform. The Liberal Democrat leader, Nick Clegg, meanwhile, told the TES in April that his party planned to ensure the tests were “scaled back”. He said of the Key Stage 2 tests: “These exams clearly don't have the confidence of many teachers. And that's not hugely surprising when you hear all the stories of 11-year-olds being stressed and [made] anxious over them, and when you think there are children spending more time practising exam techniques than learning creatively.” Most intriguing is the position of the education secretary himself. Only last summer, in an interview with Andrew Marr's Sunday morning BBC1 show, Michael Gove said that he planned to replace key stage 2 Sats with tests taken at the beginning of year seven. He said: “At the moment you have tests which are taken at the end of primary school… and one of the many concerns that people have is that that completely narrows teaching during the final year of primary school and all the focus is on drilling children just for those tests. “Now we believe that what we should do is move those tests to secondary school. And the reason why is that when we've talked to the best comprehensive schools, the one thing they tell us is that they don't completely trust the SATs tests and they run their own tests anyway to check the literacy level, the reading age of children when they arrive, and also to check their knowledge and overall competence. “And we thought, why is it the case that you need two sets of tests?” Last month, the coalition document published by the new Conservative/Liberal government said: “We will keep external assessment, but will review how Key Stage 2 tests operate in the future.” This week, curious as to why this review was not mentioned in the answer to the DfE question, I asked the Department for Education what it would amount to, and when it would happen. A DfE spokesman would only say that “external assessment at Key Stage 2 will be retained” and that “as the Coalition Agreement makes clear, we will review how Key Stage 2 tests operate in future and will provide further details in due course”. He also referred me to the Queen's Speech debate earlier this month, during which Mr Gove offered a “tribute” to Mr Balls. Mr Gove said: “[The KS2 tests] are a vital accountability measure, and [Mr Balls's] robust case for their continuation ensured a consensus across the House for more data, greater parental accountability and a relentless drive for improvement in early years education.” The questions for Mr Gove and Mr Clegg, surely, should be obvious. If they were so concerned about the “narrowing” and creativity-sapping impact of the high stakes testing regime on children's education before the election, why has their position changed towards seemingly unflinching enthusiasm now? And what is Mr Gove in particular going to do to help those children whose educational experience he clearly felt, as of last year, was being damaged by the current testing and accountability apparatus? NOTES: The DfE's question-and-answers on the curriculum can be viewed here. The coalition document, with the test reference on page 29, is here. A transcript of Mr Gove's interview with Andrew Marr is here. A report of Nick Clegg's interview with the TES is here. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=324 Thu, 17 Jun 2010 16:36:03 GMT 20100617163603 Thu, 17 Jun 2010 16:36:03 GMT 20100617163603 Thu, 17 Jun 2010 16:36:03 GMT 201006171643603 What will heads have made of the revelation, in Friday's TES, that schools adjudged to be “outstanding” by Ofsted might never have to experience an inspection again? If shrieks of joy may have been premature, I can imagine that some cautiously optimistic glints in the eye might have been glimpsed in staffrooms which emerged happy from their last encounter with Her Majesty's Inspectorate. However, I'm not sure that all schools will share that position. And I think that the public should be sceptical about these plans, as they should about the coalition's still-forming proposals on accountability more generally. At first glance, the new government's proposals to focus Ofsted inspection on “areas of failure” – as mentioned in the coalition's “programme for government” document – make sense. Michael Gove, education secretary, wants to direct inspectors' attention to schools which did not do well in their previous Ofsted report. “Outstanding” schools, then, need never be inspected again, he appeared, amazingly, to tell the TES, although closer reading reveals that he said there would be some safeguards. A school which was previously outstanding but appeared afterwards to be falling back on some measures – which the newspaper suggested could include data on achievement, pupil exclusions and teacher absences and turnover – could be re-inspected. Parents would also, presumably, be able to trigger a re-inspection by complaining about a school in number. As I say, in some ways this can be presented as making sense. First, in a time of ever-tightening budgets, it will enable resources to be focused on those schools where inspection – if it is seen to have a positive effect on bringing about change – could be expected to make the most difference. Second, it fits well with a narrative, which again at face value is right, which says an accountability system should only intrude into teachers' working lives where, without it, schools would be letting down their pupils. However, I think there are two big flaws with this plan, one reasonably obvious and the other perhaps more subtle. The more straightforward difficulty, I think, is that these proposals misunderstand the twin purposes of Ofsted inspections. They are supposed not just to put schools under pressure to improve, but also to inform parents as to the quality of each institution. However many reservations I have about the current Ofsted arrangements, without them parents considering sending their child to a school which is no longer inspected – having previously been graded outstanding - will have no detailed, rounded information at all about the quality of that institution. There will simply be no recent report on a school, with families simply told that once it was “outstanding”. I cannot see how that position squares with the new government's commitment to accentuating choice in the schools system. Without decent information, how is it possible to make an informed choice? The only other source of official information, of course, will be data published on the school. This brings me on to my second reservation. Mr Gove appears to want to place a lot of emphasis on statistics, arguing that “what we absolutely have to have is public, objective data about how schools are performing”. Information of the sort contained, then, in league tables, would necessarily be given more emphasis. If one accepts accountability as unavoidable in the modern schools system – as I think we must – there are broadly two choices: base it mainly on qualitative, rounded judgements of schools' qualities, as in inspection systems, or found it on statistical indicators, as in league tables. No system is perfect. But given the problems of the latter - including narrowing the emphasis in schools to what is measured; the seemingly unavoidable unfairness inherent in all ranking systems, none of which are “objective” and none of which can properly take into account factors outside of a school's control; and the difficulty that parents often have making sense of data - I think inspection systems, if they can be properly managed and made less data-driven and punitive than the current regime, have fewer downsides. Mr Gove's plans risk providing poorer quality information to parents: a set of figures rather than an inspection report written in English, for those schools which escape inspection. And while his suggestion of giving more freedom to successful schools from central bureaucracy may sound seductive to professionals, in reality they will still have to focus very carefully on the statistical indicators against which they will continue to be measured. Narrowing of teaching towards only those measures, and teaching the test, then, will continue, to be a serious risk. A data-driven education system, rather than one founded on human judgement, may well cut costs. But I don't think this would be money well-spent. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=313 Tue, 01 Jun 2010 09:55:23 GMT 20100601095523 Tue, 01 Jun 2010 09:55:23 GMT 20100601095523 Tue, 01 Jun 2010 11:37:26 GMT 2010060111113726 I was one of those children who loved school, generally enjoying most of the work which was thrown at me. But I have a confession: I think I would have struggled, as a youngster, with the task of writing creatively under exam conditions. It was, by the way, something I was never asked to do during my school years in the 1970s and 1980s. That, though, is the challenge which has faced our 11-year-olds for years now, as part of the key stage 2 English Sats. Amazingly, their performance in two sets of creative writing tasks on one day in May is meant to act as a definitive verdict on their progress in writing over the past four years. This, I think as an observer looking in at what now goes on in education, is crazy. During a recent online debate on Sats for the website Schoolgate, I mentioned last year's longer writing task, in which pupils were presented with an illustration of a pair of trainers and asked to write a report about them. This, I was far from the first to suggest, might not have been the most interesting challenge for a child. If they found themselves less than inspired, and were marked down as a result, I asked, how could it act as a fair reflection of their overall ability? I was challenged on this by someone who was opposed to the Sats boycott, to the effect that although that particular task may have been boring, this should not imply that the whole system was problematic. But this is, I think, to ignore the bigger picture. Last week's Sats writing tasks, as I understand it from a marker source, went down better than those of 2009, among those schools not taking part in the boycott. The 2010 longer writing task asked children to write a guide to looking after an imaginary animal called a “miptor”. The shorter assignment required them to write a recommendation setting out why someone they knew should receive an award. If many children felt enthusiastic when faced with these instructions, I am glad for them. But it does not obscure the fact that this looks to this observer to be a bizarre way to assess their progress. Hanging so much, for pupils and schools, on these two tests is odd, for several reasons. First, it does not reflect how creative writing works in the real world. Although some of us face deadlines when writing, we will not find several years' work judged on the basis of having to be spontaneous on demand during a pre-assigned timeframe of a few minutes. Second, any pupil can over- or under-perform on any test. Placing so much emphasis– including official judgements of progress over a whole key stage – on a test which would appear to be particularly vulnerable to concerns that pupils' responses could be affected greatly by the degree of their interest in the question, does not seem smart to me. Finally, I know markers have struggled with the interpretation of the mark scheme for the English tests in particular in recent years. And official studies have shown that many level judgements may be inaccurate. So, again, why are we basing so much on what could be unreliable results in these “snapshot” tests? And, if children do need to be judged on their performance in such tests, why is this not at least placed in some kind of wider context, assessing their capabilities over a longer period? At a Teachers TV debate on assessment reform recorded yesterday, for which I was one of the panellists, there seemed little argument that the current structure needed radical reform. When it works in this way, is that any surprise? My English marker contact indicates that marker training – which has been a source of frustration since at least the ETS debacle year of 2008 – has been improved this year. Markers were placed in small groups, rather than “all together in a hall”, and given more scope to discuss the mark scheme. Elements of the process for checking marker quality are taking place with computer support this year, another aspect which caused consternation during the meltdown two years ago. But my source says the systems put in place by Edexcel, the exam board which has replaced ETS in running the marking, are much better. She reports having received scripts from three quarters of her allocated schools to mark, with other examiners observing a similar pattern. Schools without scripts are, of course, likely to have boycotted the 2010 tests. She adds: “Writing topics have allowed children to write more creatively, and in my view demonstrate their skills to a greater extent than last year's topics. Marking levels seem rather tough, though, so I can see a similar pattern emerging as last: reading on the whole given higher levels than writing.” http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=305 Wed, 19 May 2010 10:25:10 GMT 20100519102510 Wed, 19 May 2010 10:25:10 GMT 20100519102510 Fri, 21 May 2010 10:28:10 GMT 2010052110102810 Less than a week to go, now, so which way will primary heads jump on whether or not to join the Sats boycott? I have always tried to stay outside the debate on the unions' tactics with regard to how to protest about the problems inherent in high-stakes testing. I have felt it was important to point out the evidence of the effects of hyper-accountability in schools, and the problems with the over-interpretation of test data, but that it was up to teachers to decide for themselves on the strategies needed to push for change. I still feel that, and certainly would not presume to urge any head to act in a particular way. That said, I do have sympathy with heads who feel that a boycott is the only way to push for action: the NAHT and other unions have been making the case for the government to take seriously the impact of its centralised testing and monitoring regime for more than a decade now, and concessions from ministers have been limited. The fact that there has never even been a proper official inquiry into this issue speaks volumes. The strength of feeling among heads at the NAHT's annual conference last weekend was notable. In the last couple of weeks, I have been watching the arguments thrown at heads who are considering joining the action, and I have to say, I find many of them weak. So what are they? The first says that children will lose out if they fail to take the 2010 Sats tests next week. Ed Balls made this point in his letter to school governors last week, arguing that pupils “should all be given an opportunity to demonstrate their achievements in tests that are set and marked properly” and that failure to administer the 2010 Sats would “disrupt children's learning”. I find this unconvincing. There is nothing to stop a head setting an alternative set of tests, such as previous years' Sats, and giving pupils levels based on their results in these assessments. Or they could decide not to set tests, and instead simply allow teachers to use their own judgements on pupils' levels based on their work over four years. Some have argued that pupils need to sit tests after all the build-up. I say we should leave that decision up to the head: taking part in the action does not close off the option of testing the children if the head believes this is the best thing for his or her pupils. In the end, a judgement on this comes down to who you think is best placed to take a decision in the pupils' interest: their head or the government . The second says that parents will lose out on the information that Sats provide on the progress of their children. But this overlooks the fact that teachers are able to make judgements for themselves. However schools provide the information – whether it is through a level generated through alternative tests, or through teacher assessment – parents will still find out. And, although externally marked tests are billed as more “objective”, in reality there have been problems with some Sats marking for years and any pupil can under- or over-perform on the day. The third says the timing is not right: schools should have been told early in the academic year of the boycott plans, because it was unfair to build children up for tests which might not happen. Yet the unions had no choice with that: industrial relations law means that ballots for action cannot take place more than a month before the action (in this case, the boycott of the tests themselves) begins. If they wanted to take a stand through industrial action, they had to do it in this way. And again, alternative tests can be used for pupils. The fourth says, again, the timing is not right: with an election always on the cards this spring, the unions should have held off for another year to wait and begin negotiations with a new government. Well, this comes down to tactics and it is a view but, after years of making their case, I can see why the NAHT and NUT have said they can't just put if off until 2011. The fifth says: the action might not be legal. Without wishing to tempt fate, the Government last month did hint at a legal challenge to the dispute, but it has yet to materialise. Mr Balls also suggested to governors that they “should not frustrate another competent person from administering the tests”, and that heads could be instructed to be absent from school “while another person administers the tests”. But separate advice from NEOST, the National Employers' Organisation for School Teachers, argues against sending the head home, and says it is not legally possible for governing bodies “to engage other agencies” to administer the tests. The NEOST letter adds that “it is unlikely that other teachers who are not members of the NAHT or NUT will be prepared to help to make arrangements to run the tests in schools where they are being boycotted”. The sixth argument, reportedly used by Mr Balls in his speech to the NAHT's annual conference over the weekend, was that Ofsted inspectors could penalise schools taking part in the boycott by marking them down. That looks like bullying to me, and would it not also be a dereliction of duty by Ofsted, whose purpose surely must be to pronounce on the quality of schools for parents, not to punish those taking part in legal industrial action? If the inspectorate really cannot pronounce on school quality without 2010 Sats data, this is not a good reflection on its processes. All this is easy for me to say, no doubt, not having to take a decision at the sharp end. I know that heads will weigh these arguments very carefully in making a judgement on what is in the best long-term interests of their pupils and of the profession. http://www.naht.org.uk/welcome/comment/blogs/warwick-mansells-blog/?blogpost=302 Wed, 05 May 2010 13:11:06 GMT 20100505131106 Wed, 05 May 2010 13:11:06 GMT 20100505131106 Wed, 05 May 2010 13:11:06 GMT 201005051311106 Ten days into the election campaign, and the economy has loomed large. All three main parties acknowledge the need to find a way of reducing the deficit, and all three say at least part of this trick can be effected through that magical phrase: “efficiency savings”. Assorted commentators have rightly pointed out that these much-used words can serve as camouflage: a way of avoiding spelling out to the electorate exactly where cuts to public services will fall. To be blunt, anyone can call for spending to be more “efficient”. But how is this to happen, and can savings simply be found by reducing wasteful activity, however that is defined? My experience in covering the fall-out to a previous round of “efficiency savings”, as they affected one organisation, the Qualifications and Curriculum Authority, suggest reasons for scepticism about what can be achieved in this field. In 2006, I wrote about how the QCA was under pressure from the Government to cut costs. Ruth Kelly, education secretary at the time, wrote to the authority asking it to cut £20 million from its £150 million annual overheads by 2007-8, following up on an efficiency drive across Whitehall overseen for Gordon Brown by Sir Peter Gershon, who is now advising the Conservatives on efficiency saving. I reported at the time that staff numbers at the QCA could be cut from 667 to 400. A separate review led to the recommendation that the QCA relocate from its offices near the Ritz hotel in London's Mayfair to the less glamorous, and less expensive, location of Coventry, a move which was completed this year. So were these targets achieved? Well, it became clear over the years that the anticipated budget cuts were proving hard to drive through. Published QCA accounts show its net spending actually continued to rise in the years 2005-8, from £133 million in 2004-5 to £170 million in 2007-8. Only in 2008-9 did it start to fall, dropping to £147 million. Even this figure was unusually low because of a £19.5 million payment the authority received from the American test marking contractor ETS, which had to pay it after its contract was terminated following the 2008 Sats marking fiasco. In 2004-5, the total number of permanent employees at the QCA was 512, with temporary staff numbering 27. By 2009, its permanent employee numbers were up to 572, while there were a further 151 temporary workers. One could argue that the QCA is a special case and that it had an extraordinary amount of work to contend with over this period, from the i