Leaders' guide to...Bew review - final report

The report recommends reducing the role of external testing, so why has it kept it at all?

We believe the evidence that external school-level accountability drives up pupils’ attainment and progress is compelling. We believe that providing evidence for external school-level accountability must remain a main purpose of any future approach to testing, assessment and accountability at the end of Key Stage 2,” says the review, adding that it heard evidence that this was particularly important for disadvantaged children.

Does the review promote teacher assessment?

Yes. Bew says summative teacher assessment provides useful information which the accountability system should take into account, and that this should continue to be published in the achievement and attainment tables.

“We believe the publication of teacher assessment results helps to show that they are of fundamental importance. Teachers know each child as an individual, and can provide an assessment across the full range of the curriculum and over a whole year. This should be acknowledged in the accountability system.”

The review notes that there is a wider gap between test and teacher assessment if the teacher assessment is published first, but says this does not mean either is wrong.

It recommends that schools should submit summative teacher assessments before getting test results, and these should be published, thus putting greater emphasis on teacher assessments. Bew also believes collecting those assessments earlier would encourage them to be passed to secondary schools earlier, to inform their planning for the new Year 7 earlier.

So why not rely entirely on teacher assessment?

The report says 83 per cent of those giving evidence online supported the use of teacher assessment, and given the support for this system, the panel spent “a considerable amount of time” considering it.

However, the panel heard a great deal of evidence about problems with this, including the risk that this would disproportionately affect ethnic minority pupils, and that there would be pressure to make generous assessments for some pupils, and that those on the borderline between levels might be given additional teaching at the expense of others.

“There are therefore clear risks that summative teacher assessment will not be sufficiently reliable in a technical sense – i.e. that judgements will not be made consistently by teachers across the country. Moreover, the research indicates that pupils who are most likely to under-perform would be most vulnerable to under-assessment.” The panel also heard evidence that moderating teacher judgements would not make such a system sufficiently reliable. Moreover, it did not believe that tests should be used to validate teacher judgements, as since they draw on different evidence bases it was perfectly legitimate for their outcomes to be different.

In addition, the panel heard concerns about workload.

It concluded: “We feel that a system entirely based on teacher summative assessment would not be sufficiently reliable for the purpose of providing school accountability data. While we believe ongoing and high quality assessment is crucial to ensuring pupils make good progress, we do not believe that schools should be held accountable through a system wholly based on moderated teacher assessment.”

What are the “broader accountability measures” being recommended by the review?

“Headline measures of attainment and progress will still remain important, but we want head teachers and teachers to be able to present a richer and more representative picture of their school each year,” says the review.

The report says half the online respondents said league tables should be removed. “We believe that a greater range of published information will reduce the likelihood that league tables will be created focusing on one indicator alone, which are unlikely to give a true comparison of schools’ performance. We believe a judgement about whether one school is more effective than another, based on just one measure, will simply not be credible when so much more information is available.”

The review is recommending the use of:

• Rolling averages: the review recommends rolling three year averages should be presented with the published statutory assessment data. This would reduce the variability in results from year to year and would particularly benefit smaller schools.

• Pupil mobility: the review recommends additional attainment and progress measures for pupils who have completed the whole of years 5 and 6 within the school, and that data on the “mobility” of pupils who joined in those years should be published.

• Publishing reading and writing results separately: but if a composite measure is necessary, the options of how it should be generated should be explored.

• Additional contextual information which the Government is publishing is welcomed

• Allowing absent pupils to take tests later: the review says the current two-day extension is not enough and suggests it should be possible to take tests up to a week later. The review recommends that the Government should run trials on this next year.

How might the new focus on progress work?

“While high expectations of attainment are of great importance to pupils, measuring the progress pupils make better reflects the school’s contribution to their education. Simply reporting pupils’ attainment fails to acknowledge the amount of effort a school has put in and the difference in effort different schools need to make to get pupils to the same standards of attainment.”

The review says the new system should focus on both attainment and progress as headline published measures, and any overall judgment of the school by the Government, local authorities or Ofsted should give at least as much weighting to progress as attainment. “Combined, the two measures will celebrate pupils’ achievement and schools’ contribution to it.”

The emphasis on progress should apply at individual pupil level, says the report, suggesting that the progress of those with low prior attainment should be clearly identified within the accountability system. It says this is particularly important as the current pupil progress measure doesn’t encourage schools to focus specifically on pupils who have entered the Key Stage at a lower level than that which is expected.

If progress becomes more important, then don’t KS1 results become more important as well?

The review heard feedback from heads, particularly of junior schools, that teacher assessment judgements in infant schools might be “inflated,” which then caused problems in showing progress at KS2. The review also heard evidence that all-through primary schools might under-assess at KS1 to improve progress rates at KS2.

The review suggest the moderation process at KS1 should be developed to be “more consistently rigorous”.

“We suggest moderation at Key Stage 1 is better targeted so that schools where attainment and progress at Key Stages 1 and 2 are inconsistent are prioritised and moderated more frequently. We realise that many local authorities already target their moderation very carefully and we believe this should consistently be the case and made a formal requirement.”

The review also suggests that there is “potential” in encouraging cross-phase moderation of Year 6 pupils’ work with teachers in year 7.

What about pupils with special needs?

The review says it shares heads’ concerns that the current system may discourage schools from admitting these children in case it affects results, but is aware of potential “perverse incentives” associated with removing them from the measures. The panel conclude that this issue is wider than the statutory assessment system and says it does not want to make any recommendations about pupils with SEN as they should be considered “alongside a wider package of changes.”

Does Bew say anything about higher-achieving pupils?

The review notes that children achieving Level 3 at the end of KS1 can then only make the expected level of progress during KS2. It welcomes the Government’s decision to make optional Level 6 tests available this year but notes concerns about the ability of primary schools to cover enough of the KS3 curriculum. The review recommends that the Level 6 tests should continue to be available and that if the National Curriculum review makes changes to the current system of levels, there should be new arrangements to ensure the most able pupils are challenged.

What did Bew think about the current KS2 tests?

The review says the evidence it heard on the tests themselves was largely positive.  It heard evidence about the degree of measurement error, but said that for school accountability, the test outcomes were a sufficiently reliable indicator of the overall performance of its pupils, particularly if looked at over a three year period. “In practice, what is tested is limited to what can be most consistently marked. This means that important areas of the curriculum may be marginalised or excluded from end of Key Stage 2 testing – in particular use and application of mathematics, speaking and listening and the more practical aspects of science. If these topics do not appear in the ‘high stakes’ testing regime, schools may choose to give them low priority in their teaching. We therefore need to ensure that any future tests strike the right balance between reliability and validity.”

The review rejected suggestions that national standards over time would better be measured by purposely-designed sample testing.

What does Bew say about… the reading test?

The panel says the current test was “reasonably well supported” by respondents although there were concerns about weaker pupils being disadvantaged. The panel makes several recommendations:

• That reading should continue to be an externally-marked test

• That changes should be incorporated into any new reading tests, adjusting the balance of text and reading time, putting the contents into a clear order of difficulty, and ensuring the texts are accessible to all pupils

• The number of written responses required should be kept under review so that it is a test of reading rather than writing

• That the government should consider the skills which should be assessed by the reading test, and these should be brought out more clearly in the design of future tests.

Finally, it says: “We feel there is a risk that being forced to over-interpret texts may take pupils away from reading for pleasure and could potentially restrict their love of reading. Pupils at Key Stage 2 should concentrate on reading fluently and regularly; and we believe it is essential that they enjoy their reading and read widely and often with texts becoming increasingly challenging.”

What does Bew say about… the writing test?

The panel heard a great deal of criticism of this during its online call for evidence, with 43 per cent rating it “inadequate” and a further 33 per cent “not very effective”.

The report says there are “significant issues” with the current tests, including the unpredictability of the writing genres, and inconsistency and subjectivity of marking. “Respondents feel they do not reflect classroom practice, whereby children take time with their writing and put effort into spelling, punctuation, grammar, vocabulary and handwriting. Others observed that many children produce their best work as part of a structured lesson following an inspired discussion or school trip, while it can be difficult to write creatively under pressured test conditions.”

The panels says there are some elements of writing where there are clearly right and wrong answers and lead themselves to externally marked testing, and some other countries do this. “These are essential skills and we recommend that externally-marked tests of spelling, punctuation, grammar and vocabulary should be developed. We suggest it may be appropriate for handwriting to be assessed in this externally-marked test too.”

However, the report says there is much more to writing than this, and it expresses concerns about the current test, including its marking and the limited choice of genre.

Although the panel feels it can be legitimate to assess writing composition through a test, it shares many of the concerns it heard and feels that in comparison with other subjects it is less valid to measure pupils’ attainment on the basis of one test paper in May. “We recommend that writing composition should be subject to summative teacher assessment only. This will encourage a broad range of writing over the course of Year 6, while avoiding the perverse incentives of the current system. It would allow Year 6 pupils to demonstrate what they can do across a range of genres, and would remove the inevitable disagreements about the marking of individual pieces of writing.”

It believes this will help develop teachers’ creativity. “We want pupils to be taught a wide range of writing genres and to be encouraged to produce their best work each time they write rather than having strict time constraints. This is more likely to lead to a ‘can do’ attitude towards writing and greater enjoyment than is the case if teaching across the year is based on a build up towards the current test.”

However, it recommends that the teacher assessments should be subject to external moderation and “if the moderator has concerns over the accuracy or reliability of the sampled teacher assessment judgements, they should be able to scrutinise additional evidence and, if they consider it appropriate, require the school to change the reported levels.”

Writing composition should always form the greater part of overall writing statutory assessment, says the review.

How will schools be expected to support their judgements on pupils’ writing?

The report says it would not be helpful or appropriate for schools to create portfolios of work or specially prepared examples. Moderators should review exercise books and other examples of marked written work for a range of purposes taken from the whole teaching year, including from other subjects. “If moderators review marked work produced in the course of everyday teaching in Year 6 (together with the teacher’s associated comments) they will get a strong sense both of the pupil’s attainment and the teacher’s assessment skill, without creating any additional workload. We feel it may also be helpful for moderators to have the option of meeting the pupils whose work they have reviewed.”

Why does Bew recommend externally-marked tests for maths?

The evidence suggests the test is widely respected, says the panel. The most frequent criticism of the maths test was the amount of reading and vocabulary required of pupils.

Does the panel make any further suggestions about maths?

Yes, it says the amount of reading in the maths test should be kept under review as test are developed, to ensure weaker readers are not unfairly disadvantaged. It also says teacher assessments on the maths attainment targets of use and application; number; shape, space and measure and handling data should be reported at pupil level to parents and secondary schools.

Did the panel talk about science testing?

It heard a range of evidence both supporting and complaining about the decision to end externally-marked testing of science, but agrees with the principle that it is difficult to measure scientific enquiry through an externally-marked test, and that to test might distort what was taught. It recommends retaining summative teacher assessment, and that this should be reported to parents and secondary schools. There should also be sample testing as now. It recommends that the current arrangements should be revisited once the new science curriculum has been created.

What about the timing of the tests?

Evidence on this was mixed, and there were particular concerns about the suggestion that the tests might be delayed until the start of Year 7. “We believe this is not the best solution to the problems with the current system,” says the report, recommending no change. 

What purpose does the review think statutory assessment should serve?

Bew notes that for many of those who gave evidence to the review, their concerns were not about the tests themselves, but the way they are used. Asked where change was most needed, half of the respondents said league tables should be abolished but only a third said the KS2 tests should be abolished.

The review identifies three key uses for statutory assessment in addition to the legal definition that it is to ascertain what pupils have achieved in relation to the attainment targets for that stage”. These are: 

• holding schools accountable for the attainment and progress made by their pupils and groups of pupils;
• informing parents and secondary schools about the performance of individual pupils;
• enabling benchmarking between schools; as well as monitoring performance locally and nationally

It also suggests that further work be done on whether using confidence intervals, which demonstrate the impact of measurement error on results, would promote greater understanding of the outcomes of the tests.

How much did the panel accept evidence of problems for schools caused by the current system?

The review says that despite fears about schools focusing on tests for the whole of Year 6, “evidence suggests that this need not be the case. Good academic outcomes do not have to come at the expense of narrowing the curriculum. There is evidence to suggest that very good primary schools are successful both in promoting pupils’ academic progress… and in ensuring high quality educational experiences across the curriculum.”

The review notes that changes to the accountability system are already underway, and says it believes publishing data and being transparent about performance is the right approach. “However, we want to ensure that, while each school is accountable for its performance, a broader range of indicators is available, including contextual information. We believe that a broader range of published data would help ensure schools are held accountable in a fair way and would allow parents and others to focus on the measures and areas that are of most interest to them. We accept that, however much data is published, there will always be a temptation rank schools by one single indicator, but we believe it would not be credible or legitimate to suggest a school is more effective than another school based on one measure alone; such conclusions should always be based on a range of measures.”

How soon could the recommended changes be made, if the Government agreed to them?

“It may be worth noting at this stage that we are aware some of our recommendations cannot be implemented straightaway due to the long lead-in times associated with the test development process. We are mindful that the development of high quality National Curriculum Tests requires a stringent test development process. We therefore recognise that the test development timescale places unavoidable practical constraints on the speed with which changes can be made to National Curriculum Tests.”

Are any changes recommended in the way pupil results are given to parents or secondary schools?

“We believe the pupil-level information provided to parents should be improved, and that this would be useful to secondary schools as well as parents,” says the review, adding that it does not believe there is sufficiently detailed information given at present for specific areas where improvement is required to be identified. It notes the frustration of primary staff that secondary schools frequently make little use of their assessment data.

“We believe the pupil-level information provided to parents should be improved, and that this would be useful to secondary schools as well as parents.”

Does the review look at national curriculum levels?

“We recommend that, as part of the review of the National Curriculum, consideration is given to creating a more appropriate ‘vertical scale’ with which to measure progress,” it says.

What about the wider use of computers in testing or marking?

The review suggests on-screen marking should be considered for other KS2 tests and the government should learn from the evidence of science sample tests and plan further trials with the aim of moving towards full on-screen marking.

It also recommends exploration and piloting of computer-administered testing.

Did the review say anything about Ofsted’s use of test data?

Although it heard evidence suggesting that inspectors rely too heavily on test data, the panel concluded that results are used to “inform rather than determine” the overall grading of a school. “We recognise that Ofsted currently takes account of more that just test data in forming its judgments. We welcome the proposal to place greater emphasis on pupils’ progress in the inspection process.”

What was the remit of the Bew review?

The parameters of the review were school autonomy accompanied by robust accountability – Michael Gove reminded the panel that the OECD concludes that external accountability was a “key driver of improvement in education,” and “particularly important for the least advantaged.”

Lord Bew and his panel, which included NAHT nominees Helen and Tim Sherriff, were asked to address the following key issues:

• How to ensure primary assessment could improve standards and progress, and help narrow gaps
• To best ensure schools were properly and fairly accountable
• To avoid perverse incentives and over rehearsal
• To give parents good quality information on their children’s progress and schools’ success
• To ensure the performance information appropriately while avoiding the risk of “crude and narrow judgements”
• To ensure the tests are rigorous, as valid and reliable as possible within an overall system of assessment
• To best ensure the assessment system can be used for international comparatives
• To make it as simple and cost effective as possible.

How did they gather the information?

The panel started with a 12-week call for evidence, which attracted around 4,000 online submissions. They heard evidence from 50 stakeholders and read many written submissions.