What does it say about the way in which accountability measures are driving the system?
"We believe that there is a genuine question as to what extent
reform of the exam system and strengthened regulation would solve the problems identified in our report, without significant changes to the accountability system that drives much behaviour in schools. This is linked to the multiple purposes served by A levels and in particular GCSEs, namely certifying achievement, ranking students and holding teachers, schools and government to account, as well as preparing young people for the next stage of education or employment."
In a strongly-worded recommendation, the report says: "The Government should not underestimate the extent to which the accountability system incentivises schools to act in certain ways with regard to exams. Sometimes these may be in students’ interests; sometimes, however, they are not. We recommend that the Government look afresh at current accountability measures, with a view to reducing the dominant influence of the measure of 5 GCSE A*–C or equivalent including English and mathematics and to increasing the credit given to schools for the progress made by all children across the ability range."
Why doesn't it think improvements can be made to the current system?
"We believe that it is unrealistic to expect significant collaboration between exam boards on syllabus development in the current system, as exam boards compete vigorously at this stage for market share on the basis of syllabus features," says the report, quoting Jon Coles, former director general for education standards at the DfE, who had told a conference that exam boards should stop marketing qualifications as “accessible”.
"Mr Coles suggested that the tactic has promoted a 'culture in which it is seen to be acceptable to say to schools, do this [exam] because it is easier [...] even if you do not use those words, that is what schools are hearing”.Mr Coles also accused the exam boards of developing exams that 'barely meet' Ofqual’s minimum requirements and called for the boards to have the 'moral courage' to ensure their qualifications have the depth and quality to exceed Ofqual’s minimum requirements.
"Beyond Jon Coles’ 'moral courage', no-one was able to help us identify incentives that exist in the current system for exam boards to exceed the minimum requirements," says the report.
It finds that there are strong incentives in the current system leading to "downward competition on content standards" and no incentives to go in a different direction.
The report adds: "Even if exam boards do not compete overtly on standards, we agree with a TES forum post that “there is every incentive for the exam boards to ensure their exams are no harder than anyone else’s”.
The committee goes on to add that the accreditation of examinations by Ofqual may be made more difficult by "a lack of assessment expertise".
There has been a lot of talk about the desirability of a single exam board -- why didn't the MPs think this was a good idea?
"We feel that the cost, heightened risk and disruption likely to be
generated by a move to a single board outweigh the potential benefits. Evidence suggests that some key issues, such as standards over time and across subjects, would remain, while other problems, such as a lack of incentive to innovate, the risk of higher fees and of reduced quality of service may be generated," says the report.
It says this would either involve the creation of a new official body, at arms-length from the Government, with substantial cost implications and the need for primary legislation, or the expansion of a current provider which the not-for profit boards might not be able to do.
"The prospect of a single, for-profit, provider of GCSEs and A levels may be of limited appeal, given that many have already expressed misgivings about exams being run on a for-profit basis," says the report.
The committee says that the creation of a single board for GCSEs in England would not by itself prevent schools using other examinations, such as those offered by the Welsh board, or iGCSEs.
They added: "There may also be the potential for increased political interference, as well as the issue of whether to limit schools’ choice of exams to those offered by the single board."
What about the idea of franchising subjects to exam boards?
The committee thought this would allow a concentration of expertise whilst removing competition on syllabus development, but concluded this was a “one way street” with significant down sides and long-term implications for the exam system.
"Should the option of franchising be pursued, the Government needs to consider the legal implications of a franchised system very carefully, including the need to constrain schools’ choices of syllabuses to those provided by the franchisee and to prevent other exam boards from marketing alternative “equivalent” qualifications. Any major structural change to the exam system needs to be managed with extreme care in order to minimise the disruption caused, bearing in mind that the greatest threat to standards is at moments of change and that the bigger the change, the bigger the risk."
So does the committee think exam board competition is a good thing on balance?
"We considered which exam board functions benefit most from
competition, splitting these functions into three broad areas: syllabus development, the setting and marking of exams and associated administration and finally exam board support. We can see no benefit to competition on syllabus content. By contrast, properly regulated, we believe that competition on the other two functions generates incentives to drive up quality and offer value for money to schools and colleges."
Why are national exam syllabuses the MPs' preferred option?
"National syllabuses would be developed by exam boards in conjunction with learned bodies and employer organisations and, at A level, higher education. They would be regarded as a national resource that could be examined by any of the English exam boards. They would remove the incentive for exam boards to compete on content and the associated downward pressure on standards, but would retain the benefits of competition on quality and the incentive for exam boards to innovate.
"There could be more than one national syllabus in a subject, to provide some choice to schools. We believe that national syllabuses, coupled with a stronger Ofqual [which would be responsible for accreditation] and the introduction of national subject committees, should help to maximise the benefits of having multiple exam boards while minimising the downsides and avoiding the cost, risk and disruption involved in major structural reform."
Is that just at GCSE?
No. The committee would like the Government to pilot the idea in one "large entry subject" as part of the forthcoming A Level reforms. Ofqual should review the effectiveness of this "with a view to extending the approach across GCSE and A Levels if appropriate."
How would national syllabuses work?
"Exam boards would be required to have at least one syllabus accredited (in any subject) in order to offer question papers linked to the national syllabuses developed by another exam board. Ofqual would need to monitor question papers, but its grading standards work would take account of, and adjust for, any differences in demand between question papers.
"An alternative approach to the creation of national syllabuses, which would then be examined by the existing boards in competition, would be to commission the syllabuses directly. Successful bidders in this process should be required to be consortia of examining boards, learned bodies, higher education and/or employers."
It sounds as though Ofqual would have an even bigger role to play than it does now.
Yes. The report comes back to this point again and again, noting that it is a relatively new organisation. "It is clear that a stronger Ofqual is needed, however the system is organised. There are encouraging signs that Ofqual is becoming more rigorous in its regulation of standards, in particular of grading standards. The effect of this is twofold: first it helps to control grade inflation and second it provides reassurance that the exam boards are not competing on grading standards.
"There is still, however, more to be done to improve Ofqual’s strength and effectiveness as a regulator. Ofqual needs to ensure it has sufficient assessment expertise, including on its Board, and to demonstrate that the methodologies it uses to regulate standards and accredit qualifications are robust and that it draws on appropriate respected subject and assessment expertise. Ofqual also needs to monitor changes in market share between the exam boards more closely, in order to account for shifts at individual qualification level and to establish whether there is any link to standards," says the report.
In addition, Ofqual would need to pay "close attention" to pricing.
Has Ofqual got a clear enough remit?
The report hints that it could be improved. "The Government needs to give a clear direction to Ofqual about its priorities on standards in GCSEs and A levels, and whether this is to maintain standards over time, to benchmark against comparable qualifications in other countries or to “toughen” exams. Both the Government and Ofqual need to be explicit about any recalibration of exam standards and of the consequences for young people."
The MPs are also concerned about the amendment to Ofqual's qualification standards objective which now requires it to benchmark English exams against those in other countries, and says the government should make its priorities clear.
In addition, the MPs are calling for a debate on the importance of standards comparability between the home nations, with a Ministerial conference to decide on any action.
Do MPs think Ofqual has enough expertise to do its job properly?
Describing Ofqual as "pivotal," the report says: "There were recurring calls ... for a stronger Ofqual, whatever organisational model is adopted."
It recommends that Ofqual "seek to build its assessment expertise and finds the resources to do so. We further recommend that Ofqual appoint an assessment expert to its board as soon as possible."
Is the committee happy with the way in which Ofqual is regulating the exam market?
"Overall the evidence we received suggests that the exam system needs a stronger regulator, whatever organisational model is adopted. [Chief Regulator] Glenys Stacey told us that 'historically, the sector has been under-regulated or not firmly regulated' and that she has regarded her work as 'turning a ship'.
"We believe that there is a strong argument in favour of allowing time for a strengthened Ofqual to take effect, as the changes it is making will take time to settle and bear fruit. But Ofqual must demonstrate that it is collecting the right sort of qualitative and quantitative evidence and using robust methodology to regulate effectively. Details of the evidence used by Ofqual in the regulation of standards, and any specific findings and regulatory action on standards, should be set out clearly in annexes to Ofqual’s annual report to Parliament. Ofqual must continue to show that it is prepared to take vigorous action when needed, in order to help increase public confidence in the exam system."
So who would be on the national subject committees? How would they work?
The report recommends that they be convened by Ofqual in "large entry" GCSE and A Level subjects "drawing their membership from learned societies, subject associations, higher education and employers. Such committees should include in their remit syllabus development and accreditation, as well as on-going monitoring of question papers and mark schemes, and oversight of comparable qualifications offered in the devolved nations."
The committee says such committees would help Ofqual to counter criticisms about its lack of in-house subject expertise.
Is the committee happy with the current plan to get universities to determine A Level content?
The report says there is plenty of evidence to support the government's intentions, but quotes Glenys Stacey's point that "the practicalities of that are all in the detail" and “there is a danger in listening to a small cadre of voices in higher education; we need to look at it in the round”.
"Ofqual has also emphasized that the full commitment of and support from
universities is essential for the new arrangements to work," says the report. It recommends that the government and Ofqual seek to increase the involvement of "learned bodies" as well as universities in A Level content, whilst allowing exam boards to control question papers and exam design.
"The Government and Ofqual must also ensure that the whole of
the university sector is consulted on the proposed A level reforms, as well as schools, colleges, learned bodies and employers. We recommend that Ofqual involve national subject committees in the development of criteria for and accreditation of new A levels," it says.
What are the perceived problems with the current system?
The MPs note a "cumulative impression" of "relatively low public confidence" in the exam system as well as serious concerns about particular aspects." In the long term, this risks compromising the credibility of the system and devaluing the qualifications achieved by young people."
It notes particular worries including grade inflation, the role and effectiveness of Ofsted, the cost of exams, problems with conflicts of interest including textbook production, the impact of accountability (including teaching to the test) and the number of exams teenagers are taking.
"We believe that changes are needed in order to increase confidence in the system and maintain its credibility. The key question is whether improvements are best achieved through fundamental administrative reform or by improving the current system."
Does it find evidence of grade inflation?
Committee advisor Professor Alan Smithers has observed that the A Level pass rate has increased from 68.2 per cent in 1982 to 97.8 per cent in 2011, with A-grades rising from under 10 per cent in 1987 to 27 per cent in 2010.
"More young people may be doing well because they are better prepared to pass their GCSE and A level exams. However, we feel that there is sufficient evidence from a variety of sources, such as universities, employers, England’s flat PISA profile and research by Durham University..., to cast doubt over whether GCSEs and A levels indicate improved preparation of young people for further education and employment and whether higher grades reflect genuine improvements in their underlying knowledge, skills and
understanding," says the report.
The committee says Ofqual should continue to investigate grading issues and need to be able to account for the "creep in grading standards".
"We suggest that occasional explicit recalibration of grading standards may be required and is preferable to slow creep downwards or upwards."
What conclusions did the committee reach on exam-specific textbooks?
While appreciating that there were strong arguments for senior examiners writing textbooks, MPs were concerned about a potential conflict of interest. They recommend that Ofqual make clear the expected role of examiners in textbook authorship and consider restricting exclusive endorsement arrangements between exam boards and publishers.
The committee was also concerned that describing a book as "all you need for your course" might encourage a narrow approach to teaching and learning, and quoted evidence from the Wellcome Trust that textbooks had increasingly become examination guides instead of providing broad and deep knowledge.
Authors told the committee they were under pressure from publishers to limit coverage to what was in the syllabus.
Tim Oates, of Cambridge Assessment, told the committee that the narrow "guide to the examination" was produced because this is precisely what an accountability-trammelled profession asks for”.
The committee said it agreed that the criticisms of endorsed textbooks "must
be seen in the context of the accountability system and the pressures on teachers and schools to focus on exam preparation, in order to achieve the best possible results in exams."
This was a classic case of the assessment tail wagging the curriculum dog, said the committee, recommending that in future A Level textbooks be endorsed by the universities involved in developing a syllabus rather than the exam board. At GCSE, "a possible way forward" might be for learned bodies endorsing textbooks rather than exam boards.
What about the accuracy of marking?
The committee notes that a drop in the number of teachers reporting that their GCSE students achieved the right grade, and a doubling of those who thought they had been given the wrong grade. Ofqual has acknowledged that this is an area that is "significantly undermining confidence" and is reviewing current arrangements, says the report.
The committee heard evidence that the issue was not so much about marking processes, but the way in which schools were treated by exam boards when querying marking or grades.
The report recommends that Ofqual must investigate allegations of improper conduct by exam boards to ensure candidates are awarded the grades they deserves, and that exam boards should continue to monitor the effectiveness of online standardisation.
Did the committee look at early and multiple entry?
Yes. It describes this as a good illustration of the way in which interaction between the exam and accountability systems might not always be in the best interests of young people.
One maths body, ACME, told the committee that early entry to GCSE mathematics was particularly common in National Challenge schools, those often under most pressure to improve results. However, ACME concluded that early entry had a "negative effect" on most students' maths education, hindering their progression to a wide range of subjects post-16 and in higher education. "It is an unfortunate example of how league tables and National
Challenge status can encourage school leaders to put the interests of the school above those of the students themselves," said the report, recommending that the Government should ask Ofqual to identify the extent of multiple entry and advise on whether it should be limited.
Why does the committee think schools switch exam boards?
They say they found it "difficult" to get beyond anecdotal explanations for this, or to say whether it had something to do with standards. "The National
Association of Headteachers suggested to us that schools do not tend to change syllabuses on a frequent basis, but noted that to the best of its knowledge 'there is not a body of evidence that seeks to explain how schools arrive at the judgement that a particular specification is right for a particular cohort of pupils'."
The committee also heard evidence that schools switched when they had had "a rotten summer" and were dissatisfied with the marking/grades and exam board response.
The report says Ofqual needs to be able to respond "publicly and convincingly" to questions about changes in market share and to confirm whether they say something about standards, particularly for large entry subjects.
