Over the course of the summer term, NAHT was in regular contact with STA to raise any member issues, concerns or need for clarification. STA expressed their appreciation that we had this positive and constructive relationship.
We formally met with STA in early August to discuss all issues which had been raised, consider how things might be improved in future and what action, if any, needed to be taken.
We also discussed the teacher assessment moderator training and standardisation process which was introduced last year to improve consistency and confidence in the system. STA are reviewing the process in terms of lessons learnt and explained some of the changes which might be implemented for 2017/18.
1. KS1 scaled score conversion tables were not available to schools on time.
This was a cross-government publishing issue outside of the control of STA. NAHT contacted STA and were notified that these were emailed to schools late afternoon and made available online the following day. NAHT communicated this to members so they were kept informed but asked that in future, STA notify us as early as possible if an issue occurs.
STA learnt lessons from this unprecedented occurrence and have reviewed their processes to ensure there would always be an alternative source of the information, such as publishing time-sensitive documents on NCA tools and confirmed NAHT will be contacted if there was a similar issue in future.
2. Concerns over the quality of the marking reported and experienced by members and the nature of the marker training guidance.
There were examples of marking errors in reading and maths as well as GPS; the pupil had clearly written the correct answer and this had not been awarded the mark. These errors may not be raised in reviews of marking and more must be done to minimise such avoidable mistakes.
The STA mark scheme is sensible in its approach to the marking of punctuation and NAHT would not disagree with this. The problem seems to be that in the marker training, markers are given additional information/instructions about punctuation whereby imperfect formation, such as the dot of the semi colon being above the level of the letters of the word, are being marked as incorrect. The hidden guidance destroys transparency. This approach would also seem to go against the spirit of the words in the STA mark scheme. Such marker guidance provided in addition to the mark scheme must be reviewed.
STA are aware of the concerns regarding marking accuracy and suggested that these could be corrected with a review. STA are also fully aware of the issues with the marker guidance and will continue to manage and monitor marking quality. In addition, theyare in discussions with their provider regarding the guidance for 2018 SATs and will be working closely with them on this.
3. One or two mark errors will not likely be raised in reviews of marking which can only be successfully submitted if they result in a change of three raw score marks.
Reviews can only be successful where there would be three raw score marks difference (or where a child would meet/not meet the expected standard). In many of the cases where schools believe there is a marking discrepancy it is of only one or two marks and so a review would not be considered successful and the school would be charged for the review (although the pupil(s) results would be amended if any items had been marked incorrectly, even if this resulted in less than a 3 mark change to the raw score).
STA highlighted that reviews can be submitted for a change of less than three marks and guidance would be updated to make this clear. However, they recognise our concerns that such a review would not be "successful" unless it resulted in a change to the pupil meeting or not meeting the expected standard and that the school would be charged for this. STA agreed to look into this issue further and will keep us updated with progress on this.
4. Attainment data being released at 00:01 on the publication day.
Many NAHT members raised concerns over this, highlighting the wellbeing issue and additional pressure placed on school leaders. Data should be released in accordance with guidance on other official data and so be within normal working hours.
STA have agreed to publish the KS2 results in the morning on return of results day with the exact time to be confirmed.
5. Use of rubbers by pupils as a trigger for maladministration investigation.
The advice team has been talking to members who feel that overuse of rubbers has been used as a criterion to investigate schools for maladministration; this has led to results for a cohort being annulled.
STA stated that overuse of rubbers is not a trigger for an investigation of maladministration. This is not something which would be able to be picked up in script scanning. They confirmed that investigations for maladministration are usually triggered by an LA visit or report from a parent, teacher, another school, etc.
6. Requirement to submit data to LA in advance of moderation visit.
NAHT notified STA of specific LAs we were made aware of. STA intervened and these LAs were reminded of the guidance which states that LAs cannot require this.
STA accept this remains an issue in some LAs. They are aware of local agreements in place between LAs and schools where certain practices are agreed, however there are issues with these in terms of when the agreement was made and how regularly it is reviewed. They intend to make more explicit reference to local agreements in the moderation guidance and suggest that these should be revised annually.
7. LA saying that 24 hour notice of moderation will be given.
Moderation guidance clearly states 48 hours.
STA will look to reinforce this in guidance and moderator training, but we agreed that we would continue to deal with LAs individually as we were made aware by members.
8. Interpretation of "large cohort" in terms of number of moderators and potential for initial sample size to increase with increased number of moderators.
Two form entry was automatically considered large cohort in some LAs. STA recognise that moderation guidance isn't explicit with regards to what would be considered a "large cohort" and clarified that a two form entry could be considered a large cohort, but not in all cases. They are supportive of a sensible interpretation and are wary of making the guidance too prescriptive. We agreed that they would develop various examples to use in the moderator training.
The guidance says that the moderators should initially sample a minimum of 15% of children. There is a concern from members that more moderators could select an initial sample much larger than this. STA needs to be notified if this is happening, so they can talk to the LAs concerned as this practice is outside the guidance. The initial sample is 15% regardless of the number of moderators. The sample is only expanded if there are issues with the standards awarded.
9. LA charging for review of additional evidence in cases of remoderation.
NAHT contacted STA as we believe this is completely unacceptable.
STA responded: "We would not expect LAs to have placed additional charges on schools for re-moderation of teacher assessment judgements. Our expectation is that the costs of these activities should have been factored in by the LAs at the beginning of the year as part of the expenditure that they retain centrally from maintained schools. Further information on centrally retaining funding can be found in the Schools Revenue Funding 2017-18: Operational Guide. Further information on the provision of services and facilities by LAs can be found in Schemes for financing schools (please see sections 8.1-8.3)."
STA will be reiterating to LAs that the funding they receive covers all elements of moderation including remoderation.
10. Schools told by LA that they couldn't change data for ANY pupils after moderation whether they were in the sample or not.
STA clarified the position on this at the time: Children can definitely be changed after a moderation visit, but the school and the LA have to agree who they are at the time of the visit – these should be identified on the note of visit. If all the school's judgements have all been correct then remoderation wouldn't be required. If there were issues with any of the judgements then any pupils moving up would need to be remoderated. If there were pupils outside of the sample in the same circumstances then the head would be responsible for ensuring that their TA judgements were correct.
NAHT highlighted that this raised another issue with the understanding of what should be recorded on the note of visit and the need for moderators to lead this process.
STA will look to make this clearer in the moderation guidance for LAs. The reason for making a note of particular children OR percentages of children where there are larger numbers, is to aid the data checking by the LA.
11. LAs asking for end of key stage data to be submitted a week earlier than the DFE required date.
Where data is sent via the LA, LAs are asking for it early so that they can do their checks but where they are asking for it a full week earlier than the DfE deadline, this could be unfair to schools. The moderation window runs to the 29th June and some LAs are effectively shortening the moderation window by a week.
STA accepted that some LAs requesting data a full week before the deadline potentially created unfairness and inconsistency. We agreed it would be reasonable that LAs should not ask for data to be submitted to them before the Monday lunchtime of the week of the STA Thursday deadline. STA will include wording to this effect in updated guidance for 2018 and reinforce this through moderator training.
Difference between moderated and non-moderated schools.
Members continue to raise concerns regarding the variation in attainment between moderated and non-moderated schools at LA level.
STA confirmed that they were aware of these concerns and are currently analysing data. STA will keep us updated with results and work with us to give some messages to members about this.