Most years, the national drama of A Level and GCSE results days in England plays out in two distinct but related acts one week apart, focusing on the performance of the education system and the young people navigating their way through it. We get to share the joy and disappointment of candidates and to agonise about ‘standards’ based on upward or downward trends. There is often some discussion of education inequalities; by ethnicity, by class, by school or college type and between male and female students. ‘Social mobility’ and widening participation in higher education get some attention. And then things move on.

This year, we experienced a wild, turbulent, continuous national psycho-drama with a constantly changing plot and contradictory narratives. It was a full-blown crisis with elements of both tragedy and farce, and the dazed audience hardly had time to keep up with the various U-turns and reversals of fortune.

It’s not the most life-threatening of the dramas unfolding around us, but for the students most affected and for the English education system overall, much damage has been done. We now know the outcome, and although it’s too soon to predict all the future ramifications, the consequences have already been serious and many lessons will have to be learned.

We may not yet be ready to write even the first draft of history for this issue, but it is useful to start jotting down some notes towards an outline of that draft. Education is a complex system with many separate interconnected parts and we need to try to understand how they interact at different levels to have any chance of grasping the whole. Responding to a crisis effectively, rather than staggering from one quick-fix to the next, requires the ability to take an holistic view. In this case it means taking into account the impacts on people, individually and collectively, including their feelings and perceptions, the policies and processes of the various agencies involved and the purpose, ideology and politics of exams and assessment more generally.

A complex system contains multiple interacting sub-systems, each with its own dynamic. The outcomes of these interactions at different levels are not predictable. Such a system doesn’t lend itself to the kind of linear decision-making where someone can use a policy lever at the centre to guarantee a particular policy outcome such as greater fairness or equality. Instead, when things are changing fast, issues which previously seemed marginal can emerge and grow in importance, tipping points are reached and new problems become major concerns. If the system isn’t able to ‘correct’ itself quickly enough then what seemed like a fairly stable structure can simply topple over. The flap of the butterfly wing triggers the storm or the calm water becomes a tsunami.

This year’s results crisis

So how did a set of A Level results which were actually ‘better’ than those of 2019, achieved through a process designed to be as fair as possible, manage to cause such controversy and come to be seen as such a disaster?

The scene was set on the results day of a smaller nation with a different exam system. The concerns raised in Scotland helped to frame the debate in England by raising key questions, such as the extent of adjustment and its impact on students by socio-economic status. Within a few days, the English A Level results were causing controversy before they were even known.

A few of the key stages:

Adjustment became ‘downgrading’

It was known from the beginning that some adjustment would be applied to Centre Assessment Grades (CAGs) to correct for inconsistencies between centres and to aim for a broadly stable grade distribution. But there was also a sense that the CAGs would carry serious weight, even if some might need to be adjusted downwards. As soon as it became clear that 39% of CAGs would be adjusted downwards, the story became one of ‘downgrading’ and of a lack of trust in the centre assessment grades. Perhaps if the proportion being adjusted had been lower, 10% maybe, this might not have become such an issue. Faced with the scale of the changes, the narratives of ‘most CAGs will not be adjusted’, ‘only 4 in 10 CAGs will be adjusted’ or ‘96% of final grades will be the same or one below the CAG’ just didn’t cut it.

A ‘fair’ formula became unfair

Exams aspire to objectivity and applying a formula sounds objective. A well designed, formula will achieve what you want it to. However, with so many factors to consider, the key question was how would these various factors would weigh against each other, and that weighting was lost in a rather opaque algorithm whose ‘fairness’ people were asked to take on trust. In the end it was possible to criticise the algorithm both for what it did take into account, such as institutional history, as well as what it seemed not to, such as institutional subject value added. Without absolute clarity, the suspicion of inhuman, formulaic, ‘computer says no’ methods took hold.

Some cohorts were treated differently

Even if the formula had been fit for purpose, it seemed it wouldn’t apply to everyone. Small cohorts, of which there are many in the system – particularly post-16, are not susceptible to statistical adjustment, while larger ones are. This meant that the grade profile of smaller cohorts would be more likely to draw on unchanged CAGs despite being just as prone to ‘optimistic’ prediction as any other cohorts. Once this became clear, the conclusion was that centres with larger cohorts, such as colleges, would be proportionately disadvantaged and this was then borne out by the data for grades A and C.

In a high-stakes norm-referenced system, high grades are a valuable currency for progression. They are the ‘prizes’ which have scarcity value and are effectively rationed. For such a system to demonstrate that it is fair to all, in its own terms, the distribution of these high grades has to be seen to be based strictly on ‘merit’. But when this year’s small cohort effect saw the proportion of A* and A grades jump by a factor of 15 times more in private fee-paying school sixth forms than in colleges, any argument that this was deserved broke down. The claim that the system had not generated any more inequality than usual could no longer be believed.

Candidates and centres felt the impact

Once centres saw their results, it was clear that they were simply wrong on an unprecedented scale. For example, colleges with large and fairly stable cohorts were seeing their grade profile and value added fall below anything they’d seen in recent years. They were experiencing very high rates of CAG ‘downgrading’; well above 39%, and often 50% or more. The CAG predictions had taken account of real student performance and an historic understanding of their value added, applied subject by subject and student by student, but they seemed to have counted for little.

The next day, the shock and anger of college and school leaders were shared by many of their students as they received their results. Too many of them felt let down by a set of arbitrary and inconsistent processes which were completely outside of their control.

People lost confidence in the system

At tipping points, the key is often the balance of how people are feeling; their perceptions and anxieties about a process which affects them deeply. Being judged, measured, sorted and classified against common standards feels personal. It goes to the heart of our sense of worth and where we are placed in relation to others speaks to our sense of fairness. In the end, candidates and those who cared about them were more concerned about relative fairness within the class of 2020 than any potential unfairness to the classes of 2019 or 2021. There is a debate to be had about what ‘maintaining standards’ means – but defining it simply as ‘achieving a very similar grade profile year on year’ just couldn’t hold up against the evidence of systemic unfairness.

Going ‘full CAG’ became a serious option

Given that the Scottish government had already conceded this, it became a viable proposition for the other nations. It was resisted for some time on the grounds of ‘maintaining standards’ between years and not all stakeholders called for ‘full CAG’ as they could see its limitations. It would make it impossible to adjust any centre grades at all, risking exchanging one set of inequalities for another.

Independent agents, such as Oxbridge colleges started to announce that they would be considering CAGs for admissions because sticking to the calculated grades would make social mobility worse. Each of these separate decisions undermined the case for calculated grades and tipped the scales back towards the CAGs. As time ran out to resolve the problem, the choice became a binary. Centre Assessed Grades had to be better because the alternative was worse. With GCSE results looming, there was no good reason for any A Level U turn not to apply to GCSE too.

Anger, mistrust, and blame

A week after the ‘first’ A Level results day, students received their new ‘corrected’ grades based on the CAGs but preserving some which had been adjusted upwards. GCSE grades were issued as scheduled, also based on ‘CAG plus’. There was talk of mass complaints, data requests and litigation as some students still felt aggrieved about their final grade with some of this anger now directed at their centre.  There were also claims that some schools had approached CAG-setting very differently and might want the whole process to be re-opened for them. The idea that all these concerns could be adequately addressed through an individual appeals process, which tends to favour the ‘sharp elbowed’, became problematic.

In summary

A process which had started with broad support in principle foundered in its implementation through an accumulation of effects, the scale of which had not been predicted. These emerged gradually and amplified each other. Trust and confidence were eroded and the narrative of ‘ensuring fairness in a difficult year’ became harder to sustain. Whatever the strength of the ‘maintaining standards’ argument at the start, it was overwhelmed by the evidence of ‘greater inequality and more high grades for the better off’.

What next?

The first draft of history is still to be written but we now know how this phase of the results drama ended – we have ‘full CAG plus’ for both A Level and GCSE. As we continue to pick over the causes and consequences of this year’s process, we also need to hold on to all our underlying concerns about the system as it was, in order to consider what needs to be done to build a better, fairer system.

See also:

Before the A Level results were published, I wrote here about the questions which needed to be asked. I also wrote here about the particular challenge of grading post-16 GCSE retakes in English and maths.

England’s unexpected exams revolution (May 2020).