It’s a love / hate thing.

On the one hand results day is a wonderful moment of celebration when all the hard work put in by students and staff is publicly celebrated, a moment when young people can reflect on what they’ve achieved and where they’re going. Everyone seems to be interested in how we’ve done and it’s one of the few occasions the media actively seek us out for positive stories.

But it’s also one of those moments when we get the sense that everything we do is being boiled down to a few numbers and letters published once a year. All our students’ growth and development, everything we’ve done to help them to become rounded and reflective young adults ready to make their mark on the world is summed up in a string of raw data. Such reductionism is outdone only in its crudeness by the harsh distillate of the single inspection grade for a whole institution. And why is it so hard to catch the media’s eye for good news stories during the rest of the year dammit?

I guess these are yin and yang; two necessary sides of the same coin. If we can’t sum up our work in very simple terms for the world to see, the world can’t begin to understand our work.

So here are a few health warnings when trying to make sense of results day news:

1. Take all claims made by institutions themselves with a heavy pinch of salt. They may be accurate but they (we) will select those nuggets of results news which reflect best on their achievements. It’s our moment in the spotlight and we need to show our best side. Be particularly wary of rankings and comparisons; one sixth form may well be the ‘top’ in its area for pass rates, others might be ‘top’ for high grades, numbers of passes or improvement or one of any number of measures.

2. Pass rates don’t tell the whole story. Pass rate news tends to dominate results day and this can smack of the ‘narcissism of small differences’. Most A-level providers now have pass rates in the high 90’s and the difference between 99% and 95% is not necessarily down to the excellence of the provider. A more selective sixth form, one which ‘weeds out’ students more stringently before entering them or one whose students simply have higher GCSE grades on entry may actually be underperforming compared to a sixth form with a slightly lower pass rate. This principle also applies to ‘high grade’ pass rates which can vary greatly based on the cohort profile.

3. Numbers of subjects achieving 100%. It’s always impressive to be able to reel off a list of subjects where students achieve 100%. However, this can be deceptive. A sixth form with an increased number of 100% subjects may have done less well because more of these subjects have very small entry numbers. 100% of 2 A-level Spanish candidates is still only 2 students.

4. Trends can be deceptive. It’s clearly a good thing if a sixth form’s results are on a strong upward trend but this is not necessarily a sign of institutionally driven improvement. It could also mean they’ve changed their entry policy and become more selective or it could be due to a completely random change in the profile of their cohort.

5. So we should be looking more at value added? Yes, although some of the measures are a bit crude, this gives a better sense of how students are doing against how they would be expected to do. The strongest predictor of A level grades is a student’s GCSE grade profile. There is a difference of several percentage points between the predicted pass rates of students with average GCSE grades of, say, A or B or C. So the proportions of these different students in a cohort makes a big difference to a sixth form’s expected pass rates (and high grade profile). Unfortunately, no value added measures are available on results day itself.

6. Don’t’ extrapolate too much from individual cases. These individual success stories help us to see the human side of achievement. But they are individual and may be exceptional. To understand the performance of a sixth form we need to know how typical it is that their students get into the university course, training opportunity or job they were aiming for. This progression information isn’t fully available on results day either.

When comparing providers based on data and averages it’s worth remembering that there are many good reasons why a particular provider may not be ‘average’. Fair judgements usually require a range of different data sliced in different ways and none of this is available on results day. Really objective judgements about sixth forms require deeper questioning as well as visits.

So, results day is all about the headlines. Let’s celebrate, tell our stories and bask in the attention. But the fine-grained analysis which helps us really understand performance has to wait for another day.

See also:

How to choose a sixth form (August 2014)

How to make a strong college application (Feb 2015)

Your college interview (Feb 2015)