England’s school and college performance tables are full of fascinating information.

This information gives a profile of different providers and the idea is that this can help us make judgements and comparisons.

Any tables based on data, assuming they are accurate, represent real evidence of …well, reality. So at the level of truth they can’t be argued with. But such tables are also based on a particular view of what is worth reporting and what this means; they offer a selection from all the possible ways of trying to represent things. Data tables may give an impression of total objectivity but it’s important to understand the underlying assumptions and to be aware of their limitations. Clearly, data are reductive; they reduce things to something less than the whole in order to try to make the big picture clearer. So, while we can’t do justice to everything, it’s always worth asking about what’s been left out and why.

Let’s imagine 3 sixth forms; Colleges A, B and C, serving the hypothetical Anytown. In 2015 each of these colleges claimed to be the ‘best in Anytown’. So let’s look at their imaginary data to see if we can establish which was the highest performing or ‘best’ college.

1. College A can claim to be the highest performing college:

Students at college A have achieved higher average point scores than students in the other two colleges and also well above the national average. College A is clearly top of the league for raw performance.

2. College B can claim to be the highest performing college:

College B has achieved exactly the national average points per student on average and this represents a big improvement on last year and it is the only college in Anytown to improve in this way. This strong improvement means that College B can claim to be the most successful in Anytown, particularly given College A’s downward slide and College C’s stagnant performance.

3. College C can claim to be the highest performing college:

College C has a significantly positive value added score overall and is the only sixth form in Anytown with positive value added. This means that, on average, students at College C are achieving higher grades than expected based on their previous GCSE grades. College C can therefore claim to be the most successful, particularly given the significantly negative value added of the other two sixth forms.

4. So which college is right?

They’re all right. Each college is basing its claims on different measures in the performance tables but none are making any misleading claims. However, there is some further information which may add to our understanding. Interestingly the average grade per entry in each of the colleges is identical. In other words the average A-level grade achieved by students in each college is exactly the same and the only reason their points per student are different is because students in each college are entered for different numbers of A-levels on average. College A clearly enters most of its students for 4 A levels, while College C enters most of its students for 3.

The other information which is relevant is the respective cohort size for each college:

Because College C is larger than both the others put together it has a larger impact on the overall figures for Anytown. The way the different measures aggregate for Anytown also explains why an area which has 3 such successful colleges seems to be sitting pretty much on the national averages, or below them in some cases. This shows that institutional success can mask system stagnation.

Conclusion:

None of this is an argument against performance tables. It simply serves to demonstrate that we need to look behind the top level measures, evaluate all the available data and decide what we value.

Each of these colleges will have targets for improvement and could learn from the others. They would be well advised to work together to improve the Anytown system as a whole.

If you asked me to choose between the 3, I would tend towards College C which is clearly helping a more inclusive intake to achieve better than predicted grades. All things being equal, it is also turning out more well qualified students for progression than both the others put together.

See also:

Post-16 performance tables: taking the long view (January 2015)

London’s sixth forms (June 2016)

A sixth form profile for the ‘Local London’ area (February 2016)