Comparing like with like

There’s been a lot of hype about the A-level results of our neighbouring 16-18 free school and there’s no doubt that their students have achieved some very good outcomes. But their claim to have the “best ever results by a sixth form college in the UK” is hard to believe and prompted a very rapid rebuttal from the Sixth Form Colleges Association (SFCA) which can be read here.

The fact is that this sixth form is very selective; setting a high bar for entry and the trouble with making crude comparisons between providers is that we are not comparing like with like.

The best predictor of achievement at 18 is achievement at 16; there is a strong correlation between individual students’ GCSE scores and their subsequent A-level scores. Different sixth forms apply different entry requirements and can therefore be working with a very different cohort mix which generates a very different set of likely outcomes.

For example, Newham sixth form college (NewVIc), which has a very broad and comprehensive intake, could easily generate very high crude A level scores simply be being more exclusive. By keeping more students out we could obtain similar scores to those achieved by selective providers. Nevertheless, more students progressed to Russell group universities from NewVIc than from any other sixth form in the borough.

Much of the success claimed by selective providers is therefore a function of who they keep out rather than who they let in. So if we are searching for measures of the value added or contribution to achievement of a sixth form we need to use those which compare like with like.

Our current national post-16 performance tables don’t help much with this as groups of students are not profiled in the way they are at key stage 4, showing the number of students in each ‘band’ of prior achievement for each school (low, middle and high attainers).

So, how could one start to compare like with like between very different sixth forms?

The key is to know more about the characteristics of each provider’s cohort, in particular their students’ prior achievement. If all the students in a selective provider have at least 5 GCSE grade A’s, or 5 grade B’s or a particular GCSE point score average on entry they can only be sensibly compared to the subset of students with the same prior achievement drawn from the more inclusive provider. Students’ outcomes should also only really be compared with those of other students on similar programmes (eg: at least 3 A levels).

Another factor which needs to be taken into account is retention; what proportion of starters actually finished the courses they started and then also achieved? A sixth form which does a big ‘clear-out’ of medium to low-achieving students after the first year is skewing its results compared to one which retains most of its students.

Here’s a proposal: why don’t we simply share the following data for our cohort of students enrolled on at least 3 AS/A levels with either: (a) 5+ GCSE grade A’s or (b) 5+ GCSE grade B’s or (c) Average GCSE point score of 6.0+ or any other appropriate measure.

  • Number of starters:
  • Number of completers entered for at least 3 A levels:
  • Average points per student and per entry:
  • Number progressing to HE / Russell group:

There are plenty of other interesting questions which could follow and value added measures could also be calculated for the comparable cohorts.

NewVIc would be happy to submit such performance data for objective and independent analysis and comparison with other providers on this basis. This would at least allow us all to evaluate the claims made by different types of provider and move us beyond the use of decontextualised data to make claims of unique excellence.

We are all doing a fairly similar job in similar circumstances and my hunch is that we are probably all doing a pretty good job overall. If this type of analysis shows up significant differences in performance, we could then try to understand what factors might be contributing and to share whatever good practice is making a difference. If any of us really do have some educational ‘magic dust’ then rather than guard it jealously I think we have a responsibility to share it with each other in the interests of all students.

So, can Newham’s post-16 providers rise to this challenge or will we remain locked in a dialogue of the deaf, each making only those claims which suit our case?

See also:

Can we celebrate success without rewriting history?

Russell group offers: hype and reality.

Celebrating success or manipulating data?

About Eddie Playfair

I am a Senior Policy Manager at the Association of Colleges (AoC) having previously been a college principal for 16 years and a teacher before that. I live in East London and I blog in a personal capacity about education and culture. I also tweet at @eddieplayfair
This entry was posted in Education, NewVIc and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s