Breaking the scales

Someone tweeted me recently to ask what I thought about a school setting an expected rate of progress of 4 scale points from pupils’ KS1 test scores. A pupil scoring 100 on their KS1 test would therefore be expected to achieve 104 in their KS2 test, and – according to the source – they are, of course, expected to make one scale point per year to reach this ‘target’.

Having discussed this on twitter, and let my feelings known, I thought the subject was worthy of a blog post, in the hope the school in question might read it and stop, or that other schools thinking this could be a good idea, read it and realise that it isn’t.

I was asked what I think. This is what I think:

This is not how VA works

The progress measure between KS1 and KS2 is a VA measure. It involves comparing each pupil’s KS2 score to the national average score of pupils with the same KS1 prior attainment. We have no idea what the benchmarks will be in advance, which is why Ofsted are saying they will not be asking for predictions of progress (because it’s impossible!). Furthermore, the progress measure is and always has been based on the KS1 teacher assessment. It can never be based on the KS1 test score because the DfE didn’t collect the KS1 test scores, but if it was we would see that, for example, progressing from 94 at KS1 to 98 at KS2 might generate a negative VA score, whilst another pupil progressing from 106 at KS1 to 110 at KS2 may well end up with a positive VA score. It’s not a simple, one-size-fits-all system. And finally, the prior attainment groups on which progress measures depend are based on a combined score at KS1 in reading, writing and maths; they do not simply link the subject at KS1 to the same subject at KS2. Admittedly we have no idea what the DfE will do when the first cohort of non-levellers reach the end of KS2 next year, but that’s another reason not to bother attempting stuff like this.

They are different scales

The KS1 scores range from 85-115; the KS2 scores range from 80-120. They are different; they are not comparable. The KS1 scale is more compressed and so has the same number of pupils spread across fewer possible outcomes. One scale does not therefore directly translate into the other. Recent analysis of KS1 scaled scores from FFT (they collected data from over 2000 schools last year) showed the ‘national’ average score at KS1 to be 102. In the same year, the KS2 average score was 105. Yes, we are dealing with different cohorts, but that does suggest that a pupil going from 102 at KS1 to 105 at KS2 has not moved in real terms.

You can’t generate scaled score in Years 3, 4 or 5

Under this ‘system’, the expectation is that pupils make one scale point per year across KS2 (where have I heard that before?) but there are no tests that reliably generate scaled scores in the years between KS1 and KS2. In the absence of anything reliable you are literally reduced to making data up. And even if they got a reputable test provider to devise a national curriculum-linked series of tests, robustly administered in a large sample of schools in order to establish expected standards and scaled scores for interim years, adding on 4 points from KS1 is still nonsense. Which brings me on to the final point…

They’re just adding on 4 because there are 4 years

One of the things that always struck me about the accepted ‘truth’ of 3 points per year in the old levels world, was how convenient it was. A 2b at KS1 was 15 points and a 4b at KS2 was 27 points (where did those point scores come from?). Subtracting one from the other gave us 12 points, which could be neatly divided by the four years of KS2 to give us 3 points, which could be neatly divided by the three terms each year to give us a point per term. More recent attempts at tracking in a post levels world have continued with this convenient approach. All too often the amount of progress a pupil is expected to make is more a reflection of how many assessment windows or data drops the school has rather than anything rooted in reality. Many schools maintain the point per term orthodoxy, but schools in RI (or those that are in some way deemed to be vulnerable) track more often – every half term – and consequently expect 6 points per year. Either that or schools just go along with whatever it is their software prescribes. The method described here is no different: take a score at KS1 and add 4. Why? Because there are 4 years. It’s so convenient.

Despite all the reassuring statements from the DfE and Ofsted regarding assessment, data, tracking and workload, schools are still so desperate for numbers they will readily resort to making them up. There must be thousands of teachers up and down the country that have to suspend their disbelief every term in order to generate this data. It’s ‘think of a number’. A unit that doesn’t exist on a scale that can’t be measured. It’s pseudoscience. But with Ofsted now saying they will no longer be looking at tracking data you really do have to ask yourself: why bother? Who is any of this for?

Someone asked me recently when I’m going to change the record. When am I going to stop banging on about stuff like this?

I’ll stop banging on about stuff like this when schools stop doing it.