6 ms·

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible. But if you're going to put this system in plac

by corin_ 6y ago

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible. But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not. They screwed it up so spectacularly that the education secretary had to make concessions to it two days before the results, then a day after the results Ofqual made a statement about who could appeal that contradicted the education secretary, before withdrawing that statement a few hours later saying they were still deciding. So sure it could be entirely down to incompetence not malice. But given either is possible, and this government's track record of cronyism I don't see why it's unreasonable to be sceptical.

mjburgess 6y ago
It doesnt "favour" any children. Those children were already going to get the highest results. There is no incompetence here. The predicted grades are a good estimate of what the results would have been. The issue is that all teachers significantly over-predicted their students grades, but less so at high-performing schools. (Why? because the grades are more consistent at those schools, so there's less room for error).
- corin_ 6y ago
  Independant schools (private schools) received 4.7% more A/A* grades than they did in 2019. The most common type of non-private school, Secondary Comprehensive, received only 2% more A/A* grades than in 2019. The algorithm didn't keep the gap between paid and unpaid education, it expanded it. (Source for these numbers and more in my first comment.) I was lucky enough to not only go to two private schools, but the second one was ranked #1 in England for exam ranks 2 of the 3 years I was there (and helped me get good grades despite being extremely unmotivated and lazy). But at least my privilege was to be given a good education that made learning / passing exams (the latter more) easy, not that the government decided my school deserved an artificial grades boost compared to state schools.
  - mjburgess 6y ago
    Right, and what was the previous yoy disparity? If the previous YoY was also 5%/2%, then it is just repeating the trend. It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance. There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many reasons to presume they'd be different.
    - corin_ 6y ago
      I can't find that data easily, if I'm bored tomorrow I might try to do the research myself - all the data is available but raw not top level overview at https://www.compare-school-performance.service.gov.uk/download-data https://www.compare-school-performance.service.gov.uk/downlo... It's only been a couple of days, we'll get more data journalists breaking it down better soon. You know what would be really helpful? If the algorithm was released to be publicly reviewed. Oh, but the government is refusing to do that, and Ofqual made anyone consulting on this project sign a 5 year NDA or they couldn't get involved. Hmm. Edit: and while I'm pretty sure the data will show schools weren't on track for that inflation, actually no I disagree with you that it would be fair to extrapolate previous cohort improvements to this year's anyway.
      - mjburgess 6y ago
        I agree. It should be published. However more important than the algorithm is just the media reporting on what ofqual actually say. I've barely read any of their actual analysis in news on the issue. Key facts: students are always ranked and graded on a curve. Student populations have a fixed number of grades awarded every year, etc. The system is specifically designed in recognition that grades are positional goods, ie., they are valuable only to the degree that other people do not have them. That's the view of the education system every year. And its the basic assumption of this process. They will never over-award A grades specifically because they want universities/business/etc. to index them to a percentile performance in the population.
- lhnz 6y ago
  > The issue is that all teachers significantly > over-predicted their students grades, > but less so at high-performing schools. "All teachers" is a sweeping statement. There are teachers in comprehensive schools that are tough-markers, too. My wife, who is a teacher in a London school, was annoyed in both directions - there were teachers that she felt had given away high grades, but there was also a teacher that had given the kids in her class grades lower than she expected they'd get (as her identity is tied to being overly careful and she liked to grade kids harder in mocks, etc). What she felt was bad was that cohorts of kids were often very different year upon year. Last year there were particularly bright kids doing Physics in her school, but this year there was only a few kids and they were much worse. But the algorithm awarded them grades above what she expected. There were also kids that she expects would have worked hard before their exams and gotten results above what they were given. They didn't get given the chance to do this.
  - mjburgess 6y ago
    I meant all teachers across school types, not all teachers individually. In both state and private schools, teachers over-predicted. But higher-performing schools have higher and more uniform results, so there's less room to actually over predict -- so there's comparatively less reduction. That's just an in-built concequence of what it means to be a high performing school
    - lhnz 6y ago
      Sure. It makes sense. I don't believe in the media conspiracy that the Tories created an algorithm that systematically awards higher grades to rich kids because of inherent classism or even by accident. But I do really disagree with the insistence that all teachers in comprehensive schools were conspiring to give their kids higher grades than they deserved -- this view was frequently repeated on social media. Even if there were some that were doing this and it was apparent from the statistics, teachers are not a homogenous group and there are many teachers that take fair grading very seriously and even take enjoyment in being tough markers. I don't know the data but my own worry is that this will have hurt the high variance or lazy kids that won't normally get high grades but would normally settle down and revise hard for such an important exam. I hope that these kids will be given the opportunity to resit the exam for free, since I doubt an algorithm could capture the variance that exists every year.
- DanBC 6y ago
  > but less so at high-performing schools. The bit you haven't understood is that teacher predicted grades were reassessed by the algorithm unless the class sizes were 5 children or fewer. This meant that the predicted grades that were not adjusted by the algo were overwhelmingly at independent fee-paying schools.
jdietrich 6y ago
Independent schools got higher grades because of a quirk in the algorithm. If there weren't enough students taking a particular course at a particular centre for the rank/previous results method to be reasonably accurate, the algorithm defaulted to partially or wholly relying on predicted grades. Independent schools generally have fewer students and a wider range of courses and predicted grades were inflated across the board, so they ended up with better grades than larger colleges and academies. I'm not sure how you could have avoided this bias without creating a lot of really weird outcomes.
- sukilot 6y ago
  Well, wealthy students who opt out of government secondary schools could be given lowest priority at government colleges, and if that offends someone they can expand government funded colleges so that all could attend.
  - jdietrich 6y ago
    >if that offends someone they can expand government funded colleges so that all could attend. Universities in England are funded primarily by tuition fees. There are consistently more university places than applicants, but not necessarily at universities you'd want to attend.
    - 082349872349872 6y ago
      You are now about to witness the extent of my oxbridge[1] knowledge: https://www.youtube.com/watch?v=McMSHqWM3G8 https://www.youtube.com/watch?v=McMSHqWM3G8 [1] to understand this locution it may help to understand that there is also a breakaway technical college somewhere off in the fens...
- DanBC 6y ago
  You could have looked at past performance of those schools. https://twitter.com/queenofswords6/status/1294471488351985667 https://twitter.com/queenofswords6/status/129447148835198566... > A levels...short thread > If anyone doubts who the winners are this year, here's a random selection of independent schools and the % of A*-A grades this year. > 2019 figures follow in brackets. Mayfield Girls' 66% (55%) Sherfield School 70% (37%) Leweston School 51% (37%) Farlington School 62% (48%) The Marist School 55% (52%) Godolphin School 50% (29.2%) St Mary's Cambridge 54% (51%) St George's Ascot 63% (35%) Rye St Anthony 48.1% (18.3%)

by corin_ 6y ago

mjburgess 6y ago

It doesnt "favour" any children. Those children were already going to get the highest results. There is no incompetence here. The predicted grades are a good estimate of what the results would have been. The issue is that all teachers significantly over-predicted their students grades, but less so at high-performing schools. (Why? because the grades are more consistent at those schools, so there's less room for error).

corin_ 6y ago
Independant schools (private schools) received 4.7% more A/A* grades than they did in 2019. The most common type of non-private school, Secondary Comprehensive, received only 2% more A/A* grades than in 2019. The algorithm didn't keep the gap between paid and unpaid education, it expanded it. (Source for these numbers and more in my first comment.) I was lucky enough to not only go to two private schools, but the second one was ranked #1 in England for exam ranks 2 of the 3 years I was there (and helped me get good grades despite being extremely unmotivated and lazy). But at least my privilege was to be given a good education that made learning / passing exams (the latter more) easy, not that the government decided my school deserved an artificial grades boost compared to state schools.
- mjburgess 6y ago
  Right, and what was the previous yoy disparity? If the previous YoY was also 5%/2%, then it is just repeating the trend. It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance. There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many reasons to presume they'd be different.
  - corin_ 6y ago
    I can't find that data easily, if I'm bored tomorrow I might try to do the research myself - all the data is available but raw not top level overview at https://www.compare-school-performance.service.gov.uk/download-data https://www.compare-school-performance.service.gov.uk/downlo... It's only been a couple of days, we'll get more data journalists breaking it down better soon. You know what would be really helpful? If the algorithm was released to be publicly reviewed. Oh, but the government is refusing to do that, and Ofqual made anyone consulting on this project sign a 5 year NDA or they couldn't get involved. Hmm. Edit: and while I'm pretty sure the data will show schools weren't on track for that inflation, actually no I disagree with you that it would be fair to extrapolate previous cohort improvements to this year's anyway.
    - mjburgess 6y ago
      I agree. It should be published. However more important than the algorithm is just the media reporting on what ofqual actually say. I've barely read any of their actual analysis in news on the issue. Key facts: students are always ranked and graded on a curve. Student populations have a fixed number of grades awarded every year, etc. The system is specifically designed in recognition that grades are positional goods, ie., they are valuable only to the degree that other people do not have them. That's the view of the education system every year. And its the basic assumption of this process. They will never over-award A grades specifically because they want universities/business/etc. to index them to a percentile performance in the population.
lhnz 6y ago
> The issue is that all teachers significantly > over-predicted their students grades, > but less so at high-performing schools. "All teachers" is a sweeping statement. There are teachers in comprehensive schools that are tough-markers, too. My wife, who is a teacher in a London school, was annoyed in both directions - there were teachers that she felt had given away high grades, but there was also a teacher that had given the kids in her class grades lower than she expected they'd get (as her identity is tied to being overly careful and she liked to grade kids harder in mocks, etc). What she felt was bad was that cohorts of kids were often very different year upon year. Last year there were particularly bright kids doing Physics in her school, but this year there was only a few kids and they were much worse. But the algorithm awarded them grades above what she expected. There were also kids that she expects would have worked hard before their exams and gotten results above what they were given. They didn't get given the chance to do this.
- mjburgess 6y ago
  I meant all teachers across school types, not all teachers individually. In both state and private schools, teachers over-predicted. But higher-performing schools have higher and more uniform results, so there's less room to actually over predict -- so there's comparatively less reduction. That's just an in-built concequence of what it means to be a high performing school
  - lhnz 6y ago
    Sure. It makes sense. I don't believe in the media conspiracy that the Tories created an algorithm that systematically awards higher grades to rich kids because of inherent classism or even by accident. But I do really disagree with the insistence that all teachers in comprehensive schools were conspiring to give their kids higher grades than they deserved -- this view was frequently repeated on social media. Even if there were some that were doing this and it was apparent from the statistics, teachers are not a homogenous group and there are many teachers that take fair grading very seriously and even take enjoyment in being tough markers. I don't know the data but my own worry is that this will have hurt the high variance or lazy kids that won't normally get high grades but would normally settle down and revise hard for such an important exam. I hope that these kids will be given the opportunity to resit the exam for free, since I doubt an algorithm could capture the variance that exists every year.
DanBC 6y ago
> but less so at high-performing schools. The bit you haven't understood is that teacher predicted grades were reassessed by the algorithm unless the class sizes were 5 children or fewer. This meant that the predicted grades that were not adjusted by the algo were overwhelmingly at independent fee-paying schools.

jdietrich 6y ago

Independent schools got higher grades because of a quirk in the algorithm. If there weren't enough students taking a particular course at a particular centre for the rank/previous results method to be reasonably accurate, the algorithm defaulted to partially or wholly relying on predicted grades. Independent schools generally have fewer students and a wider range of courses and predicted grades were inflated across the board, so they ended up with better grades than larger colleges and academies. I'm not sure how you could have avoided this bias without creating a lot of really weird outcomes.

sukilot 6y ago
Well, wealthy students who opt out of government secondary schools could be given lowest priority at government colleges, and if that offends someone they can expand government funded colleges so that all could attend.
- jdietrich 6y ago
  >if that offends someone they can expand government funded colleges so that all could attend. Universities in England are funded primarily by tuition fees. There are consistently more university places than applicants, but not necessarily at universities you'd want to attend.
  - 082349872349872 6y ago
    You are now about to witness the extent of my oxbridge[1] knowledge: https://www.youtube.com/watch?v=McMSHqWM3G8 https://www.youtube.com/watch?v=McMSHqWM3G8 [1] to understand this locution it may help to understand that there is also a breakaway technical college somewhere off in the fens...
DanBC 6y ago
You could have looked at past performance of those schools. https://twitter.com/queenofswords6/status/1294471488351985667 https://twitter.com/queenofswords6/status/129447148835198566... > A levels...short thread > If anyone doubts who the winners are this year, here's a random selection of independent schools and the % of A*-A grades this year. > 2019 figures follow in brackets. Mayfield Girls' 66% (55%) Sherfield School 70% (37%) Leweston School 51% (37%) Farlington School 62% (48%) The Marist School 55% (52%) Godolphin School 50% (29.2%) St Mary's Cambridge 54% (51%) St George's Ascot 63% (35%) Rye St Anthony 48.1% (18.3%)