4 ms·

> The very idea of fairly guessing a student's grade is problematic, and only acceptable if you forget we are discussing real people, with individuality, dreams

> The very idea of fairly guessing a student's grade is problematic, and only acceptable if you forget we are discussing real people, with individuality, dreams and purpose. They would have certainly been happier sitting their exams, whatever it took to do so safely. They almost certainly...wouldn't? The exams are stressful and hard work: both high- and low-achieving students were celebrating when the standardised exams got cancelled here in Ireland and I'm sure it was little different in the UK. At best, those who think they'd get better grades in the exam would prefer it. The ultimate use of these grades is zero-sum (there are only so many university places) so every decision that hurts one student is helping someone else. It doesn't really matter how good the model is, it's an impossible task politically. There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary to fuel the outrage machine. As if the previous setup was some kind of examplar of egalitarianism.

jgbyrne 6y ago
There is good evidence that comprehensive school pupils were disadvantaged and independent school pupils were advantaged, when you dig beneath the overall averages.
- disgruntledphd2 6y ago
  Yeah, they decided to have a magic number of 6 when they started standardising. Given that lots of private (nee public) schools have smaller class sizes, this made the bias worse.
  - IshKebab 6y ago
    I believe it is 10-15 actually. If the class is more than 15 people then it uses historical grades. If it is less than 10 it just uses the prediction (which is obviously generous). In-between they are weighted.
- dynamite-ready 6y ago
  A number of reports suggest that the algorithm has (and will be, in the case of the coming GCSE results) been selectively applied too. So your academy's history will only be taken into account, once it's determined your school is of a certain size. Which most state run schools are, of course. This is a nightmare, really. Like, no one can directly prove the malice here, but it certainly feels like there was an aspect of malevolence in the enterprise.
  - llimos 6y ago
    Who would stand to gain from that and what would they stand to gain? UK education policy over the last couple of decades has been squarely set on getting as many state school students to university as possible so it would be strange for them to suddenly switch track on the sly. I'm no fan of the UK government but I'm pretty sure this is a textbook example of Hanlon's razor
    - crispyporkbites 6y ago
      > getting as many state school students to university They want the state students to get to universities yes, but they want the private students to get to the good universities (oxbridge).
    - dynamite-ready 6y ago
      It would keep exclusive institutions exclusive. Much of the UK educational system is like that. For example, I don't know how it works elsewhere, but in England, certain students will only be offered certain exam questions, in the same subject, if the teachers determine you can take them. So immediately, you can see some aspect of these grades are based on a subjective standard. In this case, at the discretion of teachers. One could easily argue that this Covid exam fiasco is a supercharged manifestation of those very same principles. In the above case, obviously, all students sitting the same exam, for the same subject, should see the same exam papers. I mean, why would they not? I suspect the obvious answer would be the cost of teaching resource. If that's the case, this then becomes a political argument, not a technical one. But the 'exam algo' debate is definitely a technical one, and it can and should be used as a lens to examine (pun) a number of questionable practices in the UK education system.
michaelt 6y ago
> both high- and low-achieving students were celebrating when the standardised exams got cancelled here in Ireland They celebrated the message "Exams are cancelled" If the message had instead been "Exams are cancelled, at this school A grade students will receive a C instead" I suspect you'd have seen a very different response.
- dmurray 6y ago
  Well, yeah, but they'd really be thrilled at the school where they announced C grade students would get an A. There is some benefit to society to matching up the right students to the right courses, so it's not completely zero-sum. But the cost of getting it a bit wrong isn't terrible: some marginal students get to follow their dreams and study medicine, some others miss out on medicine and have to try their hand at accountancy.
  - MaxBarraclough 6y ago
    This discounts the sense of disenfranchisement that may result, which might be consequential, or that we may end up with inferior doctors.
  - deleted 6y ago
    [deleted]
  - michaelt 6y ago
    I would expect moderation to lower grades much more often than it would raise grades. After all, it's understandable why a teacher would over-estimate grades (assumption everyone else is going to do the same, and their kids will be relatively disadvantaged if they don't) and hence get moderated down. And it's understandable why a teacher would give spot-on estimated grades (honesty, valuing the credibility of their estimates) and hence not get moderated at all. But it's difficult to see why a teacher would under-estimate grades. So I suspect there are not many C grade students getting upgraded to an A.
Joeri 6y ago
If some students would prefer to take exams and some not, it seems obvious to let only those students who want take the exams. Propose the model’s grades to the student. If they accept it, you’re done, if they refuse then let them do an exam. Likely 80% or more of students would accept the proposed grades and it would make it doable to organize socially distanced exams for the rest.
- IshKebab 6y ago
  Exams are already "social distanced". That was never an issue! Has everybody forgotten what exams are like?
  - jfk13 6y ago
    It wasn't just about the exams; students missed out on months of teaching, which would have put them at a huge disadvantage if they were then expected to sit the originally-planned exams. So we'd need exams tailored to just the part of the curriculum that the students have actually covered -- which would be different for each school, making any kind of standardisation impossible. Another thing people don't seem to be considering is the knock-on effect on universities, which -- if students who didn't complete the A-level course are given a hypothetical grade, whether based on teachers' predictions or modified exams or "mocks" or whatever -- will get an intake of students who are less well prepared than normal, and will need some kind of remedial support if they're not going to flounder in their degree courses. Just giving students generous grades now, to calm the current storm, means storing up trouble for the future when university teaching staff are faced with a cohort that includes students who shouldn't be there and can't cope -- but their teachers gave over-generous estimates -- or who in principle should be OK but arrived woefully under-prepared because they missed a bunch of key A-level content.
    - PeterisP 6y ago
      I agree with most of the points you're making, but I do not think that the exams would need to be tailored to just the part of the curriculum that the students have actually covered. If one student does not know key parts of the curriculum and the other does, then this should be reflected in the grade no matter what caused that difference, whether one of them got more schooling or studied on their own or took some out-of-school tutoring. And if a diligent student who got all A's before missed out on key parts of the curriculum that's required for further study, then the final grade should reflect that they do not (yet) know the subject as expected and are going to need remedial support, i.e. their final grade should not be an A, the exam should ask for the expected content (no matter if it was not covered in school due to Covid) and the exam results and grade should illustrate whether the students have learned it otherwise or if there's a gap.
detaro 6y ago
"okay, lets do it, refuse all evaluation of our work and just push something through" is not the appropriate response to "this task might be impossible". It being "zero-sum" in some sense is also an extremely low bar we don't generally accept as sufficient.
algesten 6y ago
> It doesn't really matter how good the model is, it's an impossible task politically. There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary to fuel the outrage machine. As if the previous setup was some kind of examplar of egalitarianism. So just because every system has flaws, we should throw in the towel and give up? Taking that line of thinking all the way means we can just replace university allocations with a lottery. That's not how society works. Even if every system have flaws, even if the current suffers from racism and classism, we have to strive towards making it better. The system the UK invented here does not even try. It very clearly favors people with nice postcodes. Maybe we should replace it with a lottery, it would be equal. But I'm pretty sure the people with nice postcodes would like it much, and that's the people propping up the UK gov. Etonians, Oxford, silver spoon etc. I for one believe there must be _some way_ to fix the situation and is better than what they came up with.
- dynamite-ready 6y ago
  It's clear there's some abuse of power going on here. It doesn't sound like an entirely objective process. That the algorithm has been selectively applied, is definitely worth an independent review and some time in court. That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate, because there has been evidence of a class based skew in the UK education system for years now. And this appears to be the most overt attempt to rig the system yet seen.
  - DanBC 6y ago
    > That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate, There's a public sector equality duty under the Equality Act 2010 which is one reason people are looking at the disproportionate impact on some groups.
zimpenfish 6y ago
> There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary But if you have a transparent process and documented algorithms, it's much, much harder for them to do that. Of course, UKGOV has neither of those things in this instance...
dynamite-ready 6y ago
Well, Covid definitely meant some drastic action had to be taken, but when the UK government originally discussed suspending high school 'finals' for all students, they strongly suggested the derived grades would be based on mock exams, and teachers predictions / appraisals. Which would have at least localised the appeal system. Then any regulatory board could have worked 'backwards', so to speak... Only look at schools where there are blatant discrepancies across aggregated results. There was never any mention of the application of an opaque post-process. You would think that would be the kind of detail you would have to discuss in a national assembly before the state commissions it's implementation, but clearly, it was never raised. So while I understand your point (accusations of 'classism' are premature), I think the fact that we're only learning the details of this system now, is an outrageous abuse of trust and technology.
Mvandenbergh 6y ago
They would have been better off politically just giving them all their predicted grades and letting the universities sort out the mess. I think that's a bit cowardly but it would have taken the heat off. Concerns over long-term grade inflation are totally overblown anyway since the majority of these exam results get used that year or the next for university admission and only for that. What does it matter whether my grades from 20 years ago are comparable with grades now? It's not like we'd be competing for the same university places. The only problem with accepting the CAGs is that some centres will have been laxer with them but that could easily have been sorted by using similar techniques to the ones they used here to look for way out-there schools that clearly aren't taking the process seriously.

jgbyrne 6y ago

There is good evidence that comprehensive school pupils were disadvantaged and independent school pupils were advantaged, when you dig beneath the overall averages.

disgruntledphd2 6y ago
Yeah, they decided to have a magic number of 6 when they started standardising. Given that lots of private (nee public) schools have smaller class sizes, this made the bias worse.
- IshKebab 6y ago
  I believe it is 10-15 actually. If the class is more than 15 people then it uses historical grades. If it is less than 10 it just uses the prediction (which is obviously generous). In-between they are weighted.
dynamite-ready 6y ago
A number of reports suggest that the algorithm has (and will be, in the case of the coming GCSE results) been selectively applied too. So your academy's history will only be taken into account, once it's determined your school is of a certain size. Which most state run schools are, of course. This is a nightmare, really. Like, no one can directly prove the malice here, but it certainly feels like there was an aspect of malevolence in the enterprise.
- llimos 6y ago
  Who would stand to gain from that and what would they stand to gain? UK education policy over the last couple of decades has been squarely set on getting as many state school students to university as possible so it would be strange for them to suddenly switch track on the sly. I'm no fan of the UK government but I'm pretty sure this is a textbook example of Hanlon's razor
  - crispyporkbites 6y ago
    > getting as many state school students to university They want the state students to get to universities yes, but they want the private students to get to the good universities (oxbridge).
  - dynamite-ready 6y ago
    It would keep exclusive institutions exclusive. Much of the UK educational system is like that. For example, I don't know how it works elsewhere, but in England, certain students will only be offered certain exam questions, in the same subject, if the teachers determine you can take them. So immediately, you can see some aspect of these grades are based on a subjective standard. In this case, at the discretion of teachers. One could easily argue that this Covid exam fiasco is a supercharged manifestation of those very same principles. In the above case, obviously, all students sitting the same exam, for the same subject, should see the same exam papers. I mean, why would they not? I suspect the obvious answer would be the cost of teaching resource. If that's the case, this then becomes a political argument, not a technical one. But the 'exam algo' debate is definitely a technical one, and it can and should be used as a lens to examine (pun) a number of questionable practices in the UK education system.

michaelt 6y ago

> both high- and low-achieving students were celebrating when the standardised exams got cancelled here in Ireland They celebrated the message "Exams are cancelled" If the message had instead been "Exams are cancelled, at this school A grade students will receive a C instead" I suspect you'd have seen a very different response.

dmurray 6y ago
Well, yeah, but they'd really be thrilled at the school where they announced C grade students would get an A. There is some benefit to society to matching up the right students to the right courses, so it's not completely zero-sum. But the cost of getting it a bit wrong isn't terrible: some marginal students get to follow their dreams and study medicine, some others miss out on medicine and have to try their hand at accountancy.
- MaxBarraclough 6y ago
  This discounts the sense of disenfranchisement that may result, which might be consequential, or that we may end up with inferior doctors.
- deleted 6y ago
  [deleted]
- michaelt 6y ago
  I would expect moderation to lower grades much more often than it would raise grades. After all, it's understandable why a teacher would over-estimate grades (assumption everyone else is going to do the same, and their kids will be relatively disadvantaged if they don't) and hence get moderated down. And it's understandable why a teacher would give spot-on estimated grades (honesty, valuing the credibility of their estimates) and hence not get moderated at all. But it's difficult to see why a teacher would under-estimate grades. So I suspect there are not many C grade students getting upgraded to an A.

Joeri 6y ago

If some students would prefer to take exams and some not, it seems obvious to let only those students who want take the exams. Propose the model’s grades to the student. If they accept it, you’re done, if they refuse then let them do an exam. Likely 80% or more of students would accept the proposed grades and it would make it doable to organize socially distanced exams for the rest.

IshKebab 6y ago
Exams are already "social distanced". That was never an issue! Has everybody forgotten what exams are like?
- jfk13 6y ago
  It wasn't just about the exams; students missed out on months of teaching, which would have put them at a huge disadvantage if they were then expected to sit the originally-planned exams. So we'd need exams tailored to just the part of the curriculum that the students have actually covered -- which would be different for each school, making any kind of standardisation impossible. Another thing people don't seem to be considering is the knock-on effect on universities, which -- if students who didn't complete the A-level course are given a hypothetical grade, whether based on teachers' predictions or modified exams or "mocks" or whatever -- will get an intake of students who are less well prepared than normal, and will need some kind of remedial support if they're not going to flounder in their degree courses. Just giving students generous grades now, to calm the current storm, means storing up trouble for the future when university teaching staff are faced with a cohort that includes students who shouldn't be there and can't cope -- but their teachers gave over-generous estimates -- or who in principle should be OK but arrived woefully under-prepared because they missed a bunch of key A-level content.
  - PeterisP 6y ago
    I agree with most of the points you're making, but I do not think that the exams would need to be tailored to just the part of the curriculum that the students have actually covered. If one student does not know key parts of the curriculum and the other does, then this should be reflected in the grade no matter what caused that difference, whether one of them got more schooling or studied on their own or took some out-of-school tutoring. And if a diligent student who got all A's before missed out on key parts of the curriculum that's required for further study, then the final grade should reflect that they do not (yet) know the subject as expected and are going to need remedial support, i.e. their final grade should not be an A, the exam should ask for the expected content (no matter if it was not covered in school due to Covid) and the exam results and grade should illustrate whether the students have learned it otherwise or if there's a gap.

detaro 6y ago

"okay, lets do it, refuse all evaluation of our work and just push something through" is not the appropriate response to "this task might be impossible". It being "zero-sum" in some sense is also an extremely low bar we don't generally accept as sufficient.

algesten 6y ago

> It doesn't really matter how good the model is, it's an impossible task politically. There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary to fuel the outrage machine. As if the previous setup was some kind of examplar of egalitarianism. So just because every system has flaws, we should throw in the towel and give up? Taking that line of thinking all the way means we can just replace university allocations with a lottery. That's not how society works. Even if every system have flaws, even if the current suffers from racism and classism, we have to strive towards making it better. The system the UK invented here does not even try. It very clearly favors people with nice postcodes. Maybe we should replace it with a lottery, it would be equal. But I'm pretty sure the people with nice postcodes would like it much, and that's the people propping up the UK gov. Etonians, Oxford, silver spoon etc. I for one believe there must be _some way_ to fix the situation and is better than what they came up with.

dynamite-ready 6y ago
It's clear there's some abuse of power going on here. It doesn't sound like an entirely objective process. That the algorithm has been selectively applied, is definitely worth an independent review and some time in court. That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate, because there has been evidence of a class based skew in the UK education system for years now. And this appears to be the most overt attempt to rig the system yet seen.
- DanBC 6y ago
  > That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate, There's a public sector equality duty under the Equality Act 2010 which is one reason people are looking at the disproportionate impact on some groups.

zimpenfish 6y ago

> There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary But if you have a transparent process and documented algorithms, it's much, much harder for them to do that. Of course, UKGOV has neither of those things in this instance...

dynamite-ready 6y ago

Well, Covid definitely meant some drastic action had to be taken, but when the UK government originally discussed suspending high school 'finals' for all students, they strongly suggested the derived grades would be based on mock exams, and teachers predictions / appraisals. Which would have at least localised the appeal system. Then any regulatory board could have worked 'backwards', so to speak... Only look at schools where there are blatant discrepancies across aggregated results. There was never any mention of the application of an opaque post-process. You would think that would be the kind of detail you would have to discuss in a national assembly before the state commissions it's implementation, but clearly, it was never raised. So while I understand your point (accusations of 'classism' are premature), I think the fact that we're only learning the details of this system now, is an outrageous abuse of trust and technology.

Mvandenbergh 6y ago

They would have been better off politically just giving them all their predicted grades and letting the universities sort out the mess. I think that's a bit cowardly but it would have taken the heat off. Concerns over long-term grade inflation are totally overblown anyway since the majority of these exam results get used that year or the next for university admission and only for that. What does it matter whether my grades from 20 years ago are comparable with grades now? It's not like we'd be competing for the same university places. The only problem with accepting the CAGs is that some centres will have been laxer with them but that could easily have been sorted by using similar techniques to the ones they used here to look for way out-there schools that clearly aren't taking the process seriously.