3 ms·

>grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests Wait, this is like reverse affirm

by ppod 6y ago

>grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests Wait, this is like reverse affirmative action, no? Students at schools that previously performed badly will have their grades downweighted?

xyzal 6y ago
I understood it so as well. It seems to be the most horrible application of machine learning I have heard of lately.
chongli 6y ago
Students at schools that previously performed badly will have their grades downweighted? The article does not make it easy to understand how the actual model works. According to the IBO [1]: We will use a calculation that is based on the relationship between coursework marks, predicted grades and subject grades to estimate the subject grades candidates would have received if the exams had gone ahead. If the relationship between these elements shows that in previous sessions candidates globally tended to achieve higher outcomes on their exams than their coursework, the calculation used this session will reflect that. You can call this "reverse affirmative action", but really it's anti-grade-inflation. If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. So why would one school underperform on coursework and another underperform on exams? Grade inflation. Lots of schools make their coursework easier to try and boost the chances their students get into university. The whole point of standardized testing is to counteract that. So in this case, it may be very unfair to some students who might have been outliers on the exam, but there's no way of knowing that without actually running the exam. [1] https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-updates/ https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-...
- s1artibartfast 6y ago
  > If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. This is terrifying. It works for the average student, but distorts the performance for any student who isn't typical or better than their school. It seems that it would be better to just ignore the testing component of the IB score this year when judging students. Instead they modeled what the students would have gotten on the test, and then judged them based on their modeled performance.
  - chongli 6y ago
    distorts the performance for any student who isn't typical or better than their school. It distorts the performance of a student who underperforms in coursework relative to their peers but overperforms on exams, at a school where students overperform on coursework and underperform on exams. In other words, it punishes the student who does unusually poor on coursework that is already much easier, on average, than typical coursework from other schools. How likely are such students to succeed at a competitive university? Not very likely at all, I would think.
    - s1artibartfast 6y ago
      I don’t agree. It also punishes students with good coursework and who would have performed well on the test, but happen to be in schools which historically perform poorly on IB tests.
      - chongli 6y ago
        If the student outperforms their peers on coursework, then they won't be punished, they will have an estimated score higher than their peers.
        s1artibartfast 6y ago
        yes, they would be higher than their peers at their school, but then penalized because the cohort before them performed poorly on the IB test. For the students who would have gone on to test well on the IB test, this is clearly a disadvantage. E.g. : 1) You are in the 10th percentile of your class. 2) Students from your school in the 10th percentile scored in the 20th percentile on the IB test last year 3) You are assigned a 20th percentile on the IB test you can not take. This sucks if you would have done better than the 20th percentile on the IB test. Why model an IB test score at all? just apply a weighing to the coursework grades if you want to standardize outcomes between schools. it would be more straight-forwarded.

by ppod 6y ago

xyzal 6y ago

I understood it so as well. It seems to be the most horrible application of machine learning I have heard of lately.

chongli 6y ago

Students at schools that previously performed badly will have their grades downweighted? The article does not make it easy to understand how the actual model works. According to the IBO [1]: We will use a calculation that is based on the relationship between coursework marks, predicted grades and subject grades to estimate the subject grades candidates would have received if the exams had gone ahead. If the relationship between these elements shows that in previous sessions candidates globally tended to achieve higher outcomes on their exams than their coursework, the calculation used this session will reflect that. You can call this "reverse affirmative action", but really it's anti-grade-inflation. If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. So why would one school underperform on coursework and another underperform on exams? Grade inflation. Lots of schools make their coursework easier to try and boost the chances their students get into university. The whole point of standardized testing is to counteract that. So in this case, it may be very unfair to some students who might have been outliers on the exam, but there's no way of knowing that without actually running the exam. [1] https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-updates/ https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-...

s1artibartfast 6y ago
> If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. This is terrifying. It works for the average student, but distorts the performance for any student who isn't typical or better than their school. It seems that it would be better to just ignore the testing component of the IB score this year when judging students. Instead they modeled what the students would have gotten on the test, and then judged them based on their modeled performance.
- chongli 6y ago
  distorts the performance for any student who isn't typical or better than their school. It distorts the performance of a student who underperforms in coursework relative to their peers but overperforms on exams, at a school where students overperform on coursework and underperform on exams. In other words, it punishes the student who does unusually poor on coursework that is already much easier, on average, than typical coursework from other schools. How likely are such students to succeed at a competitive university? Not very likely at all, I would think.
  - s1artibartfast 6y ago
    I don’t agree. It also punishes students with good coursework and who would have performed well on the test, but happen to be in schools which historically perform poorly on IB tests.
    - chongli 6y ago
      If the student outperforms their peers on coursework, then they won't be punished, they will have an estimated score higher than their peers.
      - s1artibartfast 6y ago
        yes, they would be higher than their peers at their school, but then penalized because the cohort before them performed poorly on the IB test. For the students who would have gone on to test well on the IB test, this is clearly a disadvantage. E.g. : 1) You are in the 10th percentile of your class. 2) Students from your school in the 10th percentile scored in the 20th percentile on the IB test last year 3) You are assigned a 20th percentile on the IB test you can not take. This sucks if you would have done better than the 20th percentile on the IB test. Why model an IB test score at all? just apply a weighing to the coursework grades if you want to standardize outcomes between schools. it would be more straight-forwarded.