4 ms·

The Secret Algorithm That's Keeping Students Out of College

14 points by _bxg1 6y ago

ppod 6y ago
>grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests Wait, this is like reverse affirmative action, no? Students at schools that previously performed badly will have their grades downweighted?
- xyzal 6y ago
  I understood it so as well. It seems to be the most horrible application of machine learning I have heard of lately.
- chongli 6y ago
  Students at schools that previously performed badly will have their grades downweighted? The article does not make it easy to understand how the actual model works. According to the IBO [1]: We will use a calculation that is based on the relationship between coursework marks, predicted grades and subject grades to estimate the subject grades candidates would have received if the exams had gone ahead. If the relationship between these elements shows that in previous sessions candidates globally tended to achieve higher outcomes on their exams than their coursework, the calculation used this session will reflect that. You can call this "reverse affirmative action", but really it's anti-grade-inflation. If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. So why would one school underperform on coursework and another underperform on exams? Grade inflation. Lots of schools make their coursework easier to try and boost the chances their students get into university. The whole point of standardized testing is to counteract that. So in this case, it may be very unfair to some students who might have been outliers on the exam, but there's no way of knowing that without actually running the exam. [1] https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-updates/ https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-...
  - s1artibartfast 6y ago
    > If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. This is terrifying. It works for the average student, but distorts the performance for any student who isn't typical or better than their school. It seems that it would be better to just ignore the testing component of the IB score this year when judging students. Instead they modeled what the students would have gotten on the test, and then judged them based on their modeled performance.
    - chongli 6y ago
      distorts the performance for any student who isn't typical or better than their school. It distorts the performance of a student who underperforms in coursework relative to their peers but overperforms on exams, at a school where students overperform on coursework and underperform on exams. In other words, it punishes the student who does unusually poor on coursework that is already much easier, on average, than typical coursework from other schools. How likely are such students to succeed at a competitive university? Not very likely at all, I would think.
      - s1artibartfast 6y ago
        I don’t agree. It also punishes students with good coursework and who would have performed well on the test, but happen to be in schools which historically perform poorly on IB tests.
        chongli 6y ago
        If the student outperforms their peers on coursework, then they won't be punished, they will have an estimated score higher than their peers.
        s1artibartfast 6y ago
prepend 6y ago
In the future I just have to do really well in kindergarten and all the rest of my school and career will be predicted by a ML model with no correction whatsoever. I’m an IB grad and 5/6 exams could have been done electronically. Should at least give students the option to take exams.
quantified 6y ago
The article doesn’t include any commentary related to students that were mis-predicted in the other direction, i.e higher-than-expected scores. If the results stand, they’ll be an interesting case study in the predictive value of IB scores on college performance.
- MattGaiser 6y ago
  I suspect nobody is willing to go on record about that.
naveen99 6y ago
My 13 year old son just started doing classical mechanics on ocw.mit.edu No one can keep students out of open courseware. signaling is a zero sum game, but there is always a shortage of students who have the interest and aptitude to learn.

14 points by _bxg1 6y ago

ppod 6y ago

>grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests Wait, this is like reverse affirmative action, no? Students at schools that previously performed badly will have their grades downweighted?

xyzal 6y ago
I understood it so as well. It seems to be the most horrible application of machine learning I have heard of lately.
chongli 6y ago
Students at schools that previously performed badly will have their grades downweighted? The article does not make it easy to understand how the actual model works. According to the IBO [1]: We will use a calculation that is based on the relationship between coursework marks, predicted grades and subject grades to estimate the subject grades candidates would have received if the exams had gone ahead. If the relationship between these elements shows that in previous sessions candidates globally tended to achieve higher outcomes on their exams than their coursework, the calculation used this session will reflect that. You can call this "reverse affirmative action", but really it's anti-grade-inflation. If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. So why would one school underperform on coursework and another underperform on exams? Grade inflation. Lots of schools make their coursework easier to try and boost the chances their students get into university. The whole point of standardized testing is to counteract that. So in this case, it may be very unfair to some students who might have been outliers on the exam, but there's no way of knowing that without actually running the exam. [1] https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-updates/ https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-...
- s1artibartfast 6y ago
  > If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher. This is terrifying. It works for the average student, but distorts the performance for any student who isn't typical or better than their school. It seems that it would be better to just ignore the testing component of the IB score this year when judging students. Instead they modeled what the students would have gotten on the test, and then judged them based on their modeled performance.
  - chongli 6y ago
    distorts the performance for any student who isn't typical or better than their school. It distorts the performance of a student who underperforms in coursework relative to their peers but overperforms on exams, at a school where students overperform on coursework and underperform on exams. In other words, it punishes the student who does unusually poor on coursework that is already much easier, on average, than typical coursework from other schools. How likely are such students to succeed at a competitive university? Not very likely at all, I would think.
    - s1artibartfast 6y ago
      I don’t agree. It also punishes students with good coursework and who would have performed well on the test, but happen to be in schools which historically perform poorly on IB tests.
      - chongli 6y ago
        If the student outperforms their peers on coursework, then they won't be punished, they will have an estimated score higher than their peers.
        s1artibartfast 6y ago

prepend 6y ago

In the future I just have to do really well in kindergarten and all the rest of my school and career will be predicted by a ML model with no correction whatsoever. I’m an IB grad and 5/6 exams could have been done electronically. Should at least give students the option to take exams.

quantified 6y ago

The article doesn’t include any commentary related to students that were mis-predicted in the other direction, i.e higher-than-expected scores. If the results stand, they’ll be an interesting case study in the predictive value of IB scores on college performance.

MattGaiser 6y ago
I suspect nobody is willing to go on record about that.

naveen99 6y ago

My 13 year old son just started doing classical mechanics on ocw.mit.edu No one can keep students out of open courseware. signaling is a zero sum game, but there is always a shortage of students who have the interest and aptitude to learn.