3 ms·

My guess is that soon or later people will learn how to game that AI. Already been done. http://lesperelman.com/writing-assessment-robo-grading/babel-generator

by dagw 7y ago

My guess is that soon or later people will learn how to game that AI. Already been done. http://lesperelman.com/writing-assessment-robo-grading/babel-generator/ http://lesperelman.com/writing-assessment-robo-grading/babel... Here's a sample essay that is complete nonsense and got a perfect score on the GRE. http://lesperelman.com/wp-content/uploads/2015/12/6-6_ScoreItNow_2015_Feb20.pdf http://lesperelman.com/wp-content/uploads/2015/12/6-6_ScoreI...

thombat 7y ago
The final paragraph from that example is steaming gibberish that nobody could mistake for English: "Calling has not, and undoubtedly never will be aggravating in the way we encounter mortification but delineate the reprimand that should be inclination. Nonetheless, armed with the knowledge that the analysis augurs stealth with propagandists, almost all of the utterances on my authorization journey. Since sanctions are performed at knowledge, a quantity of vocation can be more gaudily inspected. Knowledge will always be a part of society.Vocation is the most presumptuously perilous assassination of mankind." Yet the robo-scoring acclaims it as: * articulates a clear and insightful position on the issue in accordance with the assigned task * develops the position fully with compelling reasons and/or persuasive examples * sustains a well-focused, well-organized analysis, connecting ideas logically * conveys ideas fluently and precisely, using effective vocabulary and sentence variety * demonstrates superior facility with the conventions of standard written English (i.e., grammar, usage, and mechanics) but may have minor errors Any teacher faced with the requirement to use such tools would be better placed instructing their class on civil disobedience.
- crankylinuxuser 7y ago
  The let me posit another idea... There's 2 ways of finding out these artifacts of AI essay grading: pure luck, and being able to afford extensive test-prep (rich). The luck one can't be accounted for. So I im lead to believe that the purpose of these essays and their AI grading is to find and escalate rich people.
  - JustSomeNobody 7y ago
    > So I im lead to believe that the purpose of these essays and their AI grading is to find and escalate rich people. Well, of course. How many poor people are allowed to decide what is good for children's education?
    - crankylinuxuser 7y ago
      The standard US response is: 'There's a reason why they're poor. Better pull themselves up by the bootstraps." Mixed alongside with poverty stricken neighborhoods are the primary funding, resulting in poor school systems. And those students obviously wont have the money or the access to get the test-prep needed to "succeed". It's all too laid out to be accidental.
maze-le 7y ago
That reads like it was written by a GPT-2 bot.
- sidpatil 7y ago
  A Markov chain would have been able to produce a text like that.
C1sc0cat 7y ago
Wow - should have been marked for improper use of : though :-)
mannykannot 7y ago
Professor Perelman had also previously demonstrated that this sort of scoring was going on when essays were scored by humans [1]. I suspect that, in addition to the scoring rules being written for speed and frugality, they were shaped by a poorly thought out attempt to make the scoring 'objective', and independent of the scorer's beliefs, attitudes and unconscious biases. In one sense, this software (I will not call it 'AI') is an extension of all those bad ideas, only greatly amplified in a way that only software can. In evolutionary biology, there is the concept of 'honest signalling'[2], a true and unfakable indicator, to a potential mate or predator, of an animal's fitness. That is what we are missing here. [1] https://www.bostonglobe.com/opinion/2014/03/13/the-man-who-killed-sat-essay/L9v3dbPXewKq8oAvOUqONM/story.html https://www.bostonglobe.com/opinion/2014/03/13/the-man-who-k... [2] https://en.wikipedia.org/wiki/Signalling_theory https://en.wikipedia.org/wiki/Signalling_theory
DoctorOetker 7y ago
Anpther issue is one of copyright: obviously the student is the author. And we all know that the ML scoring subcontractor is keeping copies, with human ratings for later retraining purpouses. At the time the student takes the test, he should be prompted with the informed choice asking him to grant either 1) no license to keep a copy for training purpouses 2) a non-exclusive license, and the website where he can get a copy of his own essay 3) a public domain license, again with the relevant domain linked so he can find his own and other's essays. 4) any of the above or other as a function of the resulting grade! At the same time he should also specify his desire for or against attribution, again probably best as a function of the resulting grade. And under what moniker he wishes this contribution to exist. These options to be filled out during exam time should have no default options (no opt-out), and preferably should be standardized by the community and lobbied for to be mandatorily enforced at state or federal level (forcing examinations to present the student with an informed choice) A public dataset of legally obtained essays (without scores or names) would already be a very important first step to invite others to make actual performant ML grading systems. I don't believe the current datasets in these "non-profit" organizations actually comply with copyright law, organizations who don't profit from the grading service towards the state, but do provide a stable ML job on the income from charging the people with financial means to test submissions, enabling a stealth class based society.

by dagw 7y ago

thombat 7y ago

The final paragraph from that example is steaming gibberish that nobody could mistake for English: "Calling has not, and undoubtedly never will be aggravating in the way we encounter mortification but delineate the reprimand that should be inclination. Nonetheless, armed with the knowledge that the analysis augurs stealth with propagandists, almost all of the utterances on my authorization journey. Since sanctions are performed at knowledge, a quantity of vocation can be more gaudily inspected. Knowledge will always be a part of society.Vocation is the most presumptuously perilous assassination of mankind." Yet the robo-scoring acclaims it as: * articulates a clear and insightful position on the issue in accordance with the assigned task * develops the position fully with compelling reasons and/or persuasive examples * sustains a well-focused, well-organized analysis, connecting ideas logically * conveys ideas fluently and precisely, using effective vocabulary and sentence variety * demonstrates superior facility with the conventions of standard written English (i.e., grammar, usage, and mechanics) but may have minor errors Any teacher faced with the requirement to use such tools would be better placed instructing their class on civil disobedience.

crankylinuxuser 7y ago
The let me posit another idea... There's 2 ways of finding out these artifacts of AI essay grading: pure luck, and being able to afford extensive test-prep (rich). The luck one can't be accounted for. So I im lead to believe that the purpose of these essays and their AI grading is to find and escalate rich people.
- JustSomeNobody 7y ago
  > So I im lead to believe that the purpose of these essays and their AI grading is to find and escalate rich people. Well, of course. How many poor people are allowed to decide what is good for children's education?
  - crankylinuxuser 7y ago
    The standard US response is: 'There's a reason why they're poor. Better pull themselves up by the bootstraps." Mixed alongside with poverty stricken neighborhoods are the primary funding, resulting in poor school systems. And those students obviously wont have the money or the access to get the test-prep needed to "succeed". It's all too laid out to be accidental.

maze-le 7y ago

That reads like it was written by a GPT-2 bot.

sidpatil 7y ago
A Markov chain would have been able to produce a text like that.

C1sc0cat 7y ago

Wow - should have been marked for improper use of : though :-)

mannykannot 7y ago

Professor Perelman had also previously demonstrated that this sort of scoring was going on when essays were scored by humans [1]. I suspect that, in addition to the scoring rules being written for speed and frugality, they were shaped by a poorly thought out attempt to make the scoring 'objective', and independent of the scorer's beliefs, attitudes and unconscious biases. In one sense, this software (I will not call it 'AI') is an extension of all those bad ideas, only greatly amplified in a way that only software can. In evolutionary biology, there is the concept of 'honest signalling'[2], a true and unfakable indicator, to a potential mate or predator, of an animal's fitness. That is what we are missing here. [1] https://www.bostonglobe.com/opinion/2014/03/13/the-man-who-killed-sat-essay/L9v3dbPXewKq8oAvOUqONM/story.html https://www.bostonglobe.com/opinion/2014/03/13/the-man-who-k... [2] https://en.wikipedia.org/wiki/Signalling_theory https://en.wikipedia.org/wiki/Signalling_theory

DoctorOetker 7y ago

Anpther issue is one of copyright: obviously the student is the author. And we all know that the ML scoring subcontractor is keeping copies, with human ratings for later retraining purpouses. At the time the student takes the test, he should be prompted with the informed choice asking him to grant either 1) no license to keep a copy for training purpouses 2) a non-exclusive license, and the website where he can get a copy of his own essay 3) a public domain license, again with the relevant domain linked so he can find his own and other's essays. 4) any of the above or other as a function of the resulting grade! At the same time he should also specify his desire for or against attribution, again probably best as a function of the resulting grade. And under what moniker he wishes this contribution to exist. These options to be filled out during exam time should have no default options (no opt-out), and preferably should be standardized by the community and lobbied for to be mandatorily enforced at state or federal level (forcing examinations to present the student with an informed choice) A public dataset of legally obtained essays (without scores or names) would already be a very important first step to invite others to make actual performant ML grading systems. I don't believe the current datasets in these "non-profit" organizations actually comply with copyright law, organizations who don't profit from the grading service towards the state, but do provide a stable ML job on the income from charging the people with financial means to test submissions, enabling a stealth class based society.