3 ms·

This is adjacent, but I’ve wondered how close we are to having these automated plagiarism detection systems start identifying large numbers of false positives.

This is adjacent, but I’ve wondered how close we are to having these automated plagiarism detection systems start identifying large numbers of false positives. Especially as they adopt machine learning, and they let their fuzzy matches get a little more aggressive. Part of the problem with plagiarism in the abstract is that much of the work being submitted is not novel. Every one of us has been taught to read something, and then regurgitate it in a mildly different form to prove our successful learning of it. And our schools are spending tax dollars to buy access to these systems.

EamonnMR 4y ago
Back in my high school days we used a platform called TurnItIn which gave our writing a 'plagarism score'-too high and we'd need to rewrite the essay. It would flag silly things like two word fragments sometimes. I remember writing a strenuous objection because the license agreement (which I read because I was a bit of a stallmanite at the time) allowed the platform to publish our writing. In effect, they could supply plagiarism sites and then sell you the solution!
- tedd4u 4y ago
  Yeah universities use Turnitin, too. “Selling arms to both sides.”
cycomanic 4y ago
> This is adjacent, but I’ve wondered how close we are to having these automated plagiarism detection systems start identifying large numbers of false positives. Especially as they adopt machine learning, and they let their fuzzy matches get a little more aggressive. "how close we are" ...? The automated plagiarism engines have always had huge amounts of false positives. Essentially they are only useful for highlighting what they detected. Some of the things that I have seen them false detect: Direct cited quotes, common phrases and sentence structures. Factual explanations, e.g. a description of a specific aperatus which can only really be described in one way (even though exact wording differed). Reference lists, this was by far the most common reason for very high turnitin scores, when I used it turnitin was completely unable to pick up a reference list. The plagiarism scores of these systems are completely useless, I've found obvious plagiarism in work with scores of less than 3% (which is a very common score even if you would write an essay now) while I've had work with scores of >50% which were completely fine. Any teacher who uses the turnitin score for anything should be fired IMO, because they do not understand plagiarism.

EamonnMR 4y ago

Back in my high school days we used a platform called TurnItIn which gave our writing a 'plagarism score'-too high and we'd need to rewrite the essay. It would flag silly things like two word fragments sometimes. I remember writing a strenuous objection because the license agreement (which I read because I was a bit of a stallmanite at the time) allowed the platform to publish our writing. In effect, they could supply plagiarism sites and then sell you the solution!

tedd4u 4y ago
Yeah universities use Turnitin, too. “Selling arms to both sides.”

cycomanic 4y ago

> This is adjacent, but I’ve wondered how close we are to having these automated plagiarism detection systems start identifying large numbers of false positives. Especially as they adopt machine learning, and they let their fuzzy matches get a little more aggressive. "how close we are" ...? The automated plagiarism engines have always had huge amounts of false positives. Essentially they are only useful for highlighting what they detected. Some of the things that I have seen them false detect: Direct cited quotes, common phrases and sentence structures. Factual explanations, e.g. a description of a specific aperatus which can only really be described in one way (even though exact wording differed). Reference lists, this was by far the most common reason for very high turnitin scores, when I used it turnitin was completely unable to pick up a reference list. The plagiarism scores of these systems are completely useless, I've found obvious plagiarism in work with scores of less than 3% (which is a very common score even if you would write an essay now) while I've had work with scores of >50% which were completely fine. Any teacher who uses the turnitin score for anything should be fired IMO, because they do not understand plagiarism.