5 ms·

Are students really comparing their work against the scores generated by these tools to game the scores lower because they're worried about being punished for f

Are students really comparing their work against the scores generated by these tools to game the scores lower because they're worried about being punished for false positives? That is, pardon my French, fucked up. Beyond that it's egregious that any academic institution would allow the success of students/researchers/whatever to hinge on scores generated by a black box algorithm that hasn't been rigorously proven as very close to 100% accurate. There should always be a human in the loop, and if we don't have enough humans for that then I think we have a problem that goes beyond automated plagiarism checkers.

dahfizz 8y ago
In my experience, that's not happening. First of all, if your work is submitted through these detection systems, you almost never have the opportunity to see your 'score' and then resubmit. That's just asking for people to cheat the system. These plagarism checkers are also not really smart enough to differentiate proper citation from actual plagarism. Thier function is to alert the person grading the papers to sections that might be problematic. It will highlight "plagarized" sections, but the professor will ignore it if it's properly quoted and attributed or if it's just a short phrase, etc.
- StavrosK 8y ago
  What stops me from renting the software and running my own paper through it to see what it says?
  - umvi 8y ago
    They usually only grant licenses to certain organizations. Same reason you can't simply buy the solutions manual to any given textbook.
    - mirimir 8y ago
      Well hey, there's a business opportunity!
    - paulmd 8y ago
      Quite often you can buy the solutions manual to a given textbook. They are on half.com or available in pdf format, a lot of the time. They won't show you the work, so copying the answer is not sufficient to get credit in practically any course. "Academic integrity" has ridiculous rules (eg self-plagiarism is not really a thing that anyone should care about) and I'm sure some people would be pissed about that, but if you're doing the work I personally don't see anything wrong with getting immediate feedback on whether you've done it correctly or not. Particularly given that many textbooks do like one or two simple examples and then jump to complex problems in the assignments... being able to check a couple problems to be sure you understand the process is extremely helpful as a learning aid.) If you want a real-world assessment of whether someone has mastered a concept, schedule an in-class quiz or a test.
      - cortesoft 8y ago
        > self-plagiarism is not really a thing that anyone should care about Sure it is. The entire methodology of schooling is to learn through exercise; if you are asked to write a paper, it isn't because the teacher wants the information in the paper. They are wanting you to learn through the act of writing the paper. If you just turn in a paper you have written for another class, you are not doing that exercise. Turning in a paper that you wrote for a different class and getting upset that the school doesn't approve is like thinking that sending a picture of you lifting weights 3 years ago is going to make you stronger.
        hopler 8y ago
        All of this nonsense comes one root evil: giving publicly viewable grades for fundamentally in original work. Grades and homework should be for the student's private benefit. The only public marks should be proctored exams for demonstrating skills, and whatever original work someone creates that has value that stands on its own outside he artificial school environment.
        dahfizz 8y ago
        That all depends on what you think school is for. I subscribe to the idea that school is for preparing people for society and provides a "great filter". Doing a task your superior asks of you and doing it well is arguably the most important thing school teaches you, with time and stress management being a close second. This is what grades at school measures. People love to talk about the virtue of being "well rounded" but the truth is that knowing bits of Shakespeare or knowing different types of Greek collumns doesn't benefit you as a person. All that information leaves your head pretty much immediately anyway. This is all especially true of high school. "Exams demonstrating skill" are not what matters to the people who look at your grades.
  - dymk 8y ago
    You can't just "rent" the software. They're going to try to verify that you work for an edu. That's way more hoops than even a semi-motivated plagiarizer is willing to jump through.
    - LifeLiverTransp 8y ago
      What stops me from buying acess via a sketchy edu?
      - saagarjha 8y ago
        It's work.
        LifeLiverTransp 8y ago
        Now hold on a minute, this whole industry has developed around shirking work. So surely, i can automatize part of this with a webcrawler and sci-hub.
        rainonmoon 8y ago
        Eh, I'll just wait for someone else to automate it and use theirs.
        LifeLiverTransp 8y ago
        Men, i would con some Venture Capital guy into building this, by giving the ilusion somebody else has already built a version, but im just too lazy. Would hire some students, but those would just copy paste it together from stack overflow.
      - dymk 8y ago
        > That's way more hoops than even a semi-motivated plagiarizer is willing to jump through
  - MiroF 8y ago
    If you're willing to go through this amount of work, why not just not plagiarize
- Gibbon1 8y ago
  If you were plagiarizing you could compare your paper with your sources. Might not be a bad idea even if you aren't to make sure you didn't fuck up and cut and paste something from your sources.
- Balgair 8y ago
  https://smallseotools.com/plagiarism-checker/ https://smallseotools.com/plagiarism-checker/ I mean, yeah, there should be a real person in the loop. But the issues are arising from when these types of tools aren't being used properly. It's not a problem when a professor is checking mid-terms from an upper-division class of 30. It is a problem when a grad-student with too little time is grading finals for a freshman class of 500. Sure, the purveyors of these tools can say: "Use this thing properly! If you don't, bad things will happen." But time and time again, bad things keep happening when people don't use the tool right. After some amount of incidents, the purveyors of the tools should just re-design the dang thing [0] such that it's harder to use it wrong. [0] Provided that the people that make the tool have any sense of morality and put human learning above dollars.
- cmroanirgo 8y ago
  That's pretty much my experience too. A good paper will use lots of citations, and hence have lots of flags from these services (eg turnitin), but this doesn't mean plagiarism - it means the student's doing the right thing: gathering supporting (or negating) evidence and quoting it for the purposes of argumentation.
bena 8y ago
I think the problem is the testing methods they may use to verify the service. They pass it the same text, it comes back as 100% plagiarized. Cool. They pass it text that is close, but clearly just slightly modified original text. Comes back as plagiarized. Cool. They pass it text that isn't close at all. Comes back as not plagiarized. Cool. It works. Right? Nope. We forgot to check if there are any cases where two different texts will show as being plagiarized despite knowing they've been independently created. We also didn't check to see if a text that was essentially plagiarized can return a result of not plagiarized. And I think those checks fall into the halting problem. You only need to find one case to throw the entire system into doubt. But to make sure there are no cases, you have to check every piece against every other piece. And if you have false positives and false negatives, you can either double check every result. In which case, you might as well not use it. Or you give up on one the case of false negatives. If a plagiarized paper passes the system, oh well. Focus on checking every paper that gets tagged as plagiarized. Double check them and make sure it truly is plagiarized.
- jerf 8y ago
  "And I think those checks fall into the halting problem." Metaphorically, if you mean that there isn't going to be a concrete metric that really determines whether two bits of text are "similar" with bright shining line, sure. If you mean halting problem literally, no, this isn't affected by the halting problem. String similarity evaluation won't be strong enough to invoke the properties that result in a halting problem. For any two strings, you can certainly terminate in a reasonably period of time, and have the "correct" answer as defined by whatever the local meaning of "answer" is. (Barring a particularly perverse implementation of a string similarity algorithm probably explicitly constructed just to screw with my statement here, that for some reason starts executing the string as instructions directly or something.)

dahfizz 8y ago

In my experience, that's not happening. First of all, if your work is submitted through these detection systems, you almost never have the opportunity to see your 'score' and then resubmit. That's just asking for people to cheat the system. These plagarism checkers are also not really smart enough to differentiate proper citation from actual plagarism. Thier function is to alert the person grading the papers to sections that might be problematic. It will highlight "plagarized" sections, but the professor will ignore it if it's properly quoted and attributed or if it's just a short phrase, etc.

StavrosK 8y ago
What stops me from renting the software and running my own paper through it to see what it says?
- umvi 8y ago
  They usually only grant licenses to certain organizations. Same reason you can't simply buy the solutions manual to any given textbook.
  - mirimir 8y ago
    Well hey, there's a business opportunity!
  - paulmd 8y ago
    Quite often you can buy the solutions manual to a given textbook. They are on half.com or available in pdf format, a lot of the time. They won't show you the work, so copying the answer is not sufficient to get credit in practically any course. "Academic integrity" has ridiculous rules (eg self-plagiarism is not really a thing that anyone should care about) and I'm sure some people would be pissed about that, but if you're doing the work I personally don't see anything wrong with getting immediate feedback on whether you've done it correctly or not. Particularly given that many textbooks do like one or two simple examples and then jump to complex problems in the assignments... being able to check a couple problems to be sure you understand the process is extremely helpful as a learning aid.) If you want a real-world assessment of whether someone has mastered a concept, schedule an in-class quiz or a test.
    - cortesoft 8y ago
      > self-plagiarism is not really a thing that anyone should care about Sure it is. The entire methodology of schooling is to learn through exercise; if you are asked to write a paper, it isn't because the teacher wants the information in the paper. They are wanting you to learn through the act of writing the paper. If you just turn in a paper you have written for another class, you are not doing that exercise. Turning in a paper that you wrote for a different class and getting upset that the school doesn't approve is like thinking that sending a picture of you lifting weights 3 years ago is going to make you stronger.
      - hopler 8y ago
        All of this nonsense comes one root evil: giving publicly viewable grades for fundamentally in original work. Grades and homework should be for the student's private benefit. The only public marks should be proctored exams for demonstrating skills, and whatever original work someone creates that has value that stands on its own outside he artificial school environment.
        dahfizz 8y ago
        That all depends on what you think school is for. I subscribe to the idea that school is for preparing people for society and provides a "great filter". Doing a task your superior asks of you and doing it well is arguably the most important thing school teaches you, with time and stress management being a close second. This is what grades at school measures. People love to talk about the virtue of being "well rounded" but the truth is that knowing bits of Shakespeare or knowing different types of Greek collumns doesn't benefit you as a person. All that information leaves your head pretty much immediately anyway. This is all especially true of high school. "Exams demonstrating skill" are not what matters to the people who look at your grades.
- dymk 8y ago
  You can't just "rent" the software. They're going to try to verify that you work for an edu. That's way more hoops than even a semi-motivated plagiarizer is willing to jump through.
  - LifeLiverTransp 8y ago
    What stops me from buying acess via a sketchy edu?
    - saagarjha 8y ago
      It's work.
      - LifeLiverTransp 8y ago
        Now hold on a minute, this whole industry has developed around shirking work. So surely, i can automatize part of this with a webcrawler and sci-hub.
        rainonmoon 8y ago
        Eh, I'll just wait for someone else to automate it and use theirs.
        LifeLiverTransp 8y ago
        Men, i would con some Venture Capital guy into building this, by giving the ilusion somebody else has already built a version, but im just too lazy. Would hire some students, but those would just copy paste it together from stack overflow.
    - dymk 8y ago
      > That's way more hoops than even a semi-motivated plagiarizer is willing to jump through
- MiroF 8y ago
  If you're willing to go through this amount of work, why not just not plagiarize
Gibbon1 8y ago
If you were plagiarizing you could compare your paper with your sources. Might not be a bad idea even if you aren't to make sure you didn't fuck up and cut and paste something from your sources.
Balgair 8y ago
https://smallseotools.com/plagiarism-checker/ https://smallseotools.com/plagiarism-checker/ I mean, yeah, there should be a real person in the loop. But the issues are arising from when these types of tools aren't being used properly. It's not a problem when a professor is checking mid-terms from an upper-division class of 30. It is a problem when a grad-student with too little time is grading finals for a freshman class of 500. Sure, the purveyors of these tools can say: "Use this thing properly! If you don't, bad things will happen." But time and time again, bad things keep happening when people don't use the tool right. After some amount of incidents, the purveyors of the tools should just re-design the dang thing [0] such that it's harder to use it wrong. [0] Provided that the people that make the tool have any sense of morality and put human learning above dollars.
cmroanirgo 8y ago
That's pretty much my experience too. A good paper will use lots of citations, and hence have lots of flags from these services (eg turnitin), but this doesn't mean plagiarism - it means the student's doing the right thing: gathering supporting (or negating) evidence and quoting it for the purposes of argumentation.

bena 8y ago

I think the problem is the testing methods they may use to verify the service. They pass it the same text, it comes back as 100% plagiarized. Cool. They pass it text that is close, but clearly just slightly modified original text. Comes back as plagiarized. Cool. They pass it text that isn't close at all. Comes back as not plagiarized. Cool. It works. Right? Nope. We forgot to check if there are any cases where two different texts will show as being plagiarized despite knowing they've been independently created. We also didn't check to see if a text that was essentially plagiarized can return a result of not plagiarized. And I think those checks fall into the halting problem. You only need to find one case to throw the entire system into doubt. But to make sure there are no cases, you have to check every piece against every other piece. And if you have false positives and false negatives, you can either double check every result. In which case, you might as well not use it. Or you give up on one the case of false negatives. If a plagiarized paper passes the system, oh well. Focus on checking every paper that gets tagged as plagiarized. Double check them and make sure it truly is plagiarized.

jerf 8y ago
"And I think those checks fall into the halting problem." Metaphorically, if you mean that there isn't going to be a concrete metric that really determines whether two bits of text are "similar" with bright shining line, sure. If you mean halting problem literally, no, this isn't affected by the halting problem. String similarity evaluation won't be strong enough to invoke the properties that result in a halting problem. For any two strings, you can certainly terminate in a reasonably period of time, and have the "correct" answer as defined by whatever the local meaning of "answer" is. (Barring a particularly perverse implementation of a string similarity algorithm probably explicitly constructed just to screw with my statement here, that for some reason starts executing the string as instructions directly or something.)