4 ms·

And now that they've already cheated and were caught, what happens? Double down? Come clean? Nope, post publicly online for ideas. This person does not seem to

by thdc 4y ago

And now that they've already cheated and were caught, what happens? Double down? Come clean? Nope, post publicly online for ideas. This person does not seem to think things through. Glad to see cheaters put in their place, although I wonder what they would have done if ChatGPT didn't exist.

filoleg 4y ago
What cheaters would do if ChatGPT didn't exist? They would probably be doing the exact same things they've been doing before ChatGPT existed. If you meant the OOP specifically, either they would cheat in some old-fashioned way or not cheat at all. Difficult to tell, especially since we don't know whether it was the first time they cheated.
g_p 4y ago
> I wonder what they would have done if ChatGPT didn't exist. This can be answered, as until late last year it effectively didn't for the average student. There were 3 main things they'd do (in my experience) for some kind of take-home assignment. In-room or at-home exam cheating are whole different subjects. 1. Pass notes or answers from year to year. Think shared Dropbox folder between cohorts, with papers and solutions to assignments. Simple fix - don't be lazy, make new graded assignments each year. Actually by doing this, students learn more as they practice the previous papers! Short circuit this by giving them the past assignments openly, and encouraging them to work together on them, then give them a fresh assignment to work on. They'll understand it and be in a good place to do it without cheating. 2. Use essay mills to pay someone to write them an "essay". These sites effectively sell bespoke written essays, marketed with pseudo legitimacy as "study materials". Some come with guarantees of uniqueness and that they won't trip plagiarism scanners as they haven't been given to anyone else. Colleges have access to tools that detect content submitted to other colleges previously, as well as content from the open web, so the essay mill pays other more advanced students or graduates to quickly whip up unique essays for paying students. Without ChatGPT, I can say with confidence this is what they would do, because it's what they did up until late last year. You could do stylistic analysis or similar to look for changes in writing style over multiple submissions if something seems "off" - with luck you'll find things like variations of spelling certain words, or specific writing style changes. Or the essay might be overly generic and missing the substance of the specific question topic. 3. Straightforward plagiarism via "copy paste", or "copy, reword a few unusual words, paste". Occasionally you get something more evolved like "copy, paraphrase, paste", or use of a thesaurus to swap words out (often inappropriately) to try avoid binary comparison detection. Fuzzy detection usually spits this though. Sometimes you'll simply get the exact same piece of work submitted by multiple students who have colluded and shared their answer with each other.
- thdc 4y ago
  Right, totally forgot essay mills were a thing. Originally, outside of plagiarism, I couldn't think of any ways to cheat on an essay. Part of writing an essay, or at least the type of essay I'm thinking of, is researching and criticizing other views on the subject, so if you find some material to draw from you might a well incorporate it properly. The other part is presenting your own thoughts - which I assume is where ChatGPT came into play. I'm not aware of anything that could handle this part before ChatGPT (or other LLMs) existed short of presenting other people's thoughts as your own (also plagiarism).
  - g_p 4y ago
    Part of the challenge around essays is likely also what type of essay is sought, and the content required. Some educators are lazy, and just want something submitted with credible sounding words, parroting something suitably deferential to the accepted norms of the field. Those kinds of essays are the ones most likely to find themselves the subjects of tech or non tech enabled cheating. They're also effectively low signal to noise ratio measures, and I'd discourage them. When someone is presenting their own views, they can still steal someone else's... But there's going to be more breadth in these, and (hopefully) more substance in the essays. I personally don't consider essays to be useful at all - maybe they're appropriate in certain disciplines, but I'd argue the more concerned a professor or department is about language models, the less intellectual merit there is in the subject, or at least the work being used to assess understanding of the subject. Unfortunately, few educators focus on really trying to assess understanding and ability to apply understanding to new situations, but that's the kind of reasoning that generative models will struggle with. A very quick way to catch language model users out might be to ask them a bit about the same topic, then in a second paper give them their own initial assignment back and ask them to critically evaluate it now. Good luck critically evaluating hollow, vapid GPT waffle in timed exam conditions!
    - zekrioca 4y ago
      It’s astonishing that the word “generative” gets applied to technologies that are designed to duplicate, adapt, interpolate, or outright plagiarize without attribution in most cases. The skills trained into these codes do a lot of things and can produce impressive results, but when these results are traced to their sources, “generation” is at best a last-place description of what they do, and may be entirely absent.

by thdc 4y ago

filoleg 4y ago

What cheaters would do if ChatGPT didn't exist? They would probably be doing the exact same things they've been doing before ChatGPT existed. If you meant the OOP specifically, either they would cheat in some old-fashioned way or not cheat at all. Difficult to tell, especially since we don't know whether it was the first time they cheated.

g_p 4y ago

> I wonder what they would have done if ChatGPT didn't exist. This can be answered, as until late last year it effectively didn't for the average student. There were 3 main things they'd do (in my experience) for some kind of take-home assignment. In-room or at-home exam cheating are whole different subjects. 1. Pass notes or answers from year to year. Think shared Dropbox folder between cohorts, with papers and solutions to assignments. Simple fix - don't be lazy, make new graded assignments each year. Actually by doing this, students learn more as they practice the previous papers! Short circuit this by giving them the past assignments openly, and encouraging them to work together on them, then give them a fresh assignment to work on. They'll understand it and be in a good place to do it without cheating. 2. Use essay mills to pay someone to write them an "essay". These sites effectively sell bespoke written essays, marketed with pseudo legitimacy as "study materials". Some come with guarantees of uniqueness and that they won't trip plagiarism scanners as they haven't been given to anyone else. Colleges have access to tools that detect content submitted to other colleges previously, as well as content from the open web, so the essay mill pays other more advanced students or graduates to quickly whip up unique essays for paying students. Without ChatGPT, I can say with confidence this is what they would do, because it's what they did up until late last year. You could do stylistic analysis or similar to look for changes in writing style over multiple submissions if something seems "off" - with luck you'll find things like variations of spelling certain words, or specific writing style changes. Or the essay might be overly generic and missing the substance of the specific question topic. 3. Straightforward plagiarism via "copy paste", or "copy, reword a few unusual words, paste". Occasionally you get something more evolved like "copy, paraphrase, paste", or use of a thesaurus to swap words out (often inappropriately) to try avoid binary comparison detection. Fuzzy detection usually spits this though. Sometimes you'll simply get the exact same piece of work submitted by multiple students who have colluded and shared their answer with each other.

thdc 4y ago
Right, totally forgot essay mills were a thing. Originally, outside of plagiarism, I couldn't think of any ways to cheat on an essay. Part of writing an essay, or at least the type of essay I'm thinking of, is researching and criticizing other views on the subject, so if you find some material to draw from you might a well incorporate it properly. The other part is presenting your own thoughts - which I assume is where ChatGPT came into play. I'm not aware of anything that could handle this part before ChatGPT (or other LLMs) existed short of presenting other people's thoughts as your own (also plagiarism).
- g_p 4y ago
  Part of the challenge around essays is likely also what type of essay is sought, and the content required. Some educators are lazy, and just want something submitted with credible sounding words, parroting something suitably deferential to the accepted norms of the field. Those kinds of essays are the ones most likely to find themselves the subjects of tech or non tech enabled cheating. They're also effectively low signal to noise ratio measures, and I'd discourage them. When someone is presenting their own views, they can still steal someone else's... But there's going to be more breadth in these, and (hopefully) more substance in the essays. I personally don't consider essays to be useful at all - maybe they're appropriate in certain disciplines, but I'd argue the more concerned a professor or department is about language models, the less intellectual merit there is in the subject, or at least the work being used to assess understanding of the subject. Unfortunately, few educators focus on really trying to assess understanding and ability to apply understanding to new situations, but that's the kind of reasoning that generative models will struggle with. A very quick way to catch language model users out might be to ask them a bit about the same topic, then in a second paper give them their own initial assignment back and ask them to critically evaluate it now. Good luck critically evaluating hollow, vapid GPT waffle in timed exam conditions!
  - zekrioca 4y ago
    It’s astonishing that the word “generative” gets applied to technologies that are designed to duplicate, adapt, interpolate, or outright plagiarize without attribution in most cases. The skills trained into these codes do a lot of things and can produce impressive results, but when these results are traced to their sources, “generation” is at best a last-place description of what they do, and may be entirely absent.