5 ms·

We need validation of GPT detection tool accuracy (false/true & negative/positive), and in what circumstances, or they are useless. Without independent informat

by mmooss 2y ago

We need validation of GPT detection tool accuracy (false/true & negative/positive), and in what circumstances, or they are useless. Without independent information about whether something is GPT-written [1], there is no way to verify it, so they can't be verified on submitted student papers where the info is from people (students) with very strong incentives to lie (not wanting to be caught). Here are a couple references in the article, but I would like to see a review of this question: > Researchers at the University of Reading recently conducted a blind test in which ChatGPT-written answers were submitted through the university’s own examination system: 94% of the AI submissions went undetected and received higher scores than those submitted by the humans. > One study at Stanford found that a number of AI detectors have a bias towards non-English speakers, flagging their work 61% of the time, as opposed to 5% of native English speakers (Turnitin was not part of this particular study). [1] That needs to be defined: Every word? mostly? GPT-written then edited? What about ideas from the humans and writing by GPT? First draft by human and editing by GPT - what about using it as a grammar checker? For suggestions on clarifying some tricky passages? ?

me_me_me 2y ago
Is that even feasible? You can affect LLMs output by asking them to form response as some specific person with specific style. Apart some hidden secret sauce by the LLM provider, detection is always going to be an arms race.
- mmooss 2y ago
  > Is that even feasible? If you mean, is it feasible to empirically test the accuracy of detection software when the LLM output is so varied? That's an interesting question. Is there a way to create a representative sample of LLM output for testing purposes? That's an important question, not least because we'd also be able to define the distribution and range of outputs (unless there's a statistical way to create a sample without knowing those things?). It goes to the question of how deterministic is the output? How predictable, even statistically? Maybe we just work with a large collection of output. Or maybe we can create a sample of inputs (the prompts), and work from that.
  - me_me_me 2y ago
    Well it all depends on how hard you want to cheat. If you know nothing about the topic you basically hack the format of the essay. If you know something you can prompt to insert some errors into logic or different perspective. Effectively making it somewhat your work too I don't think most common LLMs are deterministic. Especially if you work on the prompt. That is not just ask Write a synopsis of Romeo and Juliet. Out of curiosity I tried following > Write a synopsis of Romeo and Juliet > Rewrite it in style of a teenager, insert some slight grammar errors > Write in as if teenager was trying to sound formal > more formal > make a different take, more edgy and judgemental I have no idea how one would combat that. > Maybe we just work with a large collection of output. yes, But how do you get that large collection from a single student who uses non-default prompts? Though I wonder, are we not fighting kids using calculators in a math class?

by mmooss 2y ago

me_me_me 2y ago

Is that even feasible? You can affect LLMs output by asking them to form response as some specific person with specific style. Apart some hidden secret sauce by the LLM provider, detection is always going to be an arms race.

mmooss 2y ago
> Is that even feasible? If you mean, is it feasible to empirically test the accuracy of detection software when the LLM output is so varied? That's an interesting question. Is there a way to create a representative sample of LLM output for testing purposes? That's an important question, not least because we'd also be able to define the distribution and range of outputs (unless there's a statistical way to create a sample without knowing those things?). It goes to the question of how deterministic is the output? How predictable, even statistically? Maybe we just work with a large collection of output. Or maybe we can create a sample of inputs (the prompts), and work from that.
- me_me_me 2y ago
  Well it all depends on how hard you want to cheat. If you know nothing about the topic you basically hack the format of the essay. If you know something you can prompt to insert some errors into logic or different perspective. Effectively making it somewhat your work too I don't think most common LLMs are deterministic. Especially if you work on the prompt. That is not just ask Write a synopsis of Romeo and Juliet. Out of curiosity I tried following > Write a synopsis of Romeo and Juliet > Rewrite it in style of a teenager, insert some slight grammar errors > Write in as if teenager was trying to sound formal > more formal > make a different take, more edgy and judgemental I have no idea how one would combat that. > Maybe we just work with a large collection of output. yes, But how do you get that large collection from a single student who uses non-default prompts? Though I wonder, are we not fighting kids using calculators in a math class?