4 ms·

Is that even feasible? You can affect LLMs output by asking them to form response as some specific person with specific style. Apart some hidden secret sauce b

Is that even feasible? You can affect LLMs output by asking them to form response as some specific person with specific style. Apart some hidden secret sauce by the LLM provider, detection is always going to be an arms race.

mmooss 2y ago
> Is that even feasible? If you mean, is it feasible to empirically test the accuracy of detection software when the LLM output is so varied? That's an interesting question. Is there a way to create a representative sample of LLM output for testing purposes? That's an important question, not least because we'd also be able to define the distribution and range of outputs (unless there's a statistical way to create a sample without knowing those things?). It goes to the question of how deterministic is the output? How predictable, even statistically? Maybe we just work with a large collection of output. Or maybe we can create a sample of inputs (the prompts), and work from that.
- me_me_me 2y ago
  Well it all depends on how hard you want to cheat. If you know nothing about the topic you basically hack the format of the essay. If you know something you can prompt to insert some errors into logic or different perspective. Effectively making it somewhat your work too I don't think most common LLMs are deterministic. Especially if you work on the prompt. That is not just ask Write a synopsis of Romeo and Juliet. Out of curiosity I tried following > Write a synopsis of Romeo and Juliet > Rewrite it in style of a teenager, insert some slight grammar errors > Write in as if teenager was trying to sound formal > more formal > make a different take, more edgy and judgemental I have no idea how one would combat that. > Maybe we just work with a large collection of output. yes, But how do you get that large collection from a single student who uses non-default prompts? Though I wonder, are we not fighting kids using calculators in a math class?

mmooss 2y ago

> Is that even feasible? If you mean, is it feasible to empirically test the accuracy of detection software when the LLM output is so varied? That's an interesting question. Is there a way to create a representative sample of LLM output for testing purposes? That's an important question, not least because we'd also be able to define the distribution and range of outputs (unless there's a statistical way to create a sample without knowing those things?). It goes to the question of how deterministic is the output? How predictable, even statistically? Maybe we just work with a large collection of output. Or maybe we can create a sample of inputs (the prompts), and work from that.

me_me_me 2y ago
Well it all depends on how hard you want to cheat. If you know nothing about the topic you basically hack the format of the essay. If you know something you can prompt to insert some errors into logic or different perspective. Effectively making it somewhat your work too I don't think most common LLMs are deterministic. Especially if you work on the prompt. That is not just ask Write a synopsis of Romeo and Juliet. Out of curiosity I tried following > Write a synopsis of Romeo and Juliet > Rewrite it in style of a teenager, insert some slight grammar errors > Write in as if teenager was trying to sound formal > more formal > make a different take, more edgy and judgemental I have no idea how one would combat that. > Maybe we just work with a large collection of output. yes, But how do you get that large collection from a single student who uses non-default prompts? Though I wonder, are we not fighting kids using calculators in a math class?