3 ms·

If the text is good, and someday it will be, I don't care if an LLM wrote it. If it's bad, I don't care if a person wrote it. The only reason to care is that t

by etwigg 3y ago

If the text is good, and someday it will be, I don't care if an LLM wrote it. If it's bad, I don't care if a person wrote it. The only reason to care is that the implicit proof-of-work signal has broken because LLM text is so cheap. Open forums might need to be pay-per-submission someday...

Jedd 3y ago
The problem as I see it is that 'text is good' has two distinct meanings - first, it doesn't sound like an AI wrote it, it's interesting, entertaining, in word 'readable'. The second is that it's accurate / true. It feels like we're happy to take the first as a surrogate for the second, or at least being good at the first drops our guard on questioning the second.
neilv 3y ago
If you need to understand what a particular person is thinking, then text they wrote themself can tell you a lot about that.
jameshart 3y ago
Exactly. We are at frame 3 of https://xkcd.com/810/ https://xkcd.com/810/: "But what will you do when spammers train their bots to make automated constructive and helpful comments?" We don't need ML-generated-text detectors. We need BS detectors. If they have false positives and trigger on human-generated BS, that's just a great side-benefit. "This comment could have been written by an AI" is good enough reason to exclude it. That which does not need to be said, need not be said. As that XKCD's title text says: "And what about all the people who won't be able to join the community because they're terrible at making helpful and constructive co-- ... oh."
layer8 3y ago
If you only care about the text, maybe. But you may also care about the person who claims to have written the text. That being said, automatic detection seems like a lost cause.
- isaacfung 3y ago
  First of all I dont think we should aim at 100% precision and 100% recall. Instead it is more realistic to use the result as a filter for more downstream testings and treat it as a mean to increase the cost of cheating. Also we can use more than just the text output. A human writer doesn't generate a piece of text in one pass. Instead they go through the drafting and editing process. We can design devices to capture keypress or pen stroke(iirc ther were studies on fraud detection based on keypress patterns/mouse movment). One can attempt to train a new model to mimic themselves so we need to somehow make sure that the amount of training data required is too much to be worth the effort. For downstream testing, the goal isn't mainly to verify whether a piece of text is AI genereted but to make sure that a student who can pass the test would essentially have to know the material sufficient well(so this defeats the purpose of cheating).
rossjudson 3y ago
I'm thinking that it's a fundamental legal right to not have to read anything written by an LLM. In other words, machine-generated text shouldn't be legally binding in any way...mostly because there's infinite amounts of it. And it should be illegal to return machine-generate text in response to a discovery request. Request: "Disclose all documents written between apr 1 and apr 12 regarding topics x,y,z." Response: "There are 12 billion documents matching those parameters. Here they are."
- rossjudson 3y ago
  For good measure: Why bother storing documents at all? Instead, train an LLM on the documents, and keep only a list of questions you want to be able answer. Request: "Please respond with all documents related to X,Y,Z". Response: "There are none."

by etwigg 3y ago

Jedd 3y ago

The problem as I see it is that 'text is good' has two distinct meanings - first, it doesn't sound like an AI wrote it, it's interesting, entertaining, in word 'readable'. The second is that it's accurate / true. It feels like we're happy to take the first as a surrogate for the second, or at least being good at the first drops our guard on questioning the second.

neilv 3y ago

If you need to understand what a particular person is thinking, then text they wrote themself can tell you a lot about that.

jameshart 3y ago

Exactly. We are at frame 3 of https://xkcd.com/810/ https://xkcd.com/810/: "But what will you do when spammers train their bots to make automated constructive and helpful comments?" We don't need ML-generated-text detectors. We need BS detectors. If they have false positives and trigger on human-generated BS, that's just a great side-benefit. "This comment could have been written by an AI" is good enough reason to exclude it. That which does not need to be said, need not be said. As that XKCD's title text says: "And what about all the people who won't be able to join the community because they're terrible at making helpful and constructive co-- ... oh."

layer8 3y ago

If you only care about the text, maybe. But you may also care about the person who claims to have written the text. That being said, automatic detection seems like a lost cause.

isaacfung 3y ago
First of all I dont think we should aim at 100% precision and 100% recall. Instead it is more realistic to use the result as a filter for more downstream testings and treat it as a mean to increase the cost of cheating. Also we can use more than just the text output. A human writer doesn't generate a piece of text in one pass. Instead they go through the drafting and editing process. We can design devices to capture keypress or pen stroke(iirc ther were studies on fraud detection based on keypress patterns/mouse movment). One can attempt to train a new model to mimic themselves so we need to somehow make sure that the amount of training data required is too much to be worth the effort. For downstream testing, the goal isn't mainly to verify whether a piece of text is AI genereted but to make sure that a student who can pass the test would essentially have to know the material sufficient well(so this defeats the purpose of cheating).

rossjudson 3y ago

I'm thinking that it's a fundamental legal right to not have to read anything written by an LLM. In other words, machine-generated text shouldn't be legally binding in any way...mostly because there's infinite amounts of it. And it should be illegal to return machine-generate text in response to a discovery request. Request: "Disclose all documents written between apr 1 and apr 12 regarding topics x,y,z." Response: "There are 12 billion documents matching those parameters. Here they are."

rossjudson 3y ago
For good measure: Why bother storing documents at all? Instead, train an LLM on the documents, and keep only a list of questions you want to be able answer. Request: "Please respond with all documents related to X,Y,Z". Response: "There are none."