4 ms·

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text

by yorwba 3mo ago

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent to "This looks like spam; no human in their right mind would write this spam by hand if they could get an LLM to write it, therefore it's most likely written by an LLM."

onestay42 3mo ago
Not all humans are in their right minds, unfortunately.
- cwmoore 3mo ago
  It is much harder to tell one from the other, and for oneself, than it often seems on the surface.
- skinfaxi 3mo ago
  This is exactly the point I saw in a recent x post, that building anti-bot detection was incredibly difficult because some people exhibit bot like behavior. Blizzard employee once told me anti-botting in WoW was extremely challenging due to the number of real people that acted identically to bots. Every assumption was invalidated: - unbelievable # of consecutive hours played - consistently repetitive patterns of movement and clicks - farming patterns that aren’t considered fun (“why would anyone do that”) - solo, no external engagement - goes on for months The problem with botting is many humans ARE bots https://x.com/IceSolst/status/2076372992959959493 https://x.com/IceSolst/status/2076372992959959493
  - inigyou 3mo ago
    How do they know those were real people? Were they livestreaming their face and talking about what they were doing the whole time?
janalsncm 3mo ago
You definitely can rule out the general case a priori. If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible. As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the labels conflict for the same piece of text. The data is statistically inseparable which makes it impossible to classify from text alone.
- vlovich123 3mo ago
  It’s worse. If the data was separable in this way, you would equally be able to train an AI to mask those signs.
  - ben_w 3mo ago
    IIRC, the big names in LLMs have no real interest in cloaking the LLM-nature of the text, Google adds deliberate watermarks to text, OpenAI developed a watermark for text but reportedly arent't actually using it.
    - vlovich123 3mo ago
      Considering [1], I’m going to challenge that their techniques are currently even mildly effective. Given the absolute academic malpractice these papers are pushing, I’m calling BS; while they want to watermark it, they clearly aren’t actually able to. For images. Which are drastically easier than text. Their interest is irrelevant in the face of technical impossibility. And that’s before you get into other people who don’t care and will just build adversarial tools to bypass the attempted watermarks. It’s a losing useless battle. Google and OpenAI engage in it to try to catch competitors when there’s a lawsuit or to try to clean their datasets clean. But it’s absolutely unusable for something like “did someone cheat”. [1] https://hackerfactor.com/blog/index.php?/categories/1-Image-Analysis https://hackerfactor.com/blog/index.php?/categories/1-Image-...
      - 3mo ago
sigmoid10 3mo ago
That's like saying whether or not you're going to fall in love this year is just one bit of information, so you might be able to read it from astrology. Yeah, sure, it might happen for some people with a certain star sign. But across the population there is zero reason to believe that there is a) any significant correlation and b) enough data variation in to even distinguish classes of humans.
- lemagedurage 3mo ago
  The 80% accuracy from the article would be one reason to believe there's significant correlation, no?
  - sigmoid10 3mo ago
    I could probably find quite a lot of people who will tell you astrology is 80+% correct for them. Would you believe them or wait for an independent analysis? There are other AI "detector" systems out there that claim 99% accuracy. But independent research always found that they are actually garbage once used on real data. It's all in how you pick your tests. It's also funny to see how people on places like HN will easily dismiss stuff astrology, but fall for the exact same patterns when used in tech-y applications.
    - lemagedurage 3mo ago
      Well I don't think the position of planets when you're born has a large correlation to how your life will go. On the other hand, how an AI writes will have a big correlation to whether the written text would likely be written by an AI. The latter is more of a direct relationship. Maybe A and B are not correlated, and Y and Z are? What pattern are people falling for here?
      - strgcmc 3mo ago
        It's not really about the planets. It's about what other people believe about the planets. You could work for a boss that's a Leo, and he/she believes only Leos deserve to get promoted, or that Leos and Scorpios should never be assigned together on a shared project. Your life and career trajectory under this boss could be totally different, depending on whether you were born a Leo or not. Certainly "not all bosses" applies, but it's not really that farfetched or uncommon either. The point is that a correlation does exist, but it's a social one and not a physical or astronomical one (and it's also often a self-reinforcing/self-fulfilling one: Leos who read what astrology says Leos should do, may end up choosing to behave more like that). So in the LLM example, it may not really matter much what physical markers of provenance or physical correlations there are, as social beliefs or perceptions about suspected provenance may be the strongest correlation anyways (in terms of impact and outcomes).
grayhatter 3mo ago
> Whether a text was written by a human or not is just a single bit of information I doubt this models reality well at all. If I write the first paragraph, and AI writes the second; a float seems to model that better. If you choose to collapse a float into a bool, I don't think you can make useful conclusions based on that bit? > since even the shortest text contains more information than that. I also don't think that's how information theory and bits of information works...
chungusamongus 3mo ago
>As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Come on, that's circular reasoning.
jibal 3mo ago
> Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. This is word salad, a complete non sequitur. > to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all) Er, that's obviously not true.

by yorwba 3mo ago

onestay42 3mo ago

Not all humans are in their right minds, unfortunately.

cwmoore 3mo ago
It is much harder to tell one from the other, and for oneself, than it often seems on the surface.
skinfaxi 3mo ago
This is exactly the point I saw in a recent x post, that building anti-bot detection was incredibly difficult because some people exhibit bot like behavior. Blizzard employee once told me anti-botting in WoW was extremely challenging due to the number of real people that acted identically to bots. Every assumption was invalidated: - unbelievable # of consecutive hours played - consistently repetitive patterns of movement and clicks - farming patterns that aren’t considered fun (“why would anyone do that”) - solo, no external engagement - goes on for months The problem with botting is many humans ARE bots https://x.com/IceSolst/status/2076372992959959493 https://x.com/IceSolst/status/2076372992959959493
- inigyou 3mo ago
  How do they know those were real people? Were they livestreaming their face and talking about what they were doing the whole time?

janalsncm 3mo ago

You definitely can rule out the general case a priori. If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible. As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the labels conflict for the same piece of text. The data is statistically inseparable which makes it impossible to classify from text alone.

vlovich123 3mo ago
It’s worse. If the data was separable in this way, you would equally be able to train an AI to mask those signs.
- ben_w 3mo ago
  IIRC, the big names in LLMs have no real interest in cloaking the LLM-nature of the text, Google adds deliberate watermarks to text, OpenAI developed a watermark for text but reportedly arent't actually using it.
  - vlovich123 3mo ago
    Considering [1], I’m going to challenge that their techniques are currently even mildly effective. Given the absolute academic malpractice these papers are pushing, I’m calling BS; while they want to watermark it, they clearly aren’t actually able to. For images. Which are drastically easier than text. Their interest is irrelevant in the face of technical impossibility. And that’s before you get into other people who don’t care and will just build adversarial tools to bypass the attempted watermarks. It’s a losing useless battle. Google and OpenAI engage in it to try to catch competitors when there’s a lawsuit or to try to clean their datasets clean. But it’s absolutely unusable for something like “did someone cheat”. [1] https://hackerfactor.com/blog/index.php?/categories/1-Image-Analysis https://hackerfactor.com/blog/index.php?/categories/1-Image-...
    - 3mo ago

sigmoid10 3mo ago

That's like saying whether or not you're going to fall in love this year is just one bit of information, so you might be able to read it from astrology. Yeah, sure, it might happen for some people with a certain star sign. But across the population there is zero reason to believe that there is a) any significant correlation and b) enough data variation in to even distinguish classes of humans.

lemagedurage 3mo ago
The 80% accuracy from the article would be one reason to believe there's significant correlation, no?
- sigmoid10 3mo ago
  I could probably find quite a lot of people who will tell you astrology is 80+% correct for them. Would you believe them or wait for an independent analysis? There are other AI "detector" systems out there that claim 99% accuracy. But independent research always found that they are actually garbage once used on real data. It's all in how you pick your tests. It's also funny to see how people on places like HN will easily dismiss stuff astrology, but fall for the exact same patterns when used in tech-y applications.
  - lemagedurage 3mo ago
    Well I don't think the position of planets when you're born has a large correlation to how your life will go. On the other hand, how an AI writes will have a big correlation to whether the written text would likely be written by an AI. The latter is more of a direct relationship. Maybe A and B are not correlated, and Y and Z are? What pattern are people falling for here?
    - strgcmc 3mo ago
      It's not really about the planets. It's about what other people believe about the planets. You could work for a boss that's a Leo, and he/she believes only Leos deserve to get promoted, or that Leos and Scorpios should never be assigned together on a shared project. Your life and career trajectory under this boss could be totally different, depending on whether you were born a Leo or not. Certainly "not all bosses" applies, but it's not really that farfetched or uncommon either. The point is that a correlation does exist, but it's a social one and not a physical or astronomical one (and it's also often a self-reinforcing/self-fulfilling one: Leos who read what astrology says Leos should do, may end up choosing to behave more like that). So in the LLM example, it may not really matter much what physical markers of provenance or physical correlations there are, as social beliefs or perceptions about suspected provenance may be the strongest correlation anyways (in terms of impact and outcomes).

grayhatter 3mo ago

> Whether a text was written by a human or not is just a single bit of information I doubt this models reality well at all. If I write the first paragraph, and AI writes the second; a float seems to model that better. If you choose to collapse a float into a bool, I don't think you can make useful conclusions based on that bit? > since even the shortest text contains more information than that. I also don't think that's how information theory and bits of information works...

chungusamongus 3mo ago

>As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Come on, that's circular reasoning.

jibal 3mo ago

> Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. This is word salad, a complete non sequitur. > to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all) Er, that's obviously not true.