3 ms·

Even if this is true now, it soon won't be. Eventually there won't be any way to tell.

by PrimeMcFly 3y ago

marginalia_nu 3y ago
Why is this true?
- ironSkillet 3y ago
  I think what will happen is that there effectively be an arms race between the bot generation and detection sides. As anomalies are discovered in the distribution of content generated by bots vs humans, bots will update to remove the anomaly. And researchers will try to find new anomalies. In the end as bots get better, the difference in the two distributions will be indistinguishable in practice.
  - marginalia_nu 3y ago
    Possible, sure. Likely, maybe. But you made it sound inevitable? Of that I am less sure.
    - vidarh 3y ago
      For any accessible detector you can automate the evasion. E.g, crudest variant would be: generate text run it through the detector finetune model w/result of the detector repeat until detector fails to detect. But every time the failure rate of the detector starts getting high, the other side has a lot of development work to do finding new patterns that won't also lead to lots of false positives. So while getting a point where it's totally impossible to detect may well take a long time, it'll be periods of the detector suddenly being hard to bypass for a while, and gradually getting easier, repeat. It's highly unlikely it'll ever be impossible to bypass for any length of time. The only way of keeping such a detector working for any length of time will be to make it expensive or very hard for adversaries to get access to, which means effectively relying on only giving access to trusted users.
      - _v7gu 3y ago
        Isn't this basically how autoencoder models are trained?
      - marginalia_nu 3y ago
        This assumes the ability to perfectly model this function, which is in a sense what we're setting out to demonstrate. It can certainly not be taken for granted.
        vidarh 3y ago
        Firstly, it doesn't need to perfectly model it. It needs to model it to within a range where claiming to have detected it causes more false positives than acceptable, and given the range of quality of human output that's a pretty wide span. But secondly, LLMs wired up with a feedback loop and and appropriate IO can model Markov models (models, not chains) which are Turing complete, so a LLM wired up this way is clearly Turing complete as well. As such it's a question of training, not whether or not the model can model it.
- PrimeMcFly 3y ago
  Because eventually LLM based AI will be advanced enough to author its text with as much variety and diversity as any human, making it indistinguishable from anything authored by a human.
  - rsynnott 3y ago
    That feels like a _huge_ assumption. It would arguably be quite surprising just based on the prior history of ML-y stuff; there's a tendency for things to get 95% good enough quickly, and then never quite make it to 100% (because 100% is really a completely different sort of problem, arguably requiring AGI). My favourite example of this: speech recognition. In the 90s this went quite quickly from a joke to a tech demo that required enormously expensive hardware to a tech demo that you could run on your laptop to actually marginally useful (you may remember Apple and particularly MS claiming that it was the future, and would soon be the primary way that people interacted with computers). And then, well... like, voice recognition today is certainly better than Dragon Dictate was 25 years ago, but you probably still wouldn't want to trust it with anything important, and it's certainly far worse at it than a human is. I'm struggling to think of _any_ ML-ish thing in the human-imitating space which has gotten all the way to virtually perfect/as good as human. Maybe OCR? Though even then, you still sometimes get odd results.
    - sebzim4500 3y ago
      >I'm struggling to think of _any_ ML-ish thing in the human-imitating space which has gotten all the way to virtually perfect/as good as human. Go?
      - rsynnott 3y ago
        Oh, yeah, actually, fair, a few games. Though, can an expert human player reliably determine that they're playing against a machine? I suspect they can, but I don't know.
        sebzim4500 3y ago
- cabalamat 3y ago
  Because the LLM can train itself against the AI detector to become undetectable.

Even if this is true now, it soon won't be. Eventually there won't be any way to tell.

marginalia_nu 3y ago

Why is this true?

ironSkillet 3y ago
I think what will happen is that there effectively be an arms race between the bot generation and detection sides. As anomalies are discovered in the distribution of content generated by bots vs humans, bots will update to remove the anomaly. And researchers will try to find new anomalies. In the end as bots get better, the difference in the two distributions will be indistinguishable in practice.
- marginalia_nu 3y ago
  Possible, sure. Likely, maybe. But you made it sound inevitable? Of that I am less sure.
  - vidarh 3y ago
    For any accessible detector you can automate the evasion. E.g, crudest variant would be: generate text run it through the detector finetune model w/result of the detector repeat until detector fails to detect. But every time the failure rate of the detector starts getting high, the other side has a lot of development work to do finding new patterns that won't also lead to lots of false positives. So while getting a point where it's totally impossible to detect may well take a long time, it'll be periods of the detector suddenly being hard to bypass for a while, and gradually getting easier, repeat. It's highly unlikely it'll ever be impossible to bypass for any length of time. The only way of keeping such a detector working for any length of time will be to make it expensive or very hard for adversaries to get access to, which means effectively relying on only giving access to trusted users.
    - _v7gu 3y ago
      Isn't this basically how autoencoder models are trained?
    - marginalia_nu 3y ago
      This assumes the ability to perfectly model this function, which is in a sense what we're setting out to demonstrate. It can certainly not be taken for granted.
      - vidarh 3y ago
        Firstly, it doesn't need to perfectly model it. It needs to model it to within a range where claiming to have detected it causes more false positives than acceptable, and given the range of quality of human output that's a pretty wide span. But secondly, LLMs wired up with a feedback loop and and appropriate IO can model Markov models (models, not chains) which are Turing complete, so a LLM wired up this way is clearly Turing complete as well. As such it's a question of training, not whether or not the model can model it.
PrimeMcFly 3y ago
Because eventually LLM based AI will be advanced enough to author its text with as much variety and diversity as any human, making it indistinguishable from anything authored by a human.
- rsynnott 3y ago
  That feels like a _huge_ assumption. It would arguably be quite surprising just based on the prior history of ML-y stuff; there's a tendency for things to get 95% good enough quickly, and then never quite make it to 100% (because 100% is really a completely different sort of problem, arguably requiring AGI). My favourite example of this: speech recognition. In the 90s this went quite quickly from a joke to a tech demo that required enormously expensive hardware to a tech demo that you could run on your laptop to actually marginally useful (you may remember Apple and particularly MS claiming that it was the future, and would soon be the primary way that people interacted with computers). And then, well... like, voice recognition today is certainly better than Dragon Dictate was 25 years ago, but you probably still wouldn't want to trust it with anything important, and it's certainly far worse at it than a human is. I'm struggling to think of _any_ ML-ish thing in the human-imitating space which has gotten all the way to virtually perfect/as good as human. Maybe OCR? Though even then, you still sometimes get odd results.
  - sebzim4500 3y ago
    >I'm struggling to think of _any_ ML-ish thing in the human-imitating space which has gotten all the way to virtually perfect/as good as human. Go?
    - rsynnott 3y ago
      Oh, yeah, actually, fair, a few games. Though, can an expert human player reliably determine that they're playing against a machine? I suspect they can, but I don't know.
      - sebzim4500 3y ago
cabalamat 3y ago
Because the LLM can train itself against the AI detector to become undetectable.