3 ms·

"As the AI generated content becomes dramatically overwhelming in scale, the human content will become increasingly easy to spot (and there will be multiple cue

by jerf 3y ago

"As the AI generated content becomes dramatically overwhelming in scale, the human content will become increasingly easy to spot (and there will be multiple cues to the human content that make it fairly obvious)" I feel like you're not accounting for the amount of that AI content that will be deliberately and intelligently intended to masquerade as human. Every signal you can think of and may start using is a signal that intelligent humans can and will forge. And they'll get the ones you didn't think of either, because that's their job. I don't even have to hedge or qualify this prediction, because we have decades of experience with people already forging every signal that is technically possible to forge, and pouring very substantial effort into doing so, right up to having dark businesses that provide these things as a service. "Forge all the signals that are technically possible to forge" is a product you can buy right now. For example, from 13 days ago: https://news.ycombinator.com/item?id=36151140 https://news.ycombinator.com/item?id=36151140 I don't know what signals you think you're going to see, but I'll guarantee A: yes, you will indeed see them because there's a lot of degrees of competence in the use of these systems (I still periodically get spams where the sender sent out their template rather than filling it in with a mail merge), but B: those will just be the ones you notice, not the entire population. Rather than a binary decision, I like to measure with "how much information does it take for me to detect a forgery". For instance, things like modern architecture renders can absolutely fool me as being real at standard definition, but at 4K they still struggle (too precise, too clean, even when they try to muss it up on purpose). I need a few paragraphs of GPT text to identify it conclusively, and that's just the default tone it takes. Ask it to give you some text in a specific style and I don't know how much it would take. For all I know I'm already missing things. You're probably already missing things you didn't realize.

willcipriano 3y ago
ChatGTP has a number of topics it will refuse to discuss so any writing on those topics will be more often than not human generated.
- jerf 3y ago
  As I mentioned in another post, OpenAI does not have any sort of lock on LLMs. There's already plenty in the wild and they're only going to get better. Already the commercial AI companies are expressing concern about how much faster the truly open stuff is developing than their stuff. The companies can still throw better hardware at the problem somewhat more reliably but that's not much of a moat; it's still commodity hardware, it's just a bit much to drop on just being able to LLM better if you're not being paid to LLM.
  - willcipriano 3y ago
    Do any publicly available services like this offer a uncensored bot? I have models I can run on my laptop without censorship but they aren't very high quality. I know they can exist but from what I can get my hands on it seems saying something offensive is a way to prove you are human for the time being.
    - sbierwagen 3y ago
      Llama 65B exists and you can just run it on a cloud GPU instance.
      - willcipriano 3y ago
        Ahh thank you, still that has a cost, legal and technical bar to cross but seems to meet the criteria.
        jerf 3y ago
        Yes, I do agree with you that it is nowhere near as trivial as it is to just pop open ChatGPT and ask it something right now, and at the moment, that matters. I'm looking more long term, like, a year out, five years out, and on that time frame I think it's very reasonable to expect to see ChatGPT-levels of quality being completely commoditized. OpenAI or whatever commercial entity may by then have the next level AI, but there's definitely a threshold where the combination of LLM + human ingenuity gets to the point where there are effectively no signals of humanity left. Moreover, I'm running on the theory that detecting LLMs is in fact not an arms race, but clearly ends in a win for LLMs. Basically, there exists text that simply looks human. It doesn't matter what level of superhuman AI you throw at it, it simply is in the space of "texts that a human was likely to generate". Once that level is attained, it won't matter that there is something that can generate superhuman text quality or something. (Or, put another way, if your clever "ride the AI wave" startup plan is "create a startup that can detect LLM usage", I advise you strongly to wargame out your 1 year and 5 year futures for that tech, especially in the face of deliberate attack. I would not be optimistic.)

by jerf 3y ago

willcipriano 3y ago

ChatGTP has a number of topics it will refuse to discuss so any writing on those topics will be more often than not human generated.

jerf 3y ago
As I mentioned in another post, OpenAI does not have any sort of lock on LLMs. There's already plenty in the wild and they're only going to get better. Already the commercial AI companies are expressing concern about how much faster the truly open stuff is developing than their stuff. The companies can still throw better hardware at the problem somewhat more reliably but that's not much of a moat; it's still commodity hardware, it's just a bit much to drop on just being able to LLM better if you're not being paid to LLM.
- willcipriano 3y ago
  Do any publicly available services like this offer a uncensored bot? I have models I can run on my laptop without censorship but they aren't very high quality. I know they can exist but from what I can get my hands on it seems saying something offensive is a way to prove you are human for the time being.
  - sbierwagen 3y ago
    Llama 65B exists and you can just run it on a cloud GPU instance.
    - willcipriano 3y ago
      Ahh thank you, still that has a cost, legal and technical bar to cross but seems to meet the criteria.
      - jerf 3y ago
        Yes, I do agree with you that it is nowhere near as trivial as it is to just pop open ChatGPT and ask it something right now, and at the moment, that matters. I'm looking more long term, like, a year out, five years out, and on that time frame I think it's very reasonable to expect to see ChatGPT-levels of quality being completely commoditized. OpenAI or whatever commercial entity may by then have the next level AI, but there's definitely a threshold where the combination of LLM + human ingenuity gets to the point where there are effectively no signals of humanity left. Moreover, I'm running on the theory that detecting LLMs is in fact not an arms race, but clearly ends in a win for LLMs. Basically, there exists text that simply looks human. It doesn't matter what level of superhuman AI you throw at it, it simply is in the space of "texts that a human was likely to generate". Once that level is attained, it won't matter that there is something that can generate superhuman text quality or something. (Or, put another way, if your clever "ride the AI wave" startup plan is "create a startup that can detect LLM usage", I advise you strongly to wargame out your 1 year and 5 year futures for that tech, especially in the face of deliberate attack. I would not be optimistic.)