5 ms·

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end

by feoren 3y ago

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier. So such a model is doomed from the start, unless its parameters are a closely-guarded secret (and never leaked). Then it means it's foolable by those with access and nobody else. Which means there's a huge incentive for adversaries to make their own, etc. etc. until it's just a big arms race. It's clear the actual answer needs to be: we need better automated tools to detect quality content, whatever that might mean, whether written by a human or an AI. That would be a godsend. And if it turned into an arms race, the arms we're racing each other to build are just higher-quality content.

tessierashpool 3y ago
> You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier could you contextualize your use of the word "easily" here? I feel like "easily" might mean "with infinite funds and frictionless spherical developers."
- alexeldeib 3y ago
  GANs are established engineering. Infinite funds and frictionless spheres aside, you don't need to break ground, but copy/paste/glue existing code with some comprehension. LLMs are newer than GANs afaik, it just so happens GANs are a good fit here, not that one is "smarter" or "dumber".
asddubs 3y ago
The whole problem with AI is that it's able to copy some of the superficial indicators of quality content while feeding you lies. You cannot detect quality content without detecting truthfulness. Any heuristic you use in place of that can be copied without actually providing value (which is exactly what ChatGPT does now, when it gets things wrong)
- chinchilla2020 3y ago
  That's the whole problem with LLMs in general. They are designed to be convincing, not necessarily accurate.
  - leeoniya 3y ago
    thats my take on all this LLM hype: they're great at creative work where accuracy is not important, and assisting in technical work where the user is already able to discern an answer that is accurate from one that is slightly to fully bullshit. even if an LLM can give an amazing and correct answer 7/10 times, it still takes a human expert to cherrypick which 7 answers are amazing and which are just convincingly-assembled bs.
    - josephg 3y ago
      Yeah - but there’s a lot of domains where that is fine. I’m in France at the moment and I’ve been using chatgpt as a tour guide. I’m sure some of what it says is wrong, but I don’t honestly care. It’s also fantastic for teaching. I’ve been doing some self study lately and it’s been helping me to figure out what I should spend time learning and help direct my self study sessions toward what will help. I listened to an interview with the StabilityAI founder / ceo the other day. He said we should think about LLMs like having a bunch of clever grad students / interns floating around that we can freely offload tasks to. They aren’t experts, but they’re very diligent. The question is, how can we effectively make use of them? People who succeed at this will be much more productive.
      - PawgerZ 3y ago
        Can you explain how you use it for teaching/study? I also have used it to learn but with mixed results. Recently, I've been asking it to write me outlines so I can have somewhat of a learning plan.
        josephg 3y ago
        I'm learning AI at the moment. I gave ChatGPT the following prompt: > Write a training plan for a series of lessons to teach someone modern deep learning. The training plan should last for approximately 3 months of lessons. > The lesson plan is for a single student with a strong background in programming (systems programming, algorithms and web). But the student has little knowledge of python. And university level mathematics knowledge but relatively weak skills in linear algebra, probability and statistics. > By the end of the training process, the student should know modern deep learning methods and techniques and be able to modify, implement and deploy AI based systems. > Think through your answer. Start by listing out learning objectives, then write a teaching plan to meet those learning objectives. The response from chatgpt was super long! It gave me recommendations for what to study each week for the next 3 months. I've started going through the material it recommended. For the first 2 weeks, my goal is to learn the basics of python, and learn some linear algebra, and probability and statistics. Then its just a case of finding appropriate material online. I'm watching a lecture series on youtube teaching matrix mathematics now. [1] https://chat.openai.com/share/d6966012-0d96-4511-b96e-086b8001837f https://chat.openai.com/share/d6966012-0d96-4511-b96e-086b80...
dreamcompiler 3y ago
You just outlined an excellent proof of what might be called the AI Halting Problem.
sobellian 3y ago
Even without a GAN, it's quite possible that one could simply write `while(rejects(output)) { output = gpt(prompt) }` and obtain a sufficient output after a reasonable number of iterations.

by feoren 3y ago

tessierashpool 3y ago

> You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier could you contextualize your use of the word "easily" here? I feel like "easily" might mean "with infinite funds and frictionless spherical developers."

alexeldeib 3y ago
GANs are established engineering. Infinite funds and frictionless spheres aside, you don't need to break ground, but copy/paste/glue existing code with some comprehension. LLMs are newer than GANs afaik, it just so happens GANs are a good fit here, not that one is "smarter" or "dumber".

asddubs 3y ago

The whole problem with AI is that it's able to copy some of the superficial indicators of quality content while feeding you lies. You cannot detect quality content without detecting truthfulness. Any heuristic you use in place of that can be copied without actually providing value (which is exactly what ChatGPT does now, when it gets things wrong)

chinchilla2020 3y ago
That's the whole problem with LLMs in general. They are designed to be convincing, not necessarily accurate.
- leeoniya 3y ago
  thats my take on all this LLM hype: they're great at creative work where accuracy is not important, and assisting in technical work where the user is already able to discern an answer that is accurate from one that is slightly to fully bullshit. even if an LLM can give an amazing and correct answer 7/10 times, it still takes a human expert to cherrypick which 7 answers are amazing and which are just convincingly-assembled bs.
  - josephg 3y ago
    Yeah - but there’s a lot of domains where that is fine. I’m in France at the moment and I’ve been using chatgpt as a tour guide. I’m sure some of what it says is wrong, but I don’t honestly care. It’s also fantastic for teaching. I’ve been doing some self study lately and it’s been helping me to figure out what I should spend time learning and help direct my self study sessions toward what will help. I listened to an interview with the StabilityAI founder / ceo the other day. He said we should think about LLMs like having a bunch of clever grad students / interns floating around that we can freely offload tasks to. They aren’t experts, but they’re very diligent. The question is, how can we effectively make use of them? People who succeed at this will be much more productive.
    - PawgerZ 3y ago
      Can you explain how you use it for teaching/study? I also have used it to learn but with mixed results. Recently, I've been asking it to write me outlines so I can have somewhat of a learning plan.
      - josephg 3y ago
        I'm learning AI at the moment. I gave ChatGPT the following prompt: > Write a training plan for a series of lessons to teach someone modern deep learning. The training plan should last for approximately 3 months of lessons. > The lesson plan is for a single student with a strong background in programming (systems programming, algorithms and web). But the student has little knowledge of python. And university level mathematics knowledge but relatively weak skills in linear algebra, probability and statistics. > By the end of the training process, the student should know modern deep learning methods and techniques and be able to modify, implement and deploy AI based systems. > Think through your answer. Start by listing out learning objectives, then write a teaching plan to meet those learning objectives. The response from chatgpt was super long! It gave me recommendations for what to study each week for the next 3 months. I've started going through the material it recommended. For the first 2 weeks, my goal is to learn the basics of python, and learn some linear algebra, and probability and statistics. Then its just a case of finding appropriate material online. I'm watching a lecture series on youtube teaching matrix mathematics now. [1] https://chat.openai.com/share/d6966012-0d96-4511-b96e-086b8001837f https://chat.openai.com/share/d6966012-0d96-4511-b96e-086b80...

dreamcompiler 3y ago

You just outlined an excellent proof of what might be called the AI Halting Problem.

sobellian 3y ago

Even without a GAN, it's quite possible that one could simply write `while(rejects(output)) { output = gpt(prompt) }` and obtain a sufficient output after a reasonable number of iterations.