4 ms·

The author's prompts are all perfectly understandable to humans and expressed in a direct and clear way, and the AIs can't understand them. This comment essent

The author's prompts are all perfectly understandable to humans and expressed in a direct and clear way, and the AIs can't understand them. This comment essentially says "the prompt is faulty because the AI didn't answer it correctly". This bakes the correctness of the AI into the tautology that it always answers "good" prompts, which are defined as those it answers correctly.

vba616 4y ago
Well, humans don't know how to determine if a proposition is correct. So it's no surprise we didn't teach AIs yet! Edit: I don't understand why experts say it failed when they "asked it to reason". It doesn't have pictures of TVs on cats to produce variations of, right? How does "reason" enter into it?
- civilized 4y ago
  There are different perspectives on how much intelligence these foundation models have. Some people think they just elaborate on or mash up their training data. Others think they have internal representations that allow for more human-like creativity. Maybe they learn from the examples what cat and TV and on top of mean, and can put those together in a way not seen in the training data.
meltyness 4y ago
Right but the author's premise is that the AI fails to reason or extend ideas, not that the AI fails to reason or extend ideas in the presence of extraneous, conflicting, or malformed instructions. If you're prompting ChatGPT (which is already loaded with prompt-engineering cheez-whiz mind you) to reason as a computer, why use words like "imagine"? why phrase things as a question? I think the fragility the author is observing here is extraneous to the actual underlying technology, and a factor of the UI that's been stacked-on and purpose-built for softer-purpose use, not hard academic research such as this. Case study, even GPT2 can reason about the stack https://transformer.huggingface.co/share/bQXbmeyBZD https://transformer.huggingface.co/share/bQXbmeyBZD
- civilized 4y ago
  I don't see any extraneous, conflicting, or malformed instructions. "Can reason" is a generous interpretation of "got the right answer in one instance".
  - meltyness 4y ago
    I mean unless you've got a better heuristic, but my conclusion is primarily structural with regard to Transformer units, basically that they're Turing-complete -- if there's anything more to reasoning and abstraction, I don't know myself, perhaps Scott Aaronson is the guy to talk to.

vba616 4y ago

Well, humans don't know how to determine if a proposition is correct. So it's no surprise we didn't teach AIs yet! Edit: I don't understand why experts say it failed when they "asked it to reason". It doesn't have pictures of TVs on cats to produce variations of, right? How does "reason" enter into it?

civilized 4y ago
There are different perspectives on how much intelligence these foundation models have. Some people think they just elaborate on or mash up their training data. Others think they have internal representations that allow for more human-like creativity. Maybe they learn from the examples what cat and TV and on top of mean, and can put those together in a way not seen in the training data.

meltyness 4y ago

Right but the author's premise is that the AI fails to reason or extend ideas, not that the AI fails to reason or extend ideas in the presence of extraneous, conflicting, or malformed instructions. If you're prompting ChatGPT (which is already loaded with prompt-engineering cheez-whiz mind you) to reason as a computer, why use words like "imagine"? why phrase things as a question? I think the fragility the author is observing here is extraneous to the actual underlying technology, and a factor of the UI that's been stacked-on and purpose-built for softer-purpose use, not hard academic research such as this. Case study, even GPT2 can reason about the stack https://transformer.huggingface.co/share/bQXbmeyBZD https://transformer.huggingface.co/share/bQXbmeyBZD

civilized 4y ago
I don't see any extraneous, conflicting, or malformed instructions. "Can reason" is a generous interpretation of "got the right answer in one instance".
- meltyness 4y ago
  I mean unless you've got a better heuristic, but my conclusion is primarily structural with regard to Transformer units, basically that they're Turing-complete -- if there's anything more to reasoning and abstraction, I don't know myself, perhaps Scott Aaronson is the guy to talk to.