5 ms·

> "GPT-4 Is Incredibly Smart and Quite Dumb" Their evidence that it's 'quite dumb' consists of two prompts, one tricks it with a river crossing puzzle and anot

by ftxbro 4y ago

> "GPT-4 Is Incredibly Smart and Quite Dumb" Their evidence that it's 'quite dumb' consists of two prompts, one tricks it with a river crossing puzzle and another involves factorization. So yeah it's not an omniscient oracle. But I feel like that's a super short-sighted take. It's already shown that it's imperfect but nearly superhuman on puzzles, and improving at a fast rate on each successive version. Does the author think GPT-5 won't be able to solve those ones? Of course even if it can, there will be some harder puzzles that GPT-5 won't be able to solve. Will that mean it's 'quite dumb' too? Probably I'm thinking too much about it, and really the point of the blogpost is just to flex that they were able to fool the bot with two puzzles. So in that case well done!

famouswaffles 4y ago
There's a funny thing that happens in this field. The posts shift (which is well enough on its own) but then the conclusions stay with the old post. It's like, the definition of agi has obviously since shifted from general intelligence at the human level to somewhat superhuman (matching or exceeding human experts at all tasks) intelligence. But those people still think in terms of the old post. So they had all these "consequences" that would surely only happen with real agi years ago (at the time agi meant human level to them) but now those posts have changed, those "consequences" haven't. So now they're in this weird erroneous situation where x surely can't happen because agi surely hasn't been reached yet, forgetting x was a consequence of a lower bar. It's a form of short sightedness and a false sense of security.
- palad1n 4y ago
  I think you might be referring to one of the ideas of "AI Paradox". That whenever an AI problem has been solved well enough, it's not called "AI" anymore. Somewhere, I read that at one point, compiling a program was considered an AI task. Don't have the reference for that.
skybrian 4y ago
Predicting the future is pretty hard, especially when people are still inventing it. Debating GPT-5's capabilities is like arguing over whether Batman or Superman would win a fight. GPT-5 doesn't exist yet! It's not real. Why go there? The article is (mostly) about GPT-4. Understanding what we actually have is more useful in the short term.
snewman 4y ago
[author here] It's true that I didn't present a lot of concrete evidence of GPT-4's limitations. This is a blog post, not an academic paper. To my mind, the most concerning thing about GPT-4's performance on these two examples is not that it got the wrong answer, but that it utterly, utterly failed to understand that it was having difficulty. Even after repeated hints and prompts, it continues to make exactly the same mistakes, with no attempt to reason more carefully. There are plenty of other examples around of GPT-4 (to say nothing of earlier releases) having similar troubles. If you scroll down to the Further Reading section at the very end of the post, you'll find a reference to an extensive paper from Microsoft Research that comes to similar conclusions regarding GPT-4's limitations (I found this only after writing the initial draft of the post). For instance: > These examples illustrate some of the limitations of the next-word prediction paradigm, which manifest as the model’s lack of planning, working memory, ability to backtrack, and reasoning abilities. The model relies on a local and greedy process of generating the next word, without any global or deep understanding of the task or the output. Thus, the model is good at producing fluent and coherent texts, but has limitations with regards to solving complex or creative problems which cannot be approached in a sequential manner. The main point I tried to make is that GPT-4's "nearly superhuman" performance on a wide variety of tasks is somewhat illusory, and leans heavily on memorization. I spelled out some reasons why I think it looks more intelligent than it is. Relative to past work in AI, it is extremely impressive. Relative to the threshold required to perform economically useful work, it's... mixed; we're already seeing useful applications, but I think the majority of "information worker" tasks are still beyond it, and I'll go ahead and predict that the same statement will hold for GPT-5.
- nicpottier 4y ago
  I think your article is a worthy read and has some fun examples. I'm not so sure I agree with your conclusion though. The limitations you pose both in your article and here are real but already have answers in some cases. For one equipping GPT with upgrades for various tasks helps tremendously, IE, Wolfram Alpha for math, wikipedia to refresh its memory on facts etc.. I think you pointed out a really interesting edge case for the goat puzzle but also showed that with different words it CAN reason correctly but trusts it's memory more (and I correctly). I think you are setting far too high a bar with "majority of information worker tasks". My wild ass guess is that with some thoughtful design and focus as it stands today GPT-4 could do 25% of tasks (or make workers 25% more efficient). The economic impacts of that are vast and I'm not particularly optimistic about who will benefit most from them. That's what the fuss is about IMO, not that we have some kind of AGI on our hands, which is far too high a bar.
Turing_Machine 4y ago
> Their evidence that it's 'quite dumb' consists of two prompts, one tricks it with a river crossing puzzle and another involves factorization. We have other software tools to deal with arithmetic and logic -- quite good ones, in fact. GPT-x is for different purposes. It seems rather like complaining that a submarine is of little use for climbing a mountain.

by ftxbro 4y ago

famouswaffles 4y ago

There's a funny thing that happens in this field. The posts shift (which is well enough on its own) but then the conclusions stay with the old post. It's like, the definition of agi has obviously since shifted from general intelligence at the human level to somewhat superhuman (matching or exceeding human experts at all tasks) intelligence. But those people still think in terms of the old post. So they had all these "consequences" that would surely only happen with real agi years ago (at the time agi meant human level to them) but now those posts have changed, those "consequences" haven't. So now they're in this weird erroneous situation where x surely can't happen because agi surely hasn't been reached yet, forgetting x was a consequence of a lower bar. It's a form of short sightedness and a false sense of security.

palad1n 4y ago
I think you might be referring to one of the ideas of "AI Paradox". That whenever an AI problem has been solved well enough, it's not called "AI" anymore. Somewhere, I read that at one point, compiling a program was considered an AI task. Don't have the reference for that.

skybrian 4y ago

Predicting the future is pretty hard, especially when people are still inventing it. Debating GPT-5's capabilities is like arguing over whether Batman or Superman would win a fight. GPT-5 doesn't exist yet! It's not real. Why go there? The article is (mostly) about GPT-4. Understanding what we actually have is more useful in the short term.

snewman 4y ago

[author here] It's true that I didn't present a lot of concrete evidence of GPT-4's limitations. This is a blog post, not an academic paper. To my mind, the most concerning thing about GPT-4's performance on these two examples is not that it got the wrong answer, but that it utterly, utterly failed to understand that it was having difficulty. Even after repeated hints and prompts, it continues to make exactly the same mistakes, with no attempt to reason more carefully. There are plenty of other examples around of GPT-4 (to say nothing of earlier releases) having similar troubles. If you scroll down to the Further Reading section at the very end of the post, you'll find a reference to an extensive paper from Microsoft Research that comes to similar conclusions regarding GPT-4's limitations (I found this only after writing the initial draft of the post). For instance: > These examples illustrate some of the limitations of the next-word prediction paradigm, which manifest as the model’s lack of planning, working memory, ability to backtrack, and reasoning abilities. The model relies on a local and greedy process of generating the next word, without any global or deep understanding of the task or the output. Thus, the model is good at producing fluent and coherent texts, but has limitations with regards to solving complex or creative problems which cannot be approached in a sequential manner. The main point I tried to make is that GPT-4's "nearly superhuman" performance on a wide variety of tasks is somewhat illusory, and leans heavily on memorization. I spelled out some reasons why I think it looks more intelligent than it is. Relative to past work in AI, it is extremely impressive. Relative to the threshold required to perform economically useful work, it's... mixed; we're already seeing useful applications, but I think the majority of "information worker" tasks are still beyond it, and I'll go ahead and predict that the same statement will hold for GPT-5.

nicpottier 4y ago
I think your article is a worthy read and has some fun examples. I'm not so sure I agree with your conclusion though. The limitations you pose both in your article and here are real but already have answers in some cases. For one equipping GPT with upgrades for various tasks helps tremendously, IE, Wolfram Alpha for math, wikipedia to refresh its memory on facts etc.. I think you pointed out a really interesting edge case for the goat puzzle but also showed that with different words it CAN reason correctly but trusts it's memory more (and I correctly). I think you are setting far too high a bar with "majority of information worker tasks". My wild ass guess is that with some thoughtful design and focus as it stands today GPT-4 could do 25% of tasks (or make workers 25% more efficient). The economic impacts of that are vast and I'm not particularly optimistic about who will benefit most from them. That's what the fuss is about IMO, not that we have some kind of AGI on our hands, which is far too high a bar.

Turing_Machine 4y ago

> Their evidence that it's 'quite dumb' consists of two prompts, one tricks it with a river crossing puzzle and another involves factorization. We have other software tools to deal with arithmetic and logic -- quite good ones, in fact. GPT-x is for different purposes. It seems rather like complaining that a submarine is of little use for climbing a mountain.