[author here] It's true that I didn't present a lot of concrete evidence of GPT-4's limitations. This is a blog post, not an academic paper. To my mind, the m

[author here] It's true that I didn't present a lot of concrete evidence of GPT-4's limitations. This is a blog post, not an academic paper. To my mind, the most concerning thing about GPT-4's performance on these two examples is not that it got the wrong answer, but that it utterly, utterly failed to understand that it was having difficulty. Even after repeated hints and prompts, it continues to make exactly the same mistakes, with no attempt to reason more carefully. There are plenty of other examples around of GPT-4 (to say nothing of earlier releases) having similar troubles. If you scroll down to the Further Reading section at the very end of the post, you'll find a reference to an extensive paper from Microsoft Research that comes to similar conclusions regarding GPT-4's limitations (I found this only after writing the initial draft of the post). For instance: > These examples illustrate some of the limitations of the next-word prediction paradigm, which manifest as the model’s lack of planning, working memory, ability to backtrack, and reasoning abilities. The model relies on a local and greedy process of generating the next word, without any global or deep understanding of the task or the output. Thus, the model is good at producing fluent and coherent texts, but has limitations with regards to solving complex or creative problems which cannot be approached in a sequential manner. The main point I tried to make is that GPT-4's "nearly superhuman" performance on a wide variety of tasks is somewhat illusory, and leans heavily on memorization. I spelled out some reasons why I think it looks more intelligent than it is. Relative to past work in AI, it is extremely impressive. Relative to the threshold required to perform economically useful work, it's... mixed; we're already seeing useful applications, but I think the majority of "information worker" tasks are still beyond it, and I'll go ahead and predict that the same statement will hold for GPT-5.