3 ms·

Scott didn't understand Noam's complaints very well. He (and I [0], and many others) are pointing out that LLMs cannot think or perform reasoning or exhibit int

Scott didn't understand Noam's complaints very well. He (and I [0], and many others) are pointing out that LLMs cannot think or perform reasoning or exhibit intelligence. Not now, and not ever, because statistical counting of likely words is not intelligence. Lacking the ability to explain the fundamental reasoning concepts behind one's conclusions is a hallmark of machine "learning," which is why it isn't learning. It's word aggregation. The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. ChatGPT cannot even correctly perform math or logic, and those are fundamental to intelligence and intellectual development. Hilariously, we already have many programming languages which can do those things, so that's not much of an advance to rely on software that cannot. Aaronson's premise, that Noam Chomsky has nothing more than "sour grapes" to offer, is really puerile and uninformed. See Chomsky's original article [1] for yourself and then decide who is on the hype bandwagon and who is not. [0] https://medium.com/gitconnected/behind-the-curtain-understanding-the-magic-of-chatgpt-3bbd23f0fbb3 https://medium.com/gitconnected/behind-the-curtain-understan... [1] https://web.archive.org/web/20230308104809/https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html https://web.archive.org/web/20230308104809/https://www.nytim...

gnramires 4y ago
> The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. To be fair, ChatGPT is not a pure predictive LLM. It also undergoes RLHF, a reinfocement learning step which is much more similar to human learning, and allows it to develop a more human-like character, and seems to give it a pretty good grasp on truth(telling), ethics and context, certainly compared to the behavior of a raw predictive LLM with no prompt engineering. I recommend read this blog post for a more nuanced understanding of LLMs: https://astralcodexten.substack.com/p/janus-simulators https://astralcodexten.substack.com/p/janus-simulators
esperent 4y ago
> The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away I don't disagree with your premise. But this applies equally to human intelligence. There's no way to ever tell if the reality you experience is real or not. This is a fundamental fact of existence that we hand wave away so that we can continue to interact with and learn about the world. My favorite thought experiment on this is Boltzmann Brains. https://en.m.wikipedia.org/wiki/Boltzmann_brain https://en.m.wikipedia.org/wiki/Boltzmann_brain
cornholio 4y ago
It seems to me that much of both the hype and the hate around ChatGPT is a gut reaction from people who haven't tried to incorporate it in their workflow for at least a month, understand its limitations and strengths, and only then comment on its ineptitude or world changing abilities. The fact that it can instantly reply with a broad consensus of the ideas it has been fed is precisely what I need from it. When I ask it to write a function that performs a binary search in Rust with a given set of constraints, I'm not looking into original research or "truth", I want precisely the boring boilerplate I'm too lazy to write. For that use-case, it's invaluable and awe inspiring. It's limits are laughable too, try as I might I could not make it count the numbers in a string of '9's that fit in a 128bit integer. It hallucinated fictional names of tools that I should use, such as "cobol_to_rust". It seems clear to me that both camps have profound insights and LLM are both invaluable and wholly insufficient to reach any form of substantial "Artificial Intelligence".
- PaulHoule 4y ago
  People do a terrible job of binary search to the extent that it is towards the back of my toolbox. You can get eaten alive by off-by-one errors, mishandled integer overflows, etc. Human-written binary searches work most of the time but often fail for special inputs. I wouldn’t trust a machine that learned by reading 1000s of other people’s binary searches any more than I’d trust one I found off StackOverflow unless it came with a detailed explanation of the corner cases…. It is one to look up in an algorithms book, a recent algorithms book.
- timeon 4y ago
  I know that you are reacting to particular comment but Chomsky's article is not denying usefulness of ML in specific domains.
- Edmond 4y ago
  I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. Well...ChatGPT has been struggling. It generally struggles with anything subtle, no matter how simple. If it can't regurgitate based on what's slurped up from the internet it struggles. Do note, my question is simply meant to assume i am using a-w as my digits and "word" is a number formed via a positional number system. It is something a human with requisite knowledge will veer towards, ChatGPT doesn't. To your point, it is indeed an awe-inspiring tech for the thing it is good at, ie replacing the current crop of search engines.
7sidedmarble 4y ago
I am way, way, way far from an expert on AI. Not even particularly interested in it. But is it really fair to say these models just give you the next likeliest word based on training? If that were true shouldn't you be able to replicate what they can do with a really advanced Markov chain?

gnramires 4y ago

> The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. To be fair, ChatGPT is not a pure predictive LLM. It also undergoes RLHF, a reinfocement learning step which is much more similar to human learning, and allows it to develop a more human-like character, and seems to give it a pretty good grasp on truth(telling), ethics and context, certainly compared to the behavior of a raw predictive LLM with no prompt engineering. I recommend read this blog post for a more nuanced understanding of LLMs: https://astralcodexten.substack.com/p/janus-simulators https://astralcodexten.substack.com/p/janus-simulators

esperent 4y ago

> The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away I don't disagree with your premise. But this applies equally to human intelligence. There's no way to ever tell if the reality you experience is real or not. This is a fundamental fact of existence that we hand wave away so that we can continue to interact with and learn about the world. My favorite thought experiment on this is Boltzmann Brains. https://en.m.wikipedia.org/wiki/Boltzmann_brain https://en.m.wikipedia.org/wiki/Boltzmann_brain

cornholio 4y ago

It seems to me that much of both the hype and the hate around ChatGPT is a gut reaction from people who haven't tried to incorporate it in their workflow for at least a month, understand its limitations and strengths, and only then comment on its ineptitude or world changing abilities. The fact that it can instantly reply with a broad consensus of the ideas it has been fed is precisely what I need from it. When I ask it to write a function that performs a binary search in Rust with a given set of constraints, I'm not looking into original research or "truth", I want precisely the boring boilerplate I'm too lazy to write. For that use-case, it's invaluable and awe inspiring. It's limits are laughable too, try as I might I could not make it count the numbers in a string of '9's that fit in a 128bit integer. It hallucinated fictional names of tools that I should use, such as "cobol_to_rust". It seems clear to me that both camps have profound insights and LLM are both invaluable and wholly insufficient to reach any form of substantial "Artificial Intelligence".

PaulHoule 4y ago
People do a terrible job of binary search to the extent that it is towards the back of my toolbox. You can get eaten alive by off-by-one errors, mishandled integer overflows, etc. Human-written binary searches work most of the time but often fail for special inputs. I wouldn’t trust a machine that learned by reading 1000s of other people’s binary searches any more than I’d trust one I found off StackOverflow unless it came with a detailed explanation of the corner cases…. It is one to look up in an algorithms book, a recent algorithms book.
timeon 4y ago
I know that you are reacting to particular comment but Chomsky's article is not denying usefulness of ML in specific domains.
Edmond 4y ago
I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. Well...ChatGPT has been struggling. It generally struggles with anything subtle, no matter how simple. If it can't regurgitate based on what's slurped up from the internet it struggles. Do note, my question is simply meant to assume i am using a-w as my digits and "word" is a number formed via a positional number system. It is something a human with requisite knowledge will veer towards, ChatGPT doesn't. To your point, it is indeed an awe-inspiring tech for the thing it is good at, ie replacing the current crop of search engines.

7sidedmarble 4y ago

I am way, way, way far from an expert on AI. Not even particularly interested in it. But is it really fair to say these models just give you the next likeliest word based on training? If that were true shouldn't you be able to replicate what they can do with a really advanced Markov chain?