3 ms·

It seems to me that much of both the hype and the hate around ChatGPT is a gut reaction from people who haven't tried to incorporate it in their workflow for at

It seems to me that much of both the hype and the hate around ChatGPT is a gut reaction from people who haven't tried to incorporate it in their workflow for at least a month, understand its limitations and strengths, and only then comment on its ineptitude or world changing abilities. The fact that it can instantly reply with a broad consensus of the ideas it has been fed is precisely what I need from it. When I ask it to write a function that performs a binary search in Rust with a given set of constraints, I'm not looking into original research or "truth", I want precisely the boring boilerplate I'm too lazy to write. For that use-case, it's invaluable and awe inspiring. It's limits are laughable too, try as I might I could not make it count the numbers in a string of '9's that fit in a 128bit integer. It hallucinated fictional names of tools that I should use, such as "cobol_to_rust". It seems clear to me that both camps have profound insights and LLM are both invaluable and wholly insufficient to reach any form of substantial "Artificial Intelligence".

PaulHoule 4y ago
People do a terrible job of binary search to the extent that it is towards the back of my toolbox. You can get eaten alive by off-by-one errors, mishandled integer overflows, etc. Human-written binary searches work most of the time but often fail for special inputs. I wouldn’t trust a machine that learned by reading 1000s of other people’s binary searches any more than I’d trust one I found off StackOverflow unless it came with a detailed explanation of the corner cases…. It is one to look up in an algorithms book, a recent algorithms book.
timeon 4y ago
I know that you are reacting to particular comment but Chomsky's article is not denying usefulness of ML in specific domains.
Edmond 4y ago
I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. Well...ChatGPT has been struggling. It generally struggles with anything subtle, no matter how simple. If it can't regurgitate based on what's slurped up from the internet it struggles. Do note, my question is simply meant to assume i am using a-w as my digits and "word" is a number formed via a positional number system. It is something a human with requisite knowledge will veer towards, ChatGPT doesn't. To your point, it is indeed an awe-inspiring tech for the thing it is good at, ie replacing the current crop of search engines.
- sebzim4500 4y ago
  To be fair, the way inputs are tokenised does make per character operations much harder than you would expect. If OpenAI had instead made one character one token, then it would be worse in most senses (slower and less history) but would do much better at that kind of task.
- YeGoblynQueenne 4y ago
  >> I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. For your intended use-case (as explained in your comment) shouldn't you be phrasing the second part of this specification as "treat the characters in "word" as if they are digits"? The way you worded it introduces some ambiguity. The ambiguity can be resolved by a process of elimination - I had to think of what could possibly happen if I "assumed the characters in 'word' are digits" and realise that it doesn't make sense, so it must mean you want to use the characters as digits. ChatGPT and LLMs in general really can't be expected to deal with that kind of ambiguity, not least because it's not the entire story (you also need another little logical jump, from "treat characters as digits" to "treat characters as denoting their positional values in the alphabet" or some such). I'm saying, you're asking ChatGPT to take many reasoning steps, when everyone is having a big debate about whether ChatGPT and LLMs can reason _at all_. To make any heads or tails of ChatGPT's abilities, one needs to start with simple examples, and then get to the more complex stuff. On the other hand, I expect that if you clarified your question more, or perhaps stated it just the right way, there's a good chance you'd get a correct answer. After all, the idea of counting with characters is very common and it's hard to believe ChatGPT hasn't seen a few thousands of examples of it during training.

PaulHoule 4y ago

People do a terrible job of binary search to the extent that it is towards the back of my toolbox. You can get eaten alive by off-by-one errors, mishandled integer overflows, etc. Human-written binary searches work most of the time but often fail for special inputs. I wouldn’t trust a machine that learned by reading 1000s of other people’s binary searches any more than I’d trust one I found off StackOverflow unless it came with a detailed explanation of the corner cases…. It is one to look up in an algorithms book, a recent algorithms book.

timeon 4y ago

I know that you are reacting to particular comment but Chomsky's article is not denying usefulness of ML in specific domains.

Edmond 4y ago

I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. Well...ChatGPT has been struggling. It generally struggles with anything subtle, no matter how simple. If it can't regurgitate based on what's slurped up from the internet it struggles. Do note, my question is simply meant to assume i am using a-w as my digits and "word" is a number formed via a positional number system. It is something a human with requisite knowledge will veer towards, ChatGPT doesn't. To your point, it is indeed an awe-inspiring tech for the thing it is good at, ie replacing the current crop of search engines.

sebzim4500 4y ago
To be fair, the way inputs are tokenised does make per character operations much harder than you would expect. If OpenAI had instead made one character one token, then it would be worse in most senses (slower and less history) but would do much better at that kind of task.
YeGoblynQueenne 4y ago
>> I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. For your intended use-case (as explained in your comment) shouldn't you be phrasing the second part of this specification as "treat the characters in "word" as if they are digits"? The way you worded it introduces some ambiguity. The ambiguity can be resolved by a process of elimination - I had to think of what could possibly happen if I "assumed the characters in 'word' are digits" and realise that it doesn't make sense, so it must mean you want to use the characters as digits. ChatGPT and LLMs in general really can't be expected to deal with that kind of ambiguity, not least because it's not the entire story (you also need another little logical jump, from "treat characters as digits" to "treat characters as denoting their positional values in the alphabet" or some such). I'm saying, you're asking ChatGPT to take many reasoning steps, when everyone is having a big debate about whether ChatGPT and LLMs can reason _at all_. To make any heads or tails of ChatGPT's abilities, one needs to start with simple examples, and then get to the more complex stuff. On the other hand, I expect that if you clarified your question more, or perhaps stated it just the right way, there's a good chance you'd get a correct answer. After all, the idea of counting with characters is very common and it's hard to believe ChatGPT hasn't seen a few thousands of examples of it during training.