3 ms·

I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the lar

by Edmond 4y ago

I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. Well...ChatGPT has been struggling. It generally struggles with anything subtle, no matter how simple. If it can't regurgitate based on what's slurped up from the internet it struggles. Do note, my question is simply meant to assume i am using a-w as my digits and "word" is a number formed via a positional number system. It is something a human with requisite knowledge will veer towards, ChatGPT doesn't. To your point, it is indeed an awe-inspiring tech for the thing it is good at, ie replacing the current crop of search engines.

sebzim4500 4y ago
To be fair, the way inputs are tokenised does make per character operations much harder than you would expect. If OpenAI had instead made one character one token, then it would be worse in most senses (slower and less history) but would do much better at that kind of task.
YeGoblynQueenne 4y ago
>> I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. For your intended use-case (as explained in your comment) shouldn't you be phrasing the second part of this specification as "treat the characters in "word" as if they are digits"? The way you worded it introduces some ambiguity. The ambiguity can be resolved by a process of elimination - I had to think of what could possibly happen if I "assumed the characters in 'word' are digits" and realise that it doesn't make sense, so it must mean you want to use the characters as digits. ChatGPT and LLMs in general really can't be expected to deal with that kind of ambiguity, not least because it's not the entire story (you also need another little logical jump, from "treat characters as digits" to "treat characters as denoting their positional values in the alphabet" or some such). I'm saying, you're asking ChatGPT to take many reasoning steps, when everyone is having a big debate about whether ChatGPT and LLMs can reason _at all_. To make any heads or tails of ChatGPT's abilities, one needs to start with simple examples, and then get to the more complex stuff. On the other hand, I expect that if you clarified your question more, or perhaps stated it just the right way, there's a good chance you'd get a correct answer. After all, the idea of counting with characters is very common and it's hard to believe ChatGPT hasn't seen a few thousands of examples of it during training.

by Edmond 4y ago

sebzim4500 4y ago

To be fair, the way inputs are tokenised does make per character operations much harder than you would expect. If OpenAI had instead made one character one token, then it would be worse in most senses (slower and less history) but would do much better at that kind of task.

YeGoblynQueenne 4y ago

>> I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be. For your intended use-case (as explained in your comment) shouldn't you be phrasing the second part of this specification as "treat the characters in "word" as if they are digits"? The way you worded it introduces some ambiguity. The ambiguity can be resolved by a process of elimination - I had to think of what could possibly happen if I "assumed the characters in 'word' are digits" and realise that it doesn't make sense, so it must mean you want to use the characters as digits. ChatGPT and LLMs in general really can't be expected to deal with that kind of ambiguity, not least because it's not the entire story (you also need another little logical jump, from "treat characters as digits" to "treat characters as denoting their positional values in the alphabet" or some such). I'm saying, you're asking ChatGPT to take many reasoning steps, when everyone is having a big debate about whether ChatGPT and LLMs can reason _at all_. To make any heads or tails of ChatGPT's abilities, one needs to start with simple examples, and then get to the more complex stuff. On the other hand, I expect that if you clarified your question more, or perhaps stated it just the right way, there's a good chance you'd get a correct answer. After all, the idea of counting with characters is very common and it's hard to believe ChatGPT hasn't seen a few thousands of examples of it during training.