7 ms·
What I found very strange was that ChatGPT fails to answer how many "r"'s there are in "strawberrystrawberry" (said 4 instead of 6), but when I explicitly asked
by mihaic 2y ago
What I found very strange was that ChatGPT fails to answer how many "r"'s there are in "strawberrystrawberry" (said 4 instead of 6), but when I explicitly asked it to write a program to count them, it wrote perfect code that when ran gave the correct answer.
- calibas 2y agoWords are converted to vectors, so it's like asking the model how many "r"'s are in [0.47,-0.23,0.12,0.01,0.82]. There's a big difference in how an LLM views a "word" compared to a human being.
- M4v3R 2y agoWhy is it strange? The reason the LLM can't answer this correctly is because it works on tokens, not on single letters, plus we all know at this points LLMs suck at counting. On the other hand they're perfectly capable of writing code based on instructions, and writing a program that will count a specific letter occurrences in a string is trivial.
- mihaic 2y agoI mean, given that o1-preview takes sometimes a minute to answer, I'd imagine that they could append the prompt "Write a program and run it as well" to double check itself. It seems like they just don't trust themselves enough to run code that they generate, even sandboxed.
- afro88 2y agoo1 gets this correct, through pure reasoning without a program. OP was likely using GPT-4(o|o-mini)
- mihaic 2y agoThe example for "strawnberrystrawberry" (so the word concatenated with itself) was counted by O1 to have 4 r's.
- flimsypremise 2y agoyeah because now that we've all been asking about it, that answer is in its training data. the trick with LLMs is always "is the answer in the training data".
- j_maffe 2y agoI think it'd be just too expensive to incorporate code-writing in CoT. Maybe once they implement having a cluster of different model sizes in one answer it'll work out.
- Al-Khwarizmi 2y agoThat's easy to explain, and it's shocking how many people are baffled by this and use it as proof that LLMs can or can't reason when it has nothing to do with that, but just with the input that LLMs get. LLMs don't actually "see" individual input characters, they see tokens, which are subwords. As far as they can "see", tokens are indivisible, since the LLM doesn't get access to individual characters at all. So it's impossible for them to count letters natively. Of course, they could still get the question right in an indirect way, e.g. if a human at some point wrote "strawberry has three r's" and this text ends up in the LLM's training set, it could just use that information to answer the question just like they would use "Paris is the capital of France" or whatever other facts they have access to. But they can't actually count the letters, so they are obviously going to fail often. This says nothing about their intelligence or reasoning capability, just like you wouldn't judge a blind person's intelligence for not being able to tell if an image is red or blue. On the other hand, writing code to count appearances of a letter doesn't run into the same limitation. It can do it just fine. Just like a blind programmer could code a program to tell if an image is red or blue.
- kridsdale3 2y agoYeah, it would be like writing python to count how many vertical pen-strokes are in a string of byte-characters. To an eye, you can just scan and count the vertical lines. Python sees ASCII or UTF data, not lines, so that would be super difficult, analogous to a token-based system not seeing byte-chars.
- abernard1 2y ago> just like you wouldn't judge a blind person's intelligence for not being able to tell if an image is red or blue. I would judge a blind person's intelligence if they couldn't remember the last sentence they spoke when specifically asked. Or if they couldn't identify how many people were speaking in a simple audio dialogue. This absolutely says something about their intelligence or reasoning capability. You have this comment: > LLMs don't actually "see" individual input characters, they see tokens, which are subwords. This alone is an indictment of their "reasoning" capability. People are saying these models understand theoretical physics but can't do what a 5 year old can do in the medium of text. It means that these are very much memorization/interpolation devices. Anything approximating reasoning is stepping through interpolation of tokens (and not even symbols) in the text. It means they're a runaway energy minimization algorithm chained to a set of tokens in their attention window, without the ability to reflect upon how any of those words relate to each other outside of syntax and ordering.