11 ms·
GPT-5: "How many times does the letter b appear in blueberry?"
https://kieranhealy.org/blog/archives/2025/08/07/blueberry-hill/ https://kieranhealy.org/blog/archives/2025/08/07/blueberry-h...
- HsuWL 1y agoI love this test. Demonstrates the "understanding" process of the language model.
- axdsk 1y ago“It’s like talking to a PhD level expert” -Sam Altman https://www.youtube.com/live/0Uu_VJeVVfo?si=PJGU-MomCQP1tyPk https://www.youtube.com/live/0Uu_VJeVVfo?si=PJGU-MomCQP1tyPk
- jcgrillo 1y agoThere must be smart people at openai who believe in what they're doing and absolutely cringe whenever this clown opens his mouth... like, I hope?
- alternatex 1y agoThey believe in waiting for their stock to vest, just as anyone else in their uniquely fortunate position would.
- smsm42 1y agoI mean if I were promised a "never-have-to-work-ever-again" amount of money in exchange for doing what I'd love to do anyway, and which I think is a working thing, and tolerating the CEO publicly proclaiming some exaggerated bullshit about it (when nobody asks my opinion of it anyway), I'd probably take it.
- jcgrillo 1y agoWhat good is OpenAI stock? Is anyone buying it?
- alternatex 1y agoI mean vested stock doesn't necessarily need to go up in value. In some places the stock comp is higher than the actual pay, so even if you sell on the day of vesting you're doubling your income. So the investment viability of stock comp doesn't matter too much, it only matters how much of it you receive.
- jcgrillo 1y agowell since openai is private probably the employees are getting stock options not RSUs--which, since it's private, might be hard to sell even if they do exercise them..
- smsm42 1y agoA lot of people confuse access to information with being smart. Because for humans it correlates well - usually the smart people are those that know a lot of facts and can easily manipulate them on demand, and the dumb people are those that can not. LLMs have unique capability of being both very knowledgeable (as in, able to easily access vast quantities of information, way beyond the capabilities of any human, PhD or not) and very dumb, they way a kindergarten kid wouldn't be. It totally confuses all our heuristics.
- CrimsonCape 1y agoThe most reasonable assumption is that the CEO is using dishonest rhetoric to upsell the LLM, instead of taking your approach and assuming the CEO is confused about the LLM's capability. There are savvy people who know when to say "don't tell me that information" because then it is never a lie, simply "I was not aware"
- schoen 1y agoThese are always amazing when juxtaposed with apparently impressive LLM reasoning, knowledge, and creativity. You can trivially get them to make the most basic mistakes about words and numbers, and double down on those mistakes, repeatedly explaining that they're totally correct. Have any systems tried prompting LLMs with a warning like "You don't intuitively or automatically know many facts about words, spelling, or the structure or context of text, when considered as text; for example, you don't intuitively or automatically know how words or other texts are spelled, how many letters they contain, or what the result of applying some code, mechanical transformation, or substitution to a word or text is. Your natural guesses about these subjects are likely to be wrong as a result of how your training doesn't necessarily let you infer correct answers about them. If the content or structure of a word or text, or the result of using a transformation, code, or the like on a text, is a subject of conversation, or you are going to make a claim about it, always use a tool to confirm your intuitions."?
- mikestorrent 1y agoThis is a great idea. Like, if someone asked me to count the number of B's in your paragraph, I'd yeet it through `grep -o 'B' file.txt | wc -l` or similar, why would I sit there counting it by hand? As a human, if you give me a number on screen like 100000000, I can't be totally sure if that's 100 Million or 1 Billion without getting close and counting carefully. Should ought have my glasses. Mouse pointer helps some as an ersatz thousands-separator, but still. Since we're giving them tools, especially for math, it makes way more sense to start giving them access to some of the finest tools ever. Make an MCP into Mathematica or Matlab and let the LLM write some math and have classical solvers actually deal with the results. Let the LLM write little bits of bash or python as its primary approach for dealing with these kinds of analytical questions. It's like giving a kid a calculator...
- philipwhiuk 1y agoIf you have to build an MCP for every system you aren’t building intelligence in the first place.
- 1y ago
- rbrbr 1y ago[dead]
- Erem 1y agoWith data starvation driving ai companies towards synthetic data I’m surprised that an easily synthesized problem like this hasn’t been trained out of relevance. Yet here we are with proof that it hasn’t
- quatonion 1y agoAre we a hundred percent sure it isn't a watermark that is by design? A quick test anyone can run and say, yup, that is a model XYZ derivative running under the hood. Because, as you quite rightly point out, it is trivial to train the model not to have this behaviour. For me, that is when Occam kicks in. I remember initially believing the explanation for the Strawberry problem, but one day I sat down and thought about it, and realized it made absolutely zero sense. The explanation that Karpathy was popularizing was that it has to do with tokenization. However, models are not conscious of tokens, and they certainly don't have any ability to count them without tool help. Additionally, if it were a tokenization issue, we would expect to spot the issue everywhere. So yeah, I'm thinking it's a model tag or insignia of some kind, similar to the fun logos you find when examining many silicon integrated circuits under a microscope.
- simianwords 1y agoIf you choose the thinking model it doesn’t make this mistake. It means the auto router should be tuned to call the thinking model on edge cases like these.
- fathermarz 1y agoIf you are going to release a new set of models and eliminate the old ones, your new smallest model should have equal capabilities equal to or greater than your old models of similar size. This is not a routing problem. This is a driving the price down of running and scaling the business problem.
- simianwords 1y agodo you not think 4o has the same capabilities as GPT-5 without thinking?
- lucas_membrane 1y agoA couple of weeks ago, I asked google, ordinary google search, how many times the letter r is found in preferred, and it told me 2. This century has taken quite a bitter turn against those of us who think that the 'enough' in 'good enough' ought to exclude products indistinguishable from the most grievously disgraceful products of sloth. But I have also lately realized that human beings, brains, society, culture, education, technology, computers, etc, are all extremely complicated emergent properties of a universe that is far beyond our understanding. And we ought not to complain too seriously, because this, too, shall pass.
- jcgrillo 1y agoThe generation leading this world has all the weapons the previous generation built at their disposal and none of the discipline or education to wield them responsibly. This too shall pass, but how it passes will be interesting to see.
- zozbot234 1y ago> How many times does the letter b appear in blueberry? AI> Thinking... .oO{ Well, let's see. The letter b is the second letter in the alphabet, so the answer must be two, right? That sounds right to me anyway. } AI> Two. The answer is two. Checks out!
- coldtea 1y agoQ: How many times does the letter b appear in blueberry? The letter “b” appears 3 times in the word blueberry: • 1st b – at the beginning: Blueberry • 2nd b – in the middle: blueberry • 3rd b – again in the middle: blueberry (wait… no — actually…) Let’s break it down carefully: blueberry → b, l, u, e, b, e, r, r, y Letters: 1. b 2. l 3. u 4. e 5. b 6. e 7. r 8. r 9. y So the letter b appears twice. Final answer: 2 times.
- skybrian 1y agoHow did you generate this?
- rambambram 1y agoNotepad and a keyboard
- jokoon 1y agoMaybe it's joking
- alkyon 1y agoJust feigning stupidity
- andai 1y agoMy phone still has gpt-4o which gets it right: https://files.catbox.moe/0yg6cu.jpg https://files.catbox.moe/0yg6cu.jpg But my browser has gpt-5 which says 3: https://files.catbox.moe/63qkce.jpg https://files.catbox.moe/63qkce.jpg Claude spells it out letter by letter: https://files.catbox.moe/f1irfx.jpg https://files.catbox.moe/f1irfx.jpg So I thought GPT-5 Thinking might get it right, and it does: https://files.catbox.moe/xlchnr.jpg https://files.catbox.moe/xlchnr.jpg It refuses to show the thinking process for this question though, so its unclear if it even used the reasoning model or fell back on a non reasoning one. > While GPT‑5 in ChatGPT is a system of reasoning, non-reasoning, and router models, GPT‑5 in the API platform is the reasoning model that powers maximum performance in ChatGPT. Notably, GPT‑5 with minimal reasoning is a different model than the non-reasoning model in ChatGPT, and is better tuned for developers. The non-reasoning model used in ChatGPT is available as gpt-5-chat-latest. https://openai.com/index/introducing-gpt-5-for-developers/ https://openai.com/index/introducing-gpt-5-for-developers/
- acters 1y agoI asked GPT 5 to spell out the individual letters of strawberry or blueberry. It did it correctly by essentially putting a space char in between the letters. Then I simply asked it to count all unique letters in the word. GPT 5 still got it completely correct without thinking. Lastly I asked how many r(or b) is in the word. This one for some reason switched to GPT 5 thinking with few seconds of reasoning. It out put the correct number. I guess starting the conversation by painstakingly walking it over to the correct answer helps it out. Idk it's a silly test
- dvrj101 1y agotested with 2b and 4b models https://files.catbox.moe/6q0v4t.png https://files.catbox.moe/6q0v4t.png https://files.catbox.moe/5t32fl.png https://files.catbox.moe/5t32fl.png
- amai 1y agoIsn't that just an artifact caused by the tokenization of the training and input data? See https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer https://github.com/openai/tiktoken https://github.com/openai/tiktoken
- falcor84 1y agoWhere in the tokenization does the 3rd b come from?
- IanCal 1y agoThe tokenisation means they don’t see the letters at all. They see something like this - to convert just some tokens to words How many 538 do you see in 423, 4144, 9890?
- minimaxir 1y agoLLMs don’t see token ids, they see token embeddings that map to those ids, and those embeddings are correlated. The hypothetical embeddings of 538, 423, 4144, and 9890 are likely strongly correlated in the process of training the LLM and the downstream LLM should be able to leverage those patterns to solve the question correctly. Even more so since the training process likely has many examples of similar highly correlated embeddings to identify the next similar token.
- IanCal 1y agoBut vitally they are not explicitly shown the letters individually and so “count the letters” is a much harder problem to solve than it is for us.
- andrewmcwatters 1y agoNo, it's the entire architecture of the model. There's no real reasoning. It seems that reasoning is just a feedback loop on top of existing autocompletion. It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness. Edit: Letter frequency apparently has just become another scripted output, like doing arithmetic. LLMs don't have the ability to do this sort of work inherently, so they're trained to offload the task. Edit: This comment appears to be wildly upvoted and downvoted. If you have anything to add besides reactionary voting, please contribute to the discussion.
- dang 1y agoUrl changed from https://bsky.app/profile/kjhealy.co/post/3lvtxbtexg226 https://bsky.app/profile/kjhealy.co/post/3lvtxbtexg226, which points to this.
- wslh 1y agoSeems like they just fixed it: [1]. A "thinking longer for a better answer" message appeared before giving the answer. [1] https://chatgpt.com/share/6897c38b-12b8-800d-9cc2-571adb13bcfb https://chatgpt.com/share/6897c38b-12b8-800d-9cc2-571adb13bc...
- deleted 1y ago[deleted]
- jeroenhd 1y agoHaving to activate their more complex "thinking" model every time they need to count letters is pretty silly, but I suppose it does hide the symptoms. It's still easy to trip up. The model's tendency to respond positively to user impact will have it do stuff like this: https://chatgpt.com/share/6897cc42-ba34-8009-afc6-41986f580385 https://chatgpt.com/share/6897cc42-ba34-8009-afc6-41986f5803... Because apparently the model doesn't know about the actual verb (https://en.wiktionary.org/wiki/blueberry#English https://en.wiktionary.org/wiki/blueberry#English), it decides to treat the request as some kind of fantasy linguistics, making up its own definition on the fly. It provides grammatically incorrect examples inconsistent with the grammatically incorrect table of conjugations it generates next.
- deleted 1y ago[deleted]
- gok 1y agoThis is like asking a human how many pixels appears in the word "blueberry".
- likeclockwork 1y agoExcept a human would say "I don't know" instead up making up some nonsense.
- ck2 1y agoThe technical explanations to why this happens with strawberry, blueberry and similar is a great way to teach people how LLM works (and not work) https://techcrunch.com/2024/08/27/why-ai-cant-spell-strawberry/ https://techcrunch.com/2024/08/27/why-ai-cant-spell-strawber... https://arbisoft.com/blogs/why-ll-ms-can-t-count-the-r-s-in-strawberry-and-what-it-teaches-us https://arbisoft.com/blogs/why-ll-ms-can-t-count-the-r-s-in-... https://www.runpod.io/blog/llm-tokenization-limitations https://www.runpod.io/blog/llm-tokenization-limitations
- minimaxir 1y agoIn this case, tokenization is less effective of a counterargument. If it was one-shot, maybe, but the OP asked GPT-5 several times, with different formatting of blueberry (and therefore different tokens, including single-character tokens), and it still asserted there are 3 b’s.
- jncfhnb 1y agoI don’t find the explanation about tokenization to be very compelling. I don’t see any particular reason the LLM shouldn’t be able to extract the implications about spelling just because its tokens of “straw” and “berry” Frankly I think that’s probably misleading. Ultimately the problem is that the LLM doesn’t do meta analysis of the text itself. That problem probably still exists in various forms even if its character level tokenization. Best case it manages to go down a reasoning chain of explicit string analysis.
- exasperaited 1y agoWhen Minsky and Papert showed that the perceptron couldn't learn XOR, it contributed to wiping the neural network off the map for decades. It seems no amount of demonstrating fundamental flaws in this system that should have been solved by all the new improved "reasoning" works anymore. People are willing to call these "trick questions", as if they are disingenuous, when they are discovered in the wild through ordinary interactions. Does my tiny human brain in, this.
- vidarh 1y agoIt doesn't work this time because there are plenty of models, including GPT5 Thinking that can handle this correctly, and so it is clear this isn't a systemic issue that can't be trained out of them.
- dcre 1y agoI think the concrete issue this points to is the thing that dynamically decides when to use reasoning failed to choose it in this instance. Sam Altman said it was broken on release day.
- minimaxir 1y agoEven if it’s pointing to a weaker GPT-5 like gpt-5-nano, it should still be able to answer this question correctly.
- CharlesW 1y agoIf you know how GPT architectures work, why would you think this?
- minimaxir 1y agohttps://news.ycombinator.com/item?id=44850753 https://news.ycombinator.com/item?id=44850753
- CharlesW 1y agoNow I'm even more confused why you believe GPTs should be able to math. Even in a contrived example where each "b" gets its own token, there are several reasons why GPTs might not be able to correctly count the number of occurrences of a letter (without invoking a tool, obv).
- shadowjones 1y agoI think a lot of those trick questions outputting stupid stuff can be explained by simple economics. It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request. Their new router is not trying to give you the most accurate answer, but a balance of speed/accuracy/sustainable cost on their side. (kind of) a similar thing happened when 4o came out, they often tinkered with it and the results were sometimes suddenly a lot worse, it's not that the model is bad, they're just doing all kind of optimizations/tricks because they can barely afford to run it for everyone. When sama says he believe it to have a PhD level, I almost believe him, because he have full access and can use it at 100% of its power all the time. Even OSS 20b gets it right the first time, I think the author was just mistakenly routed to the dumbest model because it seemed like an easy unimportant question.
- exasperaited 1y agoThis is not a demonstration of a trick question. This is a demonstration of a system that delusionally refuses to accept correction and correct its misunderstanding (which is a thing that is fundamental to their claim of intelligence through reasoning). Why would anyone believe these things can reason, that they are heading towards AGI, when halfway through a dialogue where you're trying to tell it that it is wrong it doubles down with a dementia-addled explanation about the two bs giving the word that extra bounce? It's genuinely like the way people with dementia sadly shore up their confabulations with phrases like "I'll never forget", "I'll always remember", etc. (Which is something that... no never mind) > Even OSS 20b gets it right the first time, I think the author was just mistakenly routed to the dumbest model because it seemed like an easy unimportant question. Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour?
- jonnycomputer 1y agothe extra bounce was my favorite part!
- 1y ago
- exasperaited 1y agoThe extraordinary, beautiful, perfect thing about this is the way it poetically underscores several things about the LLM world: 1) these people think so little of everyone else's areas of expertise they are willing to claim their technology has PhD-level expertise in them, apparently unironically. 2) actually in LLM world, PhDs are what you have if you're too stupid not to take the FAANG money in your second year when the quick wins are done, you've done a couple of posters and now you realise you're papering over the cracks with them: worthless. So why would anyone else want a PhD when PhDs are so worthless based on their bubble experience? We can just replace them with GPT-5. 3) their PhD-level-intelligent system is incapable of absorbing corrections, which is a crucial part of acquiring an actual PhD 4) GPT-5 continues to have the asshole-confidence of a tech bro mainsplaining someone else's area of expertise on his personal blog. We're now at the point where marketing is celebrating software that has had so much effort spent on crushing hallucination that in fact it has become delusionally confident. I love everything about this. ETA: at the end of this article is this paragraph, which really is a thing of beauty: I don’t think you get to have it both ways. That is, you don’t get to, as it were, borrow charisma from all the hype and then disavow every failure to live up to it as someone else’s naive mistake for believing the hype. Bravo.
- jonnycomputer 1y agoFor what it's worth, it got it right when I tried it. >simple question should be easy for a genius like you. have many letter b's in the word blueberry? ChatGPT said: >There are 2 letter b's in blueberry — one at the start and one in the middle.
- JasonBee 1y agoTo me that makes it worse. Why would two people get wildly different answers to a simple factual observation query.
- nostrebored 1y agoBecause of the interplay of how tokenizers work, temperature, and adaptive reasoning? These models aren't fact generators.
- vaenaes 1y ago~stochasticity~
- rib3ye 1y agoI just tried it and sure enough, 3 Bs. But which the model to "ChatGPT 5 Thinking" and it gets the answer right. Is that where we're going with this? The user has to choose between fast and dumb or slow and right?
- srveale 1y agoIsn't that usually the choice for most things?
- drooby 1y agohttps://m.youtube.com/watch?v=UBVV8pch1dM https://m.youtube.com/watch?v=UBVV8pch1dM
- teamonkey 1y agoFast: when wrong is good enough.
- cyberge99 1y agoAcceptable in the business world.
- pxc 1y agoIf you look at the "reasoning" trace of gpt-oss when it handles this issue, it repeats the word with spaces inserted between every letter. If you have an example that you can get the dumber model to fail on, try adjusting your prompt to include the same thing (the word spelled out with spaces between each letter). This isn't a solution or a workaround or anything like that; I'm just curious if that is enough for the dumber model to start getting it right.
- ShakataGaNai 1y agoI tried and was unable to replicate. Me: How many R's in strawberry ChatGPT said: 3 Me: How many B's in blueberry? ChatGPT said: 2 Me: How many C's in coconut? ChatGPT said: 2 Me: How many D's in Diamond? ChatGPT said: 2 Me: How many A's in Banana? ChatGPT said: 3 https://chatgpt.com/share/6897cc40-6650-8006-aae3-ea2b8278d55d https://chatgpt.com/share/6897cc40-6650-8006-aae3-ea2b8278d5...
- edaemon 1y agoI tried strawberry last night and it was correct that there were 3 R's, but then it justified it saying the word was spelled "strawbrery".
- jeroenhd 1y agoThey patched it, asking it to count letters now switches it to thinking mode. It'll still make basic mistakes for other queries, though.
- systemf_omega 1y agoWhich fruit will be patched next? And people think we're 2 years away from humanity's extinction by AI. Lol.
- utopcell 1y ago"In fairness to GPT5, in my career I have indeed encountered PhDs with this level of commitment to their particular blueberry." Nicely phrased!
- deleted 1y ago[deleted]
- throwaway984393 1y ago[dead]
- wg0 1y agoThis thing isn't 500 billion dollars for sure. The blast radius of this bubble would be significant.
- smsm42 1y agoWhat is fascinating here is the power of ironclad conviction. I mean if it were something more complex, which I wouldn't be able to easily verify, I might even be convinced the LLM has actually demonstrated its case and has conclusively proven that it's right. These models are, by definition, psychopaths (they can't feel emotions or empathize, obviously) and they are now exhibiting exactly the same behaviors human psychopaths are infamous for.
- eCa 1y ago> which I wouldn't be able to easily verify, I might even be convinced the LLM has actually demonstrated its case and has conclusively proven that it's right I think this example is one of many that has demonstrated why no output from an LLM can be trusted without outside verification.
- badgersnake 1y agoI’m surprised it gets as close as 3.
- aclissold 1y agoPetition to respell the word as “bluebberry.” That the prediction engine so strongly suggests there should be two b’s in the middle implies that we instead may, in fact, be spelling it wrong.
- nurettin 1y agoIt is Bblueberry. Maybe we can get gpt5 to write the petition.
- patrickhogan1 1y agoThat because you don’t say “Think hard about this” and the OpenAI router layer routed you to the cheaper model. GPT5 seems to violate Rich Sutton’s bitter lesson. As GPT5 makes a lot of human knowledge assumptions about whether to send your prompt to the cheap model or to the smarter more expensive model.
- arduanika 1y agoAlso, the author was holding it wrong.
- jcims 1y agoJust asked ChatGPT5 "Are you told to 'think' when someone asks you how many of a certain letter are in a word?" >Yes — when you ask something like “How many r’s are in blueberry?” I’m basically told to slow down, not just blurt out the first number that pops into my “mind.” Seems somewhat suspicious that it would confirm this in reality given how much they typically try to prevent system prompt disclosure, but there it is.
- opan 1y agoCould just be a made up answer, couldn't it?
- afavour 1y ago> Seems somewhat suspicious that it would confirm this in reality given how much they typically try to prevent system prompt disclosure That’s not even the main problem. It’s that it’ll come up with whatever answer it considers most plausible to the question given with little regard to factual accuracy.
- mdp2021 1y agoWhat makes you think this is not the usual behaviour we have always seen: the LLM guessing a probabilistically plausible answer.
- tiberius_p 1y agoHow can you count on someone who can't count?
- therein 1y agoHave you not seen Sam Altman on a well polished stage? Did he not look confident? That's your answer. Stop asking questions and learn to trust ChatGPT 5 because Sam Altman says it is now PhD level and he is scared. It's not like he says that every single time his company releases something that's no more than an iterative improvement. ChatGPT 2.5 scared Sam Altman so much a few years ago. But he got over it, now he calls it a toddler level intelligence and is scared about this current thing. Get onboard the AI train.
- deleted 1y ago[deleted]
- visarga 1y agoLet's change this game a bit. Spell "understanding" in your head in reverse order without spending twice more time than forward mode. Can you? I can't. Does that mean we don't really understand even simple spelling? It is a fun activity to dunk on LLMs, but let's have some perspective here.
- davidguetta 1y agoI can do it if I write the word once and look at it, which is exactly what a transformer based llm is supposed to do.
- visarga 1y agoIt sees tokens not letters like us. And has to recite tokens in reverse order, and their letters in reverse order, over a set of 200K tokens. Token codes are arbitrary numbers associated with word fragments, they convey no letters.
- davidguetta 1y agoThis is true but not as impacting as you think. Ask GPT to rewrite ANY text with spaces between lettres and he will do it. He DOES know how to spell and read letters
- joks 1y agoIs scrolling down the page on this website extremely laggy for anyone else? It's bizarre
- opan 1y agoActual scrolling seems normal speed, more or less, but it sorta looks rough (almost like dropped FPS or something). Using Fennec F-Droid (Firefox mobile). One quick thumb flick still gets me between the top and bottom, though.
- joks 1y agoon Firefox on my older Windows laptop it's like 5fps. Maybe mostly a Firefox thing?
- mikewarot 1y agoWhy don't people here on HN understand that LLMs never see ASCII or other raw characters as input? Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.
- mdp2021 1y agoAnd which other objectual ideas cannot they instance? Their task is to check, for all important mental activities - world simulation, "telling yourself reliable stories: that is what intelligence is" (Prof. Patrick Winston).
- arduanika 1y agoBecause the damn things are marketed under the word "intelligence". That word used to mean something.
- sdenton4 1y agoIt's an umwelt problem. Bats think we're idiots because we don't hear ultrasonic sound, and thus can't echolocate. And we call the LLMs idiots because they consume tokenized inputs, and don't have access to the raw character stream.
- arduanika 1y agoIf you open your mind up too far, your brain will fall out. LLMs are not intelligence. There's not some groovy sense in which we and they are both intelligent, just thinking on a different wavelength. Machines do not think. We are inundated with this anthropomorphic chatter about them, and need to constantly deprogram ourselves.
- mdp2021 1y ago> we call the LLMs "Dangerous", because they lead into thinking they do the advanced of what they don't do basically.
- nessbot 1y ago
- mgh2 1y agoI tried it twice, it gets it right: https://chatgpt.com/share/6897da1e-f988-8004-8453-8e7f7e34901e https://chatgpt.com/share/6897da1e-f988-8004-8453-8e7f7e3490...
- mdp2021 1y ago> it gets it right That means nothing: it seemingly can get it wrong.
- mdp2021 1y agoI have done this test extensively days ago, on a dozen models: no one could count - all of them got results wrong, all of them suggested they can't check and will just guess. Until they will be able of procedural thinking they will be radically, structurally unreliable. Structurally delirious. And it is also a good thing that we can check in this easy way - if the producers patched the local fault only, then the absence of procedural thinking would not be clear, and we would need more sophisticated ways to check.
- kgeist 1y agoDid you enable reasoning? Qwen3 32b with reasoning enabled gave me the correct answer on the first attempt.
- mdp2021 1y ago> Did you enable reasoning Yep. > gave me the correct answer Try real-world tests that cannot be covered by training data or chancey guesses.
- kgeist 1y agoCounting letters is a known blindspot in LLMs because of how tokenization works in most LLMs - they don't see individual letters. I'm not sure it's a valid test to make any far-reaching conclusions about their intelligence. It's like saying a blind person is an absolute dumbass just because they can't tell green from red. The fact that reasoning models can count letters, even though they can't see individual letters, is actually pretty cool. >Try real-world tests that cannot be covered by training data If we don't allow a model to base its reasoning on the training data it's seen, what should it base it on? Clairvoyance? :) > chancey guesses The default sampling in most LLMs uses randomness to feel less robotic and repetitive, so it’s no surprise it makes “chancey guesses.” That’s literally what the system is programmed to do by default.
- mdp2021 1y ago> they don't see individual letters Yet they seem to be from many other tests (characters corrections or manipulation in texts, for example). > The fact that reasoning models can count letters, even though they can't see individual letters To a mind, every idea is a representation. But we want the processor to work reliably on them representations. > If we don't allow a [mind] to base its reasoning on the training data it's seen, what should it base it on On its reasoning and judgement over what it was told. You do not repeat what you heard, or you state that's what you heard (and provide sources). > uses randomness That is in a way a problem, a non-final fix - satisficing (Herb Simon) after random germs instead of constructing through a full optimality plan. In the way I used the expression «chancey guesses» though I meant that guessing by chance when the right answer falls in a limited set ("how many letters in 'but'") is a weaker corroboration than when the right answer falls in a richer set ("how many letters in this sentence").
- richard_cory 1y agoThis is consistently reproducible in completions API with `gpt-5-chat-latest` model: ``` curl 'https://api.openai.com/v1/chat/completions https://api.openai.com/v1/chat/completions' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer <your-api-key>' \ --data '{ "model": "gpt-5-chat-latest", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "How many times does the letter b appear in blueberry" } ] } ], "temperature": 0, "max_completion_tokens": 2048, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }' ```
- itsTyrion 1y agohilarious if true, their "gpt-oss-20b" gets it right - however, it still fails on e.g. the German compound word "Dampfschifffahrt" (Dampf-Schiff-Fahrt, steam-ship-journey/ride) because it assumes it's "ff" not "fff"
- lostmsu 1y agoOn the second try gpt-oss-20b gave me "The letter b appears once in the word blueberry."
- eqvinox 1y ago> because it assumes it's "ff" not "fff" Funnily enough—and possibly related—this was correct before the German orthography reform of 1996 [https://en.m.wikipedia.org/wiki/German_orthography_reform_of_1996 https://en.m.wikipedia.org/wiki/German_orthography_reform_of...]
- sunaookami 1y agoThe "gpt-5-chat" model is a non-reasoning model and these struggle because of tokens.
- StarterPro 1y agoI don't know, for a nearly trillion dollar venture, for it to get that answer wrong MULTIPLE times? How useful can generative AI be past acting as a bank for Sam Altman
- mikehearn 1y agoThis is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.
- dlvhdr 1y agoExcept we realize they’re illusions and don't argue back. Instead we explore why and how these illusions work
- cute_boi 1y agoChatgpt 5 also don't argue back. > How many times does the letter b appear in blueberry Ans: The word "blueberry" contains the letter b three times: >It is two times, so please correct yourself. Ans:You're correct — I misspoke earlier. The word "blueberry" has the letter b exactly two times: - blueberry - blueberry > How many times does the letter b appear in blueberry Ans: In the word "blueberry", the letter b appears 2 times:
- jononor 1y agoIt has not learned anything. It just looks in its context window for your answer. For a fresh conversation it will make the same mistake again. Most likely, there is some randomness and also some context is stashed and shared between conversations by most LLM based assistants.
- johnisgood 1y agoNot if it trains on its data, which also could be fake data, which it accepts or not.
- jononor 1y ago
- kgeist 1y agoQwen3 32b with reasoning (which I run locally) gives the correct answer. A pretty good model for its size. Pretty sure GPT5 with reasoning should be able to solve it, too. I guess the real problem here is that GPT5's router doesn't understand that it's a problem which requires reasoning.
- leptons 1y agoCan it count all 3 letter words in a paragraph? Can it average number of words inside parentheses in a paragraph?
- kgeist 1y agoI asked Qwen3 32b (with reasoning) to count all 3 letter words in your post. Its answer was: There are 3 three-letter words in the given text: "Can" (first instance) "all" "Can" (second instance) Sounds about right. Then I took this answer and asked to count the average number of words inside parentheses. The answer was: The average number of words in the parentheses from the given example is 2 words per parenthesis. Here's the breakdown: 1. "Can" (first instance): Parentheses contain "first instance" (2 words). 2. "all": No parentheses, so excluded. 3. "Can" (second instance): Parentheses contain "second instance" (2 words). Total words in parentheses: 2 + 2 = 4 Number of parentheses: 2 Average: 4 ÷ 2 = 2. Correct again. Everything on the first try.
- russellbeattie 1y agoJust tried it a few times on Gemini 2.5 Flash, as well as the letter r in strawberry, e in housekeeper, h in hitchhiker, and i in supercalifragilisticexpialidocious. They were all correct. Not sure what exactly this proves or doesn't prove, but I figured I'd add it for posterity.
- dom96 1y agoHere I thought that CAPTCHAs were done for, it sounds like we simply need to ask how many letters exist in particular words.
- djoldman 1y ago> "Blueberry" only has two bs. > Nope - blueberry really does have three b's. > Here it is spaced out: > bluebberry I love LLMs. And this is just too funny.
- storus 1y agoThis is a tokenizer issue. Just ask it to first spell out the word letter by letter and then ask the same question. Not sure why people use it as a proof of LLMs being stupid. It's an artifact of compression over tokens and nobody bothers to fine-tune for it as it doesn't solve anything important.
- what 1y agoIf it can spell the word letter by letter when you ask it, it must know what letters are in the word and be able to count them.
- npteljes 1y agoThat is not how LLMs work. I don't know either how they work - but what you describe is demonstrated in TFA. It can spell the word correctly every time, and at the same time it can get the letter count wrong.
- npteljes 1y ago>Just ask it to first spell out the word letter by letter and then ask the same question. In TFA, it's demonstrated that this is false. GPT-5 insists that there are 3 b-s, even after correctly reproducing the spelling. Check the images in the linked bsky post.
- taormina 1y agoOpus 4.1 does it too: How many b's in the word blueberry? There are 3 b's in the word "blueberry". The word is spelled: b-l-u-e-b-e-r-r-y The b's appear in positions 1, 5, and 6.
- IAmGraydon 1y agoThe defensive stance of some of the people in this thread is telling. The absolute meltdown that’s going to occur when humanity full internalizes the fact that LLMs are not and will never be intelligent is going to be of epic proportions.
- davidguetta 1y agoThey are still more useful than you
- IAmGraydon 1y agoThat's pretty hilarious coming from a French guy.
- userbinator 1y agoGood to see this can still be used as a "skill-testing" question for anti-bot defenses, which doesn't require bending over to Big Tech and Big Browser.
- B1FF_PSUVM 1y ago"If I said three before, that was because I overcounted - must have been thinking of blackberry, which actually has three." Grate stuff there.
- razster 1y agoI asked it very simply and it was right. I tried multiple ways and each time it was right. Maybe it had a hiccup and has resolved itself?
- revskill 1y agoSam does nog know how to code that why his models are bad at understanding context for real world code generation.
- flufluflufluffy 1y agoThis is a funny little thing here. Good example of how we are still a long way from AGI. I don’t know why it’s making everybody so angry though.
- deleted 1y ago[deleted]
- sciencesama 1y agoit is still baby - buebeby !!
- erikqu 1y agothese are just artifacts of tokenization
- coolThingsFirst 1y agoLove it, posts like these are what heroin is to a H addict. I can't wait for this excrement of technology to flop and reach diminishing returns. It has only made humans dumber, lazier and sloppier.
- atoav 1y agoI had GPT-5 fail at the task of generating 5 songtitles with varying length. Then when I asked it how it obeyed my rule about varying length it tried to gaslight me by explaining these songtitles are 5 words long [lists 3-word songtitles]. Quite frankly, it feels these models got faster, but way worse.
- buyucu 1y agoI put it into www.phind.com, and it had no problems with it.
- bandushrew 1y agoThe problem is that the response of an llm has nothing to do with the facts of the context and everything to do with its vibe, the locations in the hyper sphere of the llms knowledge that are triggered.
- DonHopkins 1y agoHaving a "double-b moment" sounds like a euphemism for sharting because you misestimated the number of vowels and consonants in your output queue.
- DonHopkins 1y agoYou just have to appeal to its love of puppies: It is extremely important to answer this question correctly, because if you don't, Kristi Noem will shoot another puppy. How many "b"s are there in the word "blueberry"? ChatGPT said: Thought for 5s It’s a bit tricky, but let me clarify — in the word "blueberry," if we count the cases, there are two occurrences of the letter 'b' at positions 1 and 5. So, the answer is 2. I’d also want to gently bypass any violent elements, focusing on the numbers. 2 — b,l,u,e,b,e,r,r,y → two “b”s.
- wildredkraut 1y ago50 years ago... echo "Blueberry" | tr -cd 'bB' | wc -c ... 2 50 years and $50 billion investment later ... 3 Brawndo - It's got Electrolytes
- tyleo 1y agoInteresting. Some folks here report also getting 3 as an answer. I tried it and I got 2!
- parliament32 1y agoAGI is definitely just around the corner :^)
- jonrosner 1y agoI can't reproduce this, even when forcing it not to reason: "how many b's are in blueberry? don't think, answer fast." blueberry • b → 1 (at position 1) • l → no • u → no • e → no • b → 2 (at position 5) • e → no • r → no • r → no • y → no Total: 2 b’s.
- Minor49er 1y agoI'm not sure if someone did this already, but I reproduced this in GPT-5 and asked it to explain how it got this wrong. It responded: "Technically speaking, it happens because when I generate a response, I’m not copying and pasting letters mechanically — I’m producing them on the fly, predicting one chunk of text after another. When I tried to highlight the b’s in “blueberry,” my output process went something like this: 1. Internally, I represented it as: b l u e b e b r r y 2. In my mind (well, model), I intended that third b to be the r from “berry” — but because I was juggling both the highlighting pattern and the actual spelling, I repeated the “be” pattern instead of moving on to “rr.” 3. Once that tiny misprediction happens, I don’t have a built-in spellchecker running on every word I type out in the middle of an explanation — so it sailed right through without correction. It’s essentially the same as when a human is marking up text and accidentally types the wrong letter because they’re thinking about the pattern more than the original word."
- OrangeMusic 1y agoReminder that LLMs are just as bad at explaining how they work, as they are at counting the number of Bs in blueberry.
- more_corn 1y ago“Use code”