3 ms·

Can modern humans count the number of tokens in "blueberry"? (You're not allowed to cheat and use https://platform.openai.com/tokenizer https://platform.openai

by rwmj 1y ago

Can modern humans count the number of tokens in "blueberry"? (You're not allowed to cheat and use https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer)

ryanmcbride 1y ago
I see this argument every time this particular failing is brought up and like, regardless of the reasoning, it's still completely failing to do something a second grader should be able to do and until it stops failing in that way, being like "Um technically it's a lot harder than you think" isn't going to win over AI skeptics.
- strbean 1y ago
  A Chinese second grader would probably not be able to answer "How many Rs are in the word 草莓?" It's a valid critique.
  - ryanmcbride 1y ago
    You're right they probably wouldn't be able to to it, but that child isn't being marketed as an invaluable business tool to assist doctors in diagnosing cancer, or whatever.
    - strbean 1y ago
      Sure. A non-English speaking adult wouldn't be able to answer the question either, after the question was translated to their language. Maybe you wouldn't want a non-English speaker helping you to write an acrostic in English. Luckily nobody is marketing LLMs as "great for designing word puzzles" though.
      - ryanmcbride 1y ago
        You can make comparisons all you want but you can't change the fact that it just plain looks bad.
dgfitz 1y ago
Uh, who cares? That is literally useless knowledge.
- strbean 1y ago
  When's the last time you needed to know the number of Rs in "blueberry"?
  - crtasm 1y ago
    When's the last time you needed to count the number of things inside a larger thing?
    - strbean 1y ago
      What's your point? Parent said "That's literally useless knowledge" in response to not knowing the number of tokens in "blueberry". Are we talking about the specific question or the general notion of counting? The point is that LLMs don't speak / think in English. Asking them about spelling is like asking a Chinese speaker, during a text chat with translation, about English spelling. We can give the Chinese speaker access to an app to translate back to English so they can answer these questions. But they (the LLM) don't currently have access to that.
      - dgfitz 1y ago
        > The point is that LLMs don't speak / think in English. They don't think or speak anything at all. They use a statistical model to predict the next most likely token to display in response to a prompt.
        strbean 1y ago
        Distinction without a difference. s/speak \/ think/operate/ If that makes you happy.
        1y ago
bgwalter 1y ago
One is an unambiguous question, the other depends on the tokenizing algorithm.
xigoi 1y ago
No, but I can recognize my inability to do so instead of making the answer up.

by rwmj 1y ago

Can modern humans count the number of tokens in "blueberry"? (You're not allowed to cheat and use https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer)

ryanmcbride 1y ago

I see this argument every time this particular failing is brought up and like, regardless of the reasoning, it's still completely failing to do something a second grader should be able to do and until it stops failing in that way, being like "Um technically it's a lot harder than you think" isn't going to win over AI skeptics.

strbean 1y ago
A Chinese second grader would probably not be able to answer "How many Rs are in the word 草莓?" It's a valid critique.
- ryanmcbride 1y ago
  You're right they probably wouldn't be able to to it, but that child isn't being marketed as an invaluable business tool to assist doctors in diagnosing cancer, or whatever.
  - strbean 1y ago
    Sure. A non-English speaking adult wouldn't be able to answer the question either, after the question was translated to their language. Maybe you wouldn't want a non-English speaker helping you to write an acrostic in English. Luckily nobody is marketing LLMs as "great for designing word puzzles" though.
    - ryanmcbride 1y ago
      You can make comparisons all you want but you can't change the fact that it just plain looks bad.

dgfitz 1y ago

Uh, who cares? That is literally useless knowledge.

strbean 1y ago
When's the last time you needed to know the number of Rs in "blueberry"?
- crtasm 1y ago
  When's the last time you needed to count the number of things inside a larger thing?
  - strbean 1y ago
    What's your point? Parent said "That's literally useless knowledge" in response to not knowing the number of tokens in "blueberry". Are we talking about the specific question or the general notion of counting? The point is that LLMs don't speak / think in English. Asking them about spelling is like asking a Chinese speaker, during a text chat with translation, about English spelling. We can give the Chinese speaker access to an app to translate back to English so they can answer these questions. But they (the LLM) don't currently have access to that.
    - dgfitz 1y ago
      > The point is that LLMs don't speak / think in English. They don't think or speak anything at all. They use a statistical model to predict the next most likely token to display in response to a prompt.
      - strbean 1y ago
        Distinction without a difference. s/speak \/ think/operate/ If that makes you happy.
        1y ago

bgwalter 1y ago

One is an unambiguous question, the other depends on the tokenizing algorithm.

xigoi 1y ago

No, but I can recognize my inability to do so instead of making the answer up.