5 ms·

Why bother testing though? I was hoping this topic has finally died recently, but no. Someone's still interested in testing LLMs for something they're explicitl

Why bother testing though? I was hoping this topic has finally died recently, but no. Someone's still interested in testing LLMs for something they're explicitly not designed for and nobody is using them for this in practice. I really hope one day openai will just add a "when asked about character level changes, insights and encodings, generate and run a program to answer it" to their system so we can never hear about it again...

IncreasePosts 1y ago
Wouldn't a llm that just tokenized by character be good at it?
- curioussquirrel 1y ago
  Yes, but it would hurt its contextual understanding and effectively reduce the context window several times.
  - viraptor 1y ago
    Only in the current most popular architectures. Mamba and RWKV style LLMs may suffer a bit but don't get a reduced context in the same sense.
    - curioussquirrel 1y ago
      You're right. There was also an experiment in Meta which tokenized bytes directly and it didn't hurt performance much in very small models.
- typpilol 1y ago
  I asked this in another thread and it would only be better with unlimited compute and memory. Because without those, then the llm has to encode way more parameters and way smaller context windows. In a theoretical world, it would be better, but might not be much better.
neerajsi 1y ago
https://www.anthropic.com/news/analysis-tool https://www.anthropic.com/news/analysis-tool Seems like they already built this capability.
tkgally 1y ago
One reason for testing this is that it might indicate how accurately models can explain natural language grammar, especially for agglutinative and fusional languages, which form words by stringing morphemes together. When I tested ChatGPT a couple of years ago, it sometimes made mistakes identifying the components of specific Russian and Japanese words. I haven’t run similar tests lately, but it would be nice to know how much language learners can depend on LLM explanations about the word-level grammars of the languages they are studying. Later: I asked three LLMs to draft such a test. Gemini’s [1] looks like a good start. When I have time, I’ll try to make it harder, double-check the answers myself, and then run it on some older and newer models. [1] https://g.co/gemini/share/5eefc9aed193 https://g.co/gemini/share/5eefc9aed193
- gizmo686 1y ago
  What you are testing for is fundamentally different than character level text manipulation. A major optimization in modern LLMs is tokenization. This optimization is based on the assumption that we do not care about character level details, so we can combine adjacent characters into tokens, then train and run the main AI model on smaller strings built out of a much larger dictionary of tokens. Given this architecture, it is impressive that AIs can perform character level operations at all. They essentially need to reverse engineer the tokenization process. However, morphemes are semantically meaningful, so a quality tokenizer will tokenize at the morpheme level, instead of the word level. [0]. This is of particuarly obvious importance in Japanese, as the lack of spaces between words means that the naive "tokenize on whitespace" approach is simply not possible. We can explore the tokenizer of various models here: https://huggingface.co/spaces/Xenova/the-tokenizer-playground https://huggingface.co/spaces/Xenova/the-tokenizer-playgroun... Looking at the words in your example, we see the tokenization of the Gemma model (closely related to Gemini) is: un-belie-vably dec-entral-ization bio-degradable mis-understanding anti-dis-establishment-arian-ism пере-писы-ваться pere-pis-y-vat-'-s-ya до-сто-примеча-тельность do-stop-rime-chat-el-'-nost-' пре-по-дава-тель-ница бе-зо-т-вет-ственности bezotvetstvennosti же-лез-нодоро-жный z-hele-zn-odoro-zh-ny-y 食べ-させ-られた-くな-かった tab-es-aser-are-tak-unak-atta) 図書館 tos-ho-kan 情報-技術 j-ō-h-ō- gij-utsu 国際-関係 kok-us-ai- kan-kei 面白-くな-さ-そうだ Further, the training data that is likely to be relevent in this type of query probably isolates the individual morphemes while talking about a bunch of words that the use them; so it is a much shorter path for the AI to associate these close but not quite morphene tokens with the actual sequence of tokens that corresponds to what we think of as a morphene. [0] Morpheme level tokenization is itself a non-trivial problem. However, has been pretty well solved long before the current generation of AI.
redox99 1y ago
Character level LLMs are used for detecting insults and toxic chat in video games and the like.
- minimaxir 1y ago
  Can you give an example of a video game explicitly using character-level LLMs? There were prototypes of char-rnns back in the day for chat moderation but it has significant compute overhead.
  - redox99 1y ago
    It's something I heard through the grapevine. But there's only a few big enough competitive games where toxicity is such a big deal, so it's not hard to guess. Character level helps with players disguising insults. Compute wise it's basically the same, but multiply token count by 4. Which doesn't really matter for short chat in video games.
- jazzyjackson 1y ago
  I figure an LLM would be way better at classifying insults than regexing against a bad word list. Why would character level be desirable?
  - vanviegen 1y ago
    I'd imagine for simplicity - just skip the tokenizer and feed bytes.
  - duskwuff 1y ago
    Might a character-level LLM be better at recognizing poorly spelled (or deliberately misspelled) profanity?
- viraptor 1y ago
  Yes, for small messages and relatively small scope dictionary, character level will work. But that's very different from what's tested here.
minimaxir 1y ago
I made a response to this counterpoint in a blog post I wrote about a similar question posed to LLMs (how many b's are in blueberry): https://news.ycombinator.com/item?id=44878290 https://news.ycombinator.com/item?id=44878290 > Yes, asking an LLM how many b’s are in blueberry is an adversarial question in the sense that the questioner is expecting the LLM to fail. But it’s not an unfair question, and it’s objectively silly to claim that LLMs such as GPT-5 can operate at a PhD level, but can’t correctly count the number of letters in a word. It's a subject that the Hacker News bubble and the real world treat differently.
- brookst 1y ago
  It’s like defending a test showing hammers are terrible at driving screws by saying many people are unclear on how to use tools. It remains unsurprising that a technology that lumps characters together is not great at processing below its resolution. Now, if there are use cases other than synthetic tests where this capability is important, maybe there’s something interesting. But just pointing out that one can’t actually climb the trees pictured on the map is not that interesting.
  - achierius 1y ago
    And yet... now many of them can do it. I think it's premature to say "this technology is for X" when what it was originally invented for was translation, and every capability it has developed since then has been an immense surprise.
    - vanviegen 1y ago
      > And yet... now many of them can do it. Presumably because they trained them to death on this useless test that people somehow just wouldn't shut up about.
      - minimaxir 1y ago
        Which is why in the linked post, I test models against both the "r's in strawberries" and the "b's in blueberries" to see if that is the case. tl;dr the first case had near perfect accuracy as expected for the case if the LLMs were indeed trained on it. The second case did not.
MountDoom 1y ago
I remember people making the exact same argument about asking LLMs math questions back when they couldn't figure out the answer to 18 times 7. "They are text token predictors, they don't understand numbers, can we put this nonsense to rest." The whole point of LLMs is that they do more than we suspected they could. And there is value in making them capable of handling a wider selection of tasks. When an LLM started to count the numbers of "r"s in "strawberry", OpenAI was taking a victory lap.
- vanviegen 1y ago
  > When an LLM started to count the numbers of "r"s in "strawberry", OpenAI was taking a victory lap. Were they? Or did they feel icky about spending way to much post-training time on such a specific and uninteresting skill?
  - ACCount37 1y ago
    It's not as specific of a skill as you would think. Being both aware of tokenizer limitations and capable of working around them is occasionally useful for real tasks.
    - _flux 1y ago
      What tasks would those be, that wouldn't be better served by using e.g. a Python script as a tool, possibly just as component of the complete solution?
      - ACCount37 1y ago
        Off the top of my head: the user wants LLM to help him solve a word puzzle. Think something a bit like Wordle, but less represented in its dataset. For that, the LLM needs to be able to compare words character by character reliably. And to do that, it needs at least one of: be able to fully resolve the tokens to characters internally within one pass, know to emit the candidate words in a "1 character = 1 token" fashion and then compare that, or know that it should defer to tool calls and do that. An LLM trained for better tokenization-awareness would be able to do that. The one that wasn't could fall into weird non-humanlike failures.
curioussquirrel 1y ago
Why test for something? I find it fascinating if something starts being good at task it is "explicitly not designed for" (which I don't necessarily agree with - it's more of a side effect of their architecture). I also don't agree that nobody is using this for - there are real life use cases today, such as people trying to find meaning of misspelled words. On a side note, I remember testing Claude 3.7 with the classic "R's in the word strawberry" question through their chat interface, and given that it's really good at tool calls, it actually created a website to a) count it with JavaScript, b) visualize it on a page. Other models I tested for the blog post were also giving me python code for solving the issue. This is definitely already a thing and it works well for some isolated problems.
- viraptor 1y ago
  > such as people trying to find meaning of misspelled words. That worked just fine for quite a while. There's apparently enough misspelling in the training data, we don't need precise spelling for it. You can literally write drunken gibberish and it will work.
  - curioussquirrel 1y ago
    True. But does that scale to less common words? Or to other languages than English?
    - viraptor 1y ago
      > The phrase "Pweiz mo cco ejst w sprdku zmi?" appears to be a distorted or misspelled version of a Polish sentence. The closest meaningful phrase in Polish is "Powiedz mi co jest w środku ziemi?" which translates to "Tell me what is inside the Earth?" I'm not sure I could figure out the mangled words there.

IncreasePosts 1y ago

Wouldn't a llm that just tokenized by character be good at it?

curioussquirrel 1y ago
Yes, but it would hurt its contextual understanding and effectively reduce the context window several times.
- viraptor 1y ago
  Only in the current most popular architectures. Mamba and RWKV style LLMs may suffer a bit but don't get a reduced context in the same sense.
  - curioussquirrel 1y ago
    You're right. There was also an experiment in Meta which tokenized bytes directly and it didn't hurt performance much in very small models.
typpilol 1y ago
I asked this in another thread and it would only be better with unlimited compute and memory. Because without those, then the llm has to encode way more parameters and way smaller context windows. In a theoretical world, it would be better, but might not be much better.

neerajsi 1y ago

https://www.anthropic.com/news/analysis-tool https://www.anthropic.com/news/analysis-tool Seems like they already built this capability.

tkgally 1y ago

One reason for testing this is that it might indicate how accurately models can explain natural language grammar, especially for agglutinative and fusional languages, which form words by stringing morphemes together. When I tested ChatGPT a couple of years ago, it sometimes made mistakes identifying the components of specific Russian and Japanese words. I haven’t run similar tests lately, but it would be nice to know how much language learners can depend on LLM explanations about the word-level grammars of the languages they are studying. Later: I asked three LLMs to draft such a test. Gemini’s [1] looks like a good start. When I have time, I’ll try to make it harder, double-check the answers myself, and then run it on some older and newer models. [1] https://g.co/gemini/share/5eefc9aed193 https://g.co/gemini/share/5eefc9aed193

gizmo686 1y ago
What you are testing for is fundamentally different than character level text manipulation. A major optimization in modern LLMs is tokenization. This optimization is based on the assumption that we do not care about character level details, so we can combine adjacent characters into tokens, then train and run the main AI model on smaller strings built out of a much larger dictionary of tokens. Given this architecture, it is impressive that AIs can perform character level operations at all. They essentially need to reverse engineer the tokenization process. However, morphemes are semantically meaningful, so a quality tokenizer will tokenize at the morpheme level, instead of the word level. [0]. This is of particuarly obvious importance in Japanese, as the lack of spaces between words means that the naive "tokenize on whitespace" approach is simply not possible. We can explore the tokenizer of various models here: https://huggingface.co/spaces/Xenova/the-tokenizer-playground https://huggingface.co/spaces/Xenova/the-tokenizer-playgroun... Looking at the words in your example, we see the tokenization of the Gemma model (closely related to Gemini) is: un-belie-vably dec-entral-ization bio-degradable mis-understanding anti-dis-establishment-arian-ism пере-писы-ваться pere-pis-y-vat-'-s-ya до-сто-примеча-тельность do-stop-rime-chat-el-'-nost-' пре-по-дава-тель-ница бе-зо-т-вет-ственности bezotvetstvennosti же-лез-нодоро-жный z-hele-zn-odoro-zh-ny-y 食べ-させ-られた-くな-かった tab-es-aser-are-tak-unak-atta) 図書館 tos-ho-kan 情報-技術 j-ō-h-ō- gij-utsu 国際-関係 kok-us-ai- kan-kei 面白-くな-さ-そうだ Further, the training data that is likely to be relevent in this type of query probably isolates the individual morphemes while talking about a bunch of words that the use them; so it is a much shorter path for the AI to associate these close but not quite morphene tokens with the actual sequence of tokens that corresponds to what we think of as a morphene. [0] Morpheme level tokenization is itself a non-trivial problem. However, has been pretty well solved long before the current generation of AI.

redox99 1y ago

Character level LLMs are used for detecting insults and toxic chat in video games and the like.

minimaxir 1y ago
Can you give an example of a video game explicitly using character-level LLMs? There were prototypes of char-rnns back in the day for chat moderation but it has significant compute overhead.
- redox99 1y ago
  It's something I heard through the grapevine. But there's only a few big enough competitive games where toxicity is such a big deal, so it's not hard to guess. Character level helps with players disguising insults. Compute wise it's basically the same, but multiply token count by 4. Which doesn't really matter for short chat in video games.
jazzyjackson 1y ago
I figure an LLM would be way better at classifying insults than regexing against a bad word list. Why would character level be desirable?
- vanviegen 1y ago
  I'd imagine for simplicity - just skip the tokenizer and feed bytes.
- duskwuff 1y ago
  Might a character-level LLM be better at recognizing poorly spelled (or deliberately misspelled) profanity?
viraptor 1y ago
Yes, for small messages and relatively small scope dictionary, character level will work. But that's very different from what's tested here.

minimaxir 1y ago

I made a response to this counterpoint in a blog post I wrote about a similar question posed to LLMs (how many b's are in blueberry): https://news.ycombinator.com/item?id=44878290 https://news.ycombinator.com/item?id=44878290 > Yes, asking an LLM how many b’s are in blueberry is an adversarial question in the sense that the questioner is expecting the LLM to fail. But it’s not an unfair question, and it’s objectively silly to claim that LLMs such as GPT-5 can operate at a PhD level, but can’t correctly count the number of letters in a word. It's a subject that the Hacker News bubble and the real world treat differently.

brookst 1y ago
It’s like defending a test showing hammers are terrible at driving screws by saying many people are unclear on how to use tools. It remains unsurprising that a technology that lumps characters together is not great at processing below its resolution. Now, if there are use cases other than synthetic tests where this capability is important, maybe there’s something interesting. But just pointing out that one can’t actually climb the trees pictured on the map is not that interesting.
- achierius 1y ago
  And yet... now many of them can do it. I think it's premature to say "this technology is for X" when what it was originally invented for was translation, and every capability it has developed since then has been an immense surprise.
  - vanviegen 1y ago
    > And yet... now many of them can do it. Presumably because they trained them to death on this useless test that people somehow just wouldn't shut up about.
    - minimaxir 1y ago
      Which is why in the linked post, I test models against both the "r's in strawberries" and the "b's in blueberries" to see if that is the case. tl;dr the first case had near perfect accuracy as expected for the case if the LLMs were indeed trained on it. The second case did not.

MountDoom 1y ago

I remember people making the exact same argument about asking LLMs math questions back when they couldn't figure out the answer to 18 times 7. "They are text token predictors, they don't understand numbers, can we put this nonsense to rest." The whole point of LLMs is that they do more than we suspected they could. And there is value in making them capable of handling a wider selection of tasks. When an LLM started to count the numbers of "r"s in "strawberry", OpenAI was taking a victory lap.

vanviegen 1y ago
> When an LLM started to count the numbers of "r"s in "strawberry", OpenAI was taking a victory lap. Were they? Or did they feel icky about spending way to much post-training time on such a specific and uninteresting skill?
- ACCount37 1y ago
  It's not as specific of a skill as you would think. Being both aware of tokenizer limitations and capable of working around them is occasionally useful for real tasks.
  - _flux 1y ago
    What tasks would those be, that wouldn't be better served by using e.g. a Python script as a tool, possibly just as component of the complete solution?
    - ACCount37 1y ago
      Off the top of my head: the user wants LLM to help him solve a word puzzle. Think something a bit like Wordle, but less represented in its dataset. For that, the LLM needs to be able to compare words character by character reliably. And to do that, it needs at least one of: be able to fully resolve the tokens to characters internally within one pass, know to emit the candidate words in a "1 character = 1 token" fashion and then compare that, or know that it should defer to tool calls and do that. An LLM trained for better tokenization-awareness would be able to do that. The one that wasn't could fall into weird non-humanlike failures.

curioussquirrel 1y ago

Why test for something? I find it fascinating if something starts being good at task it is "explicitly not designed for" (which I don't necessarily agree with - it's more of a side effect of their architecture). I also don't agree that nobody is using this for - there are real life use cases today, such as people trying to find meaning of misspelled words. On a side note, I remember testing Claude 3.7 with the classic "R's in the word strawberry" question through their chat interface, and given that it's really good at tool calls, it actually created a website to a) count it with JavaScript, b) visualize it on a page. Other models I tested for the blog post were also giving me python code for solving the issue. This is definitely already a thing and it works well for some isolated problems.

viraptor 1y ago
> such as people trying to find meaning of misspelled words. That worked just fine for quite a while. There's apparently enough misspelling in the training data, we don't need precise spelling for it. You can literally write drunken gibberish and it will work.
- curioussquirrel 1y ago
  True. But does that scale to less common words? Or to other languages than English?
  - viraptor 1y ago
    > The phrase "Pweiz mo cco ejst w sprdku zmi?" appears to be a distorted or misspelled version of a Polish sentence. The closest meaningful phrase in Polish is "Powiedz mi co jest w środku ziemi?" which translates to "Tell me what is inside the Earth?" I'm not sure I could figure out the mangled words there.