6 ms·

Because tokenization is an imperfect way to encode the required information. Think about how we chunk words[1] and recognize them. We have whole word(shape rec

Because tokenization is an imperfect way to encode the required information. Think about how we chunk words[1] and recognize them. We have whole word(shape recognition), morphme recognition, and spelling(letter-by-letter chunking). Text models receive tokens(akin to morpheme chunks) and don't have access to the underlying letters(spelling data) unless that was part of their training. For the most part, individual letters, something I think we can agree is necessary for rendering text, is not accessible. An appropriate analogy is an illiterate artist. Someone who can hear chunks of words and recognizes them verbally I'd asked to do their best job at painting text. They can piece together letter clusters based on inference, but they cannot spell. 1. https://en.m.wikipedia.org/wiki/Chunking_(psychology) https://en.m.wikipedia.org/wiki/Chunking_(psychology)

bonzini 3y ago
> An appropriate analogy is an illiterate artist. Or tattoos with Chinese/Japanese characters. More often than not they use the "wrong" character (even if it is technically correct) and the calligraphy is considered not artistic by native speakers.
radarsat1 3y ago
> Text models receive tokens(akin to morpheme chunks) and don't have access to the underlying letters Heh, this gives me the idea of training an image generator not on tokens but on rendered text of the prompt in some 8x8 font, that would be a fun experiment ;)
- kelseyfrog 3y ago
  That would be an interesting experiment! I'd also be curious how text rendering performance changes if the tokenizer could be made aware of quoted strings and instead tokenizes the contents as characters instead. Surely someone has tried this, (right?) but I haven't seen it in the literature.
gwern 3y ago
Correct. Most image models to date have used very small text models, which are unable to spell many words (https://arxiv.org/abs/2108.11193 https://arxiv.org/abs/2108.11193) - if you cannot spell a word letter by letter because you don't know what the letters are, how are you going to generate a pixel image of each of those letters...? (It is not as if letters were intrinsically hard. GANs were generating letters easily like 7 years ago.) Their general inability is due to BPEs and non-character-based tokenization. This was noted in the DALL-E 2 paper (https://arxiv.org/pdf/2204.06125.pdf#page=16&org=openai https://arxiv.org/pdf/2204.06125.pdf#page=16&org=openai), and it can be experimentally established by swapping out even a very large LLM like PaLM for a humble, small, weak, but not badly-tokenized ByT5 and noting instant solution of the 'problem' (https://arxiv.org/abs/2212.10562#google https://arxiv.org/abs/2212.10562#google). Skip to the appendix of the second paper if you have any doubts about the difference that switching to ByT5 makes in terms of spelling. The solution is just scaling up the LLM models (which is necessary to get better instruction-following and image quality in general, quite aside from spelling inside images) and eventually switching to character tokenization.* See, as always, https://gwern.net/gpt-3#bpes https://gwern.net/gpt-3#bpes (Hands and cats, however, are just genuinely difficult and require biting the bullet of scaling. And I wonder if it will take video supervision to truly solve them?) * on a recent-news note, I suspect Claude-3 may have done something interesting with tokenization - possibly but not necessarily switching to character/byte encodings - and this is part of why it confabulates in ways unusual for ChatGPT but also is a lot more pleasant to use.
barkingcat 3y ago
A good analogy is asking a English speaking tattoo artist who doesn't know any Japanese to tattoo a Kanji letter. The tattoo artist knows roughly what a lettershape is, but has no idea how to write it. Leads to hilarious tattoos

bonzini 3y ago

> An appropriate analogy is an illiterate artist. Or tattoos with Chinese/Japanese characters. More often than not they use the "wrong" character (even if it is technically correct) and the calligraphy is considered not artistic by native speakers.

radarsat1 3y ago

> Text models receive tokens(akin to morpheme chunks) and don't have access to the underlying letters Heh, this gives me the idea of training an image generator not on tokens but on rendered text of the prompt in some 8x8 font, that would be a fun experiment ;)

kelseyfrog 3y ago
That would be an interesting experiment! I'd also be curious how text rendering performance changes if the tokenizer could be made aware of quoted strings and instead tokenizes the contents as characters instead. Surely someone has tried this, (right?) but I haven't seen it in the literature.

gwern 3y ago

Correct. Most image models to date have used very small text models, which are unable to spell many words (https://arxiv.org/abs/2108.11193 https://arxiv.org/abs/2108.11193) - if you cannot spell a word letter by letter because you don't know what the letters are, how are you going to generate a pixel image of each of those letters...? (It is not as if letters were intrinsically hard. GANs were generating letters easily like 7 years ago.) Their general inability is due to BPEs and non-character-based tokenization. This was noted in the DALL-E 2 paper (https://arxiv.org/pdf/2204.06125.pdf#page=16&org=openai https://arxiv.org/pdf/2204.06125.pdf#page=16&org=openai), and it can be experimentally established by swapping out even a very large LLM like PaLM for a humble, small, weak, but not badly-tokenized ByT5 and noting instant solution of the 'problem' (https://arxiv.org/abs/2212.10562#google https://arxiv.org/abs/2212.10562#google). Skip to the appendix of the second paper if you have any doubts about the difference that switching to ByT5 makes in terms of spelling. The solution is just scaling up the LLM models (which is necessary to get better instruction-following and image quality in general, quite aside from spelling inside images) and eventually switching to character tokenization.* See, as always, https://gwern.net/gpt-3#bpes https://gwern.net/gpt-3#bpes (Hands and cats, however, are just genuinely difficult and require biting the bullet of scaling. And I wonder if it will take video supervision to truly solve them?) * on a recent-news note, I suspect Claude-3 may have done something interesting with tokenization - possibly but not necessarily switching to character/byte encodings - and this is part of why it confabulates in ways unusual for ChatGPT but also is a lot more pleasant to use.

barkingcat 3y ago

A good analogy is asking a English speaking tattoo artist who doesn't know any Japanese to tattoo a Kanji letter. The tattoo artist knows roughly what a lettershape is, but has no idea how to write it. Leads to hilarious tattoos