3 ms·

So how many 1 bits are in strawberry? Thats about the same kind of question, given that LLMs see tokens instead of characters, and we see characters instead o

So how many 1 bits are in strawberry? Thats about the same kind of question, given that LLMs see tokens instead of characters, and we see characters instead of bits.

doctorpangloss 2y ago
Do you sincerely think the folks training these models haven't tried alternative tokenizations?
- Scarblac 2y ago
  Of course they have, I assume counting letters in words wasn't a priority for them.