3 ms·
LLMs can count characters, but they need to dedicate a lot of tokens to the task. That is, they need a lot of tokens describing the task of counting, and in my
by Buttons840 3y ago
LLMs can count characters, but they need to dedicate a lot of tokens to the task. That is, they need a lot of tokens describing the task of counting, and in my experience that allows them to accurately count.
- dannyw 3y agoSource? LLMs have no “hidden tokens” they dedicate. Or you mean — if the tokenizer was trained differently…
- Buttons840 3y agoNot hidden tokens, actual tokens. Ask a LLM to guess the letter count like 20 times and often it will converge on the correct count. I suppose all those guesses provide enough "resolution" (for lack of a better term) that it can count the letters.
- 8organicbits 3y ago> often it will converge on the correct count That's a pretty low bar for something like counting words.
- 2-718-281-828 3y agobut then it will either overfit or you need to train it on 20 times the amount of data ...
- Buttons840 3y agoI'm taking about when using a LLM, which doesn't involve training and thus no overfitting.
- 2-718-281-828 3y agofor an llm to exhibit a verbal relationship between counting and tokens you have to train it on that. maybe you mean something like a plugin or extension but that's something else and has nothing to do with llms specifically.
- PoignardAzur 3y agoThat reminds of something I've wondered about for months: can you improve a LLM's performance by including a large amount of spaces at the end of your prompt? Would the LLM "recognize" that these spaces are essentially a blank slate and use them to "store" extra semantic information and stuff?