11 ms·
Not hidden tokens, actual tokens. Ask a LLM to guess the letter count like 20 times and often it will converge on the correct count. I suppose all those guesses
by Buttons840 3y ago
Not hidden tokens, actual tokens. Ask a LLM to guess the letter count like 20 times and often it will converge on the correct count. I suppose all those guesses provide enough "resolution" (for lack of a better term) that it can count the letters.
- 8organicbits 3y ago> often it will converge on the correct count That's a pretty low bar for something like counting words.
- 2-718-281-828 3y agobut then it will either overfit or you need to train it on 20 times the amount of data ...
- Buttons840 3y agoI'm taking about when using a LLM, which doesn't involve training and thus no overfitting.
- 2-718-281-828 3y agofor an llm to exhibit a verbal relationship between counting and tokens you have to train it on that. maybe you mean something like a plugin or extension but that's something else and has nothing to do with llms specifically.
- PoignardAzur 3y agoThat reminds of something I've wondered about for months: can you improve a LLM's performance by including a large amount of spaces at the end of your prompt? Would the LLM "recognize" that these spaces are essentially a blank slate and use them to "store" extra semantic information and stuff?