3 ms·
I told mine to remove all unnecessary words from a sentence and talk like caveman, which should result in another 50% savings ;)
by andai 7mo ago
I told mine to remove all unnecessary words from a sentence and talk like caveman, which should result in another 50% savings ;)
- esperent 7mo agoHave you tried asking it to remove vowels?
- andai 7mo agoNot sure that would help due to how tokenization works, but I remember from the early GPT-4 days that LLMs have the ability to "compress" a message into an incomprehensible string of Unicode, which the LLM itself understands perfectly, and which is 5-10x shorter than the English text. That was a big deal when the context size was 8K; now that tokens are cheap and context is huge, nobody seems to be investigating that anymore.
- dbg31415 7mo ago"I told it don't make mistakes, and don't use a lot of tokens! I'm a 10x Engineer now!" (=
- verdverm 7mo agoI'm a fan of Dr Seuss mimicry, the extra tokens are worth the entertainment.
- _rwo 7mo agoMemory unlocked: https://www.youtube.com/watch?v=_K-L9uhsBLM https://www.youtube.com/watch?v=_K-L9uhsBLM