7 ms·
That is what I am looking for. a) LLMs are trained using compressed text tokens and b) use compressed prompts. Don't know how..but that is what I was hoping for
by the_arun 1y ago
That is what I am looking for. a) LLMs are trained using compressed text tokens and b) use compressed prompts. Don't know how..but that is what I was hoping for.
- deepdarkforest 1y agoThe whole point of embeddings and tokens are that they are a compressed version of text, a lower dimensionality. now, how low depends on performance, lower amount of vectors=more lossy (usually). https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard You can train your own with very very compressed, i mean you could even go down to each token=just 2 float numbers. It will train, but it will be terrible, because it can essentially only capture distance. Prompting a good LLM to summarize the context is probably funnily enough the best way of actually "compressing" context
- rattyJ2 1y agoTokens are already compressed. That's what tokenisation is.