2 ms·
RAG is basically a fancy name to augment a prompt with data. Companies are being sold they can augment their LLM with their unstructured massive dataset but it
by Foobar8568 2y ago
RAG is basically a fancy name to augment a prompt with data.
Companies are being sold they can augment their LLM with their unstructured massive dataset but it's all wishful thinking.
- Workaccount2 2y agoYeah, LLM capabilities are measured with fresh context windows, yet people want to use them with 50k, 100k, 500k tokens. As you pack in more and more context the model's abilities really start to deteriorate. The first 10k tokens are the juiciest, after that it just gets worse and worse.
- eternityforest 2y agoOh wow, I was thinking 500 tokens was way too much, since I've only ever done anything programmatic with tiny models on CPUs....