3 ms·

We've been testing it in the local llm Discords, turns out its just a llama 7B finetune that can run on any old GPU (which is cool). https://huggingface.co/bru

• simcop2387 3y ago
  > And its long context recall is quite good! We've already kind of discovered this with Yi, but there are some things one can do with a mega context that you just can't get with RAG. I've got to imagine that a mega-context like this can help RAG work in ways that just isn't possible otherwise. i.e. bring in many more search results or surrounding context around the results so that the processing can do much more.
• andy_xor_andrew 3y ago
  > but there are some things one can do with a mega context that you just can't get with RAG. Can you elaborate? In my mind, RAG and "mega context" are orthogonal - RAG is something done by adding documents to the context for the LLM to reference, and "mega context" is just having a big context. No?
  • bradfox2 3y ago
    I think he means not needing to have a great search system to identify rag chunks. Just throw everything in.
    • brucethemoose2 3y ago
      Or more specifically, be able to look at everything all at once instead of in chunks.