3 ms·

RAFT: Sailing Llama towards better domain-specific RAG

• mpalmer 2y ago
  Just because the acronym is right there on the ground doesn't mean you have to pick it up. There's already a pretty famous Raft in computing.
  • halosghost 2y ago
    I feel like this is happening quite a lot in the generative-AI space, a /lot/ of words/names that already have wide/common use in the tech-sphere are being reused for new developments in generative AI. I'm trying to not take it as an intentional extension of the already problematic effect these same technologies are having on the effectiveness of web-search… All the best, -HG
• anon373839 2y ago
  Retrieval-Augmented Fine-Tuning is a really promising technique. FTA: > Tianjun and Shishir were looking to improve these deficiencies of RAG. They hypothesized that a student who studies the textbooks before the open-book exam would be more likely to perform better than a student who references the textbook only during the exam. Translating that back to LLMs, if a model “studied” the documents beforehand, could that improve its RAG performance? Incidentally, the team who wrote the paper released some nice code to generate domain-specific fine-tuning datasets: https://github.com/ShishirPatil/gorilla/tree/main/raft https://github.com/ShishirPatil/gorilla/tree/main/raft