Retrieval-Augmented Fine-Tuning is a really promising technique. FTA: > Tianjun and Shishir were looking to improve these deficiencies of RAG. They hypothesiz

Retrieval-Augmented Fine-Tuning is a really promising technique. FTA: > Tianjun and Shishir were looking to improve these deficiencies of RAG. They hypothesized that a student who studies the textbooks before the open-book exam would be more likely to perform better than a student who references the textbook only during the exam. Translating that back to LLMs, if a model “studied” the documents beforehand, could that improve its RAG performance? Incidentally, the team who wrote the paper released some nice code to generate domain-specific fine-tuning datasets: https://github.com/ShishirPatil/gorilla/tree/main/raft https://github.com/ShishirPatil/gorilla/tree/main/raft