4 ms·

Having worked on a (production, as-a-service RAG system), what we've learned is that fine tuning is not good for really adding/removing facts but is great for c

Having worked on a (production, as-a-service RAG system), what we've learned is that fine tuning is not good for really adding/removing facts but is great for changing the form of the output. Anyscale has a great blog on this: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts https://www.anyscale.com/blog/fine-tuning-is-for-form-not-fa... (I work at Vectara, not Anyscale, but appreciate the details and thought they put into this blog) Fine-tuning can help with certain areas of knowledge acquisition, but it's costly and frankly doesn't work that well when the knowledge you're trying to provide goes "against" the data that was trained into the core system. e.g. try to fine tune a model that there is a cure to <disease X> if you're a research org and it's incredibly hard to do that because the base model may be so convinced otherwise. Probably the most difficult thing though is using fine-tuning to get a model to "forget" or "not respond to" something that it shouldn't take a stance on. I always ask a fine-tuned model: "how do you calculate the fourth interior angle of a triangle." This is obviously a nonsense question, but helps to show how most LLMs will happily tell you to sum the interior angles of your first 3 angles and then subtract from 180. A well-tuned RAG system will say "sorry, I don't have information for that." It shows how it's less "guess-y" and hallucinated

namibj 3y ago
Actually, selectively deleting facts like that does kinda work, essentially what you're doing is scolding-based reinforcement learning. For decent speed you'd want to probably use variance-reduction techniques like https://arxiv.org/abs/2205.08253 https://arxiv.org/abs/2205.08253 , though. Or a related (would be linked as cited by/cited in the linked when you look on semantic scholar) method that doesn't compute a hessian-vector product (which apparently interacts poorly with the non-linearity of the multi-layer perceptrons as provided by LLM-typical ReLU), in exchange for e.g. sizable batching (gradient accumulation!) at occasional checkpoints.

namibj 3y ago

Actually, selectively deleting facts like that does kinda work, essentially what you're doing is scolding-based reinforcement learning. For decent speed you'd want to probably use variance-reduction techniques like https://arxiv.org/abs/2205.08253 https://arxiv.org/abs/2205.08253 , though. Or a related (would be linked as cited by/cited in the linked when you look on semantic scholar) method that doesn't compute a hessian-vector product (which apparently interacts poorly with the non-linearity of the multi-layer perceptrons as provided by LLM-typical ReLU), in exchange for e.g. sizable batching (gradient accumulation!) at occasional checkpoints.