3 ms·
Yeah, I think training for facts in general is kind of problematic since you often have to overfit and the model may lose capability in other areas. I suspect t
by hushpiper 3y ago
Yeah, I think training for facts in general is kind of problematic since you often have to overfit and the model may lose capability in other areas. I suspect that the only situations where it really makes sense to train on facts are where the facts are very nuanced and require a lot of interpretation, or more often where the facts are just so extensive that they can't be crammed effectively into the context window you're working with. Otherwise, you're better off with a vector db and a well-written prompt.
- qup 3y agoWhat if we just train it to respect facts in general, then couldn't we just supply it a list of facts? Sort of how they made chatGPT way more likely to obey requests?
- valine 3y agoYou can supply the model with a list of facts already, that’s not the problem. Within the context window the model is able to learn and generalize new information. Fine tuning is very unintelligent in the sense that it doesn’t take the context of the training samples into account. It’s a dumb optimizer that’s trying to minimize next token loss. Gradient descent is not beholden to the behaviors you taught in the instruct fine tune step.