4 ms·
I think the utility of memory editing is that training is slow and costly; updating is cheap and fast. Presumably, if you’re running a GPT, you might want to fi
by ttul 4y ago
I think the utility of memory editing is that training is slow and costly; updating is cheap and fast. Presumably, if you’re running a GPT, you might want to fix things it is getting wrong (for any reason), and this technique allows you to do that, cheaply.
- brucethemoose2 4y agoLots of LLM services already ask for user feedback to "improve the model" and I imagine this would make that feedback even more useful.
- londons_explore 4y agoBut training to update a single fact isn't slow and costly. finetuning for just a few facts could be done in a matter of seconds with just a few text examples of the correct answer.
- skybrian 4y agoIt uses up limited prompt space, though.
- johntb86 4y agoFinetuning uses a limited number of examples and gradient descent to modify network weights, so it doesn't need to modify the prompt. Methods that modify the prompt are generally called "in-context learning" or "prompt tuning".
- 1024core 4y ago"Fine tuning" has nothing to do with prompting.
- lonesword 4y agoNo finetuning for just a few facts could indeed end up being very costly. If you have 5 new examples that you want to fine-tune your model on, you probably wont fine-tune your existing model for 3000 training steps on just those 5 new examples. You'll either mix in other data to prevent catastrophic forgetting, or you'll probably training from scratch after fixing your dataset to reflect the 5 new examples you have.
- londons_explore 4y agoYou'll probably finetune for one step for each of the 5 examples. Choose the learning rate carefully to get the results you want without much forgetting. Total time is mere seconds. If you save the adam optimizer parameters from previous runs, you'll do even better at preventing forgetting.