4 ms·
They probably won't share how they did it, but there's been a lot of research over the past 6 months showing how you don't have to retrain the entire model to a
by dualityoftapirs 3y ago
They probably won't share how they did it, but there's been a lot of research over the past 6 months showing how you don't have to retrain the entire model to add in new sources. I know nothing about this stuff, but my limited understanding from blog posts is it's easier than anyone had thought to add in new data to a pre-existing model.
- skilled 3y agoDo you by any chance have any of these blog posts available for my own reference? If not you maybe someone else does, I don't recall seeing it but it sounds interesting.
- circuit10 3y agoI assume they mean fine tuning? https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning) https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning) https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning
- dpoljak 3y ago[flagged]
- rafaelero 3y agoI think there was a paper from Google showing that if you included 5% of your original dataset together with the new data during the finetuning then catastrophical forgetting didn't occur. Perhaps it's that simple.