4 ms·
Going to reply to myself here to say that it does seem like there is definitely information from December/November 2021 available, like clear-cut facts. I'm ju
by skilled 3y ago
Going to reply to myself here to say that it does seem like there is definitely information from December/November 2021 available, like clear-cut facts.
I'm just curious why OpenAI didn't make an announcement or what's the deal here, wouldn't this warrant retraining the entire model?
- dualityoftapirs 3y agoThey probably won't share how they did it, but there's been a lot of research over the past 6 months showing how you don't have to retrain the entire model to add in new sources. I know nothing about this stuff, but my limited understanding from blog posts is it's easier than anyone had thought to add in new data to a pre-existing model.
- skilled 3y agoDo you by any chance have any of these blog posts available for my own reference? If not you maybe someone else does, I don't recall seeing it but it sounds interesting.
- circuit10 3y agoI assume they mean fine tuning? https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning) https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning) https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning
- dpoljak 3y ago[flagged]
- rafaelero 3y agoI think there was a paper from Google showing that if you included 5% of your original dataset together with the new data during the finetuning then catastrophical forgetting didn't occur. Perhaps it's that simple.
- formvoltron 3y agothey are working on including information up to 2025. now THAT will be a big announcement.
- whimsicalism 3y agoChatgpt model is constantly training and it doesn’t require retraining the whole model.
- skilled 3y agoCan you provide specifics?
- whimsicalism 3y agoOnly that openai has said they are constantly changing the chatgpt model.
- skilled 3y agoRight...