3 ms·
This is kinda creepy. But at the same time, how do they do that? I thought the training of these models stopped in September 2021/2022. So how do they do these
by behnamoh 3y ago
This is kinda creepy. But at the same time, how do they do that? I thought the training of these models stopped in September 2021/2022. So how do they do these incremental trainings?
- infinityio 3y agoThe exact phrase they previously used on the homepage was "Limited knowledge of world and events after 2021" - so maybe as a finetune?
- behnamoh 3y agobut doesn’t finetuning result in forgetting previous knowledge? it seems that finetuning is most usable to train “structures” not new knowledge. am i missing something?
- theptip 3y agoAll the public and (leaked) private statements I have seen state that this is not happening. As siblings noted, MoE probably explains this variance. AIUI they are using current chat data for training GPT-5, not re-finetuning the existing models.