3 ms·
>1. Chinchilla has demonstrated models are currently unnecessarily large, and would benefit more from data scaling. Not convinced. It showed this for the origi
by sebzim4500 4y ago
>1. Chinchilla has demonstrated models are currently unnecessarily large, and would benefit more from data scaling.
Not convinced. It showed this for the original self supervised task, but it might be true that the spare parameters end up being useful for the later finetuning/RLHF stages.