3 ms·
> if DeepSeek really has found a way to train models just as good as frontier ones for a hundredth of the hardware investment Frontier models are heavily compu
by zozbot234 2y ago
> if DeepSeek really has found a way to train models just as good as frontier ones for a hundredth of the hardware investment
Frontier models are heavily compute constrained - the leading AI model makers have got way more training data already than they could do anything with. Any improvement in training compute-efficiency is great news for them, no matter where it comes from. Especially since the DeepSeek folks have gone into great detail wrt. documenting their approach.
- rpcope1 2y ago> leading AI model makers have got way more training data already than they could do anything with. Citation needed.
- nullc 2y agoIf you include multimodal data then I think it's pretty obvious that training is compute limited. Also current SOTA models are good enough that you can generate endless training data by letting the model operate stuff like a C compiler, python interpreter, Sage computer algebra, etc.