3 ms·
One thing I hadn't really thought through is that the reason these models are trained to point X is that they just decide to release it then. But most of these
by nl 3y ago
One thing I hadn't really thought through is that the reason these models are trained to point X is that they just decide to release it then.
But most of these companies _keep training_ the same model. A lot of the time there is no "ok we are done, we will do a new thing now" - it just keeps going.
Obviously thinking this though it explains a lot of the vast spend on GPUs. They need new compute because the existing compute is occupied _and always will be_. Model training will never finish!
- petesergeant 3y ago> But most of these companies _keep training_ the same model Do they? Genuine question. Why do you think so?
- sva_ 3y agoIf you look at the number of different gpt4 models, it seems like a reasonable thing to say: https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4 https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-... While it appears like they all share the same gpt4 base, it isn't really known how these are trained (fine-tuned?)
- JKCalhoun 3y agoHow many models do they have? Is model availability becoming the bottleneck?