3 ms·
I agree with your general premise, but I think you left a couple of points of your list at the end: it is obscenely expensive to keep training + there are othe
by muskmusk 3y ago
I agree with your general premise, but I think you left a couple of points of your list at the end:
it is obscenely expensive to keep training + there are other more low hanging fruit + you expect hardware to get better over time.
I don't think Altman is trying to fool anyone. Even if he were it wouldn't work. The competition is not that stupid and he knows that :)
It's just that hardware tends to get better at a rate that resembles Moore's law so in 18 months the cost of training a 100 mill dollar model is 50 mill dollar. You certainly can just throw money at the problem, but it's expensive and there are other options that are just as effective for now. Why spend money on things that are half as valuable in 18 months when you can spend money on things that don't devalue as fast like producing more/better data?
All that being said you can bet your ass there will be a gpt5 :)