3 ms·
$5M to train GPT-4 is the best investment I've ever seen. I've seen startups waste more money for tremendously smaller impact.
by bagels 4y ago
$5M to train GPT-4 is the best investment I've ever seen. I've seen startups waste more money for tremendously smaller impact.
- sailingparrot 4y agoAs I stated in my comment, $5M is assuming you can do a much much better job than OpenAI at optimizing your training, only need to make a single training run, your employees salaries are $0, and you get a clean dataset for essentially free. Real cost is 10-20x that. That's still a good investment though. But the issue is you could very well sink $50M into this endeavour and end up with a model that actually is not really good and gets rendered useless by an open-source model that gets released 1 month later. OpenAI truly has unique expertise in this field that is very, very hard to replicate.
- moffkalast 4y ago> and end up with a model that actually is not really good and gets rendered useless ahem Bard ahem