3 ms·
DeepSeek wouldn't be able to train R1 without their ~600B parameters base model, so you should consider the cost of that model when you compare with Grok. In a
by tinyhouse 2y ago
DeepSeek wouldn't be able to train R1 without their ~600B parameters base model, so you should consider the cost of that model when you compare with Grok.
In any case, Elon won't win this race cause the best talent will not work for him. He used to have good reputation and a lot of money, which is a deadly combination. Now he only has the latter -- not enough when leading AI people can make 7 figures in other companies.
To be clear 1: I'm not saying that people who currently work on Grok are not great. It's not about hiring some great people. It's about competing in the long run - people with other options (e.g. offers from leading AI labs) are more likely to accept those offers than joining his research lab.
To be clear 2: I'm not talking about Elon's reputation due to his politics. I'm only talking about his reputation as an employer.
He has the vision and marketing skills but it's not going to be enough for leading the AI race.
- gmerc 2y agoIt’s not like Grok3 didn’t have precious work to build on either, but point taken. I think the situations are a bit comparable given timelines however. A perfect analogy for AI … your ability to replace talent with money. And if you don’t have the talent, it’s gonna cost you 100x more.
- tastyfreeze 2y ago> your ability to replace talent with money That sure seems to be the message given in Apple AI commercials. From those commercials the tag line for AI should be "enabling idiots everywhere".
- andy12_ 2y agoActually, the 5 million figure is for the compute cost for the base 600B parameter model. Training R1 was just 8000 steps of reinforcement learning, so I expect that the vast, vast majority of the training cost is already included in the pretraining stage.
- deleted 2y ago[deleted]