4 ms·
> I submit as evidence Llama 3. Llama 3 has likely costed billions to Meta and most of it went to Nvidia. What are you talking about. As much as what you think
by YetAnotherNick 2y ago
> I submit as evidence Llama 3.
Llama 3 has likely costed billions to Meta and most of it went to Nvidia. What are you talking about. As much as what you think inference costs, it is tiny in comparison to training the model.
- zarzavat 2y agoPre-training happens once then it’s done. You can’t make a multi-trillion dollar business out of providing hardware for that. Inference is where the money is because it’s recurring.
- YetAnotherNick 2y agoNo, while what you are saying sounds intuitive on one hand, OpenAI is building $100b supercomputer for training new models. As long as people want to train better models than a year before(which seems to be a good assumption for new few years), training will keep on giving good revenue to Nvidia. Inference requires 1/3rd the FLOPs than training per token and current models are already trained on entirety of internet. You can generate 3 times the internet for 1 training run.