4 ms·
How about spending $50 billion in capex to do $100 billion worth of inference? Everyone is hung up on the cost of training R1 but it’s 685 billion parameters.
by throwup238 2y ago
How about spending $50 billion in capex to do $100 billion worth of inference?
Everyone is hung up on the cost of training R1 but it’s 685 billion parameters. We still need all of those GPUs to actually use the model.
- fspeech 2y agoThat's not how commodities are priced. Commodities are priced at the marginal cost of production.
- deleted 2y ago[deleted]
- mrbungie 2y agoInference can be more easily offloaded to other kinds of processing units, which also probably are more efficient, like TPUs. That makes both NVDA stock and big AI infrastructure spending less compelling, as those needs are scaled down via software efficiency and chip alternatives.
- blitzar 2y ago$100 billion worth of inference is 25x the inference that OpenAi has done to date.