3 ms·
1/1000 of inference compute is a non-trivial workload at scale. Gartner estimates ~$28B in inference spend for 2026 making this a $28 million dollar per year wo
by pipsterwo 2mo ago
1/1000 of inference compute is a non-trivial workload at scale. Gartner estimates ~$28B in inference spend for 2026 making this a $28 million dollar per year workload (edit: based on the assumption above)
Source: https://www.gartner.com/en/newsroom/press-releases/2026-07-20-gartner-forecasts-worldwide-ai-platforms-and-models-market-to-grow-63-percent-in-2026 https://www.gartner.com/en/newsroom/press-releases/2026-07-2...
- boroboro4 2mo agoThe issue is it’s cpu compute which is underutilized in gpu clusters anyway, so practically it’s not really 1/1000.
- pipsterwo 2mo agoTotally, edited my comment to specify "based on the assumption above." The main takeaway I was going for was 0.1% is not a small number in this context