2 ms·
Because llm inference is not the only workload a GPU can do and custom silicon cost $10b a chip.
by noosphr 3mo ago
Because llm inference is not the only workload a GPU can do and custom silicon cost $10b a chip.
- eru 3mo agoAI optimised cards is the biggest cashcow for nvidia. They are definitely doing custom silicon. And Google et al have cards that don't even pretend to be able to do graphics.
- noosphr 3mo agoThere is more than one workload in AI. Inference for llms is memory constrained on even a single card. Training for llms is memory constrained on the level of racks. In both cases you hardly ever see more than 40% of advertised flops used.
- eru 3mo agoSo the question is: why are they designing AI cards so unbalanced?