3 ms·
Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their m
by dantillberg 2mo ago
Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.
- NuclearPM 2mo agoWhy is that true? Can’t you just make more stuff parallel and shrink the ASIC chips accordingly?
- jmalicki 2mo agoNo - inability to do so is part of the design of a good cryptographic hash, quite explicitly. https://en.wikipedia.org/wiki/Avalanche_effect https://en.wikipedia.org/wiki/Avalanche_effect
- noosphr 2mo agoSo does llm inference. You're lucky if you hit 40% of the advertised flops.
- fwipsy 2mo agoRight, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
- noosphr 2mo agoNo they don't.
- eru 2mo agoWhy not? Seems like they would be poorly optimised?
- noosphr 2mo agoBecause llm inference is not the only workload a GPU can do and custom silicon cost $10b a chip.
- eru 2mo agoAI optimised cards is the biggest cashcow for nvidia. They are definitely doing custom silicon. And Google et al have cards that don't even pretend to be able to do graphics.
- noosphr 2mo agoThere is more than one workload in AI. Inference for llms is memory constrained on even a single card. Training for llms is memory constrained on the level of racks. In both cases you hardly ever see more than 40% of advertised flops used.
- eru 2mo agoSo the question is: why are they designing AI cards so unbalanced?
- dapperdrake 2mo agoThen where are the HDMI ports on Nvidia's current data center GPU product lines?