4 ms·
I am actually curious who is using these types of GPUs (H100 currently). Is it exclusively used for LLM training? I can’t really see another market where it’d
by upbeat_general 3y ago
I am actually curious who is using these types of GPUs (H100 currently). Is it exclusively used for LLM training?
I can’t really see another market where it’d make sense.
- gyrovagueGeist 3y agoAlthough it is not as hyped, HPC is much wider than just machine learning. H100 are incredibly useful in numerical simulation and scientific computing.
- Jordanpomeroy 3y agoAlready answered below, but the bulk of the market for H100’s appears to be AI. HPC was historically the largest market for this class of accelerator, but it appears to have shifted to hyper scale cloud computing centers. HPC’s are used to distribute the computation of differential equations using numerical methods and AI appears to be making inroads here as well. I would be interested in others’ opinion on whether it’s a settled question on whether or not AI will be able to produce equivalent solutions.
- gyrovagueGeist 3y agoIt's an iterative thing! Neural Network based surrogate models are becoming popular, but you need the compute to solve the PDEs to train the AI and compute the loss in the first place ;)
- wmf 3y agoSupercomputers also use them, like the 4th, 5th, 6th, 8th, and 9th largest ones.
- Jordanpomeroy 3y agoThe list of supercomputers using H100’s continues past #9
- uniclaude 3y agoNot only training. Inference as well.
- ajcp 3y agoI feel like this is more and more in the majority too. So much so that AMD is talking about their chips for inference as much as training [0]. 0. https://www.theverge.com/23894647/amd-ceo-lisa-su-ai-chips-nvidia-supply-chain-interview-decoder https://www.theverge.com/23894647/amd-ceo-lisa-su-ai-chips-n...
- dkjaudyeqooe 3y ago> TDP 450W - 1000W Also useful as a space heater.
- lambda_matt 3y agoThat's practically the TDP for an entire x1 chassis though. Check out the TDP for a H100 DGX Pod. The performance per watt of GH200 *should* be a massive improvement over x86 with Hopper, especially for clusters
- pests 3y agoLiterally. 1000w heaters going for $20 on amazon. Good savings!
- justapassenger 3y ago> Is it exclusively used for LLM training? While LLMs is what cool kids all talk about nowadays, there's way, way, way more to the machine learning than them. I'd say that minority of GPUs are used for LLMs in the ML space. There's likely way more used them where they directly bring billions of dollars to the companies (like all the rankers across big tech, for content, ads, search, etc).
- upbeat_general 3y agoAre they price-competitive for those applications though? My understanding is that the main focus is Memory/GPU-interconnect for large models. Are recommendation/ranking models large enough to take advantage of this? I don't think these cards are generally competitive in throughput/$.
- refulgentis 3y agoIt's not really clear what you mean, maybe there's gaps in my ML knowledge, but generally the answer depends on "what throughput do you need for your use case?" Generally I agree that if you only need to deliver mail once daily on a mile long route, a car that takes 24 hours to travel a mile is fine.
- upbeat_general 3y agoLots of ML applications don't generally care about latency, and throughput only matters per dollar (as you can just scale horizontally). To my knowledge, most inferencing (at least for simpler models) happens on cheaper, slower GPUs that have better throughput/dollar.
- ShamelessC 3y agoDon’t take this the wrong way but have you been drinking?
- refulgentis 3y ago? Have you? (Note I'm not op)
- sebastiennight 3y ago[dead]