5 ms·
> H200 GPU with 1.5TB host memory, While yes it's one GPU.. It's not exactly a slim one.
by bilekas 6mo ago
> H200 GPU with 1.5TB host memory,
While yes it's one GPU.. It's not exactly a slim one.
- nekusar 6mo agoWhen the comparison is again 128 H100's , yeah, this is a crazy good upgrade. And you can rent H100's and H200s for not that much per hour.
- p_stuart82 6mo ago$2-4/hr always sounds cheap until you multiply by wall clock and reruns
- jeremyjh 6mo agoYes but they are getting only 341 tok/s. A 2.5 trillion run would take over 200 years.
- anshumankmr 6mo ago>cries in RTX 3060
- rldjbpin 6mo agoeven if it scales down, the host-to-gpu memory ratio is around 10 to 1 (1.5 TB vs 141 GB) [1]. might need some odd system builds, improbable with current pricing, to replicate or better the ratio. such as a ~256 GB system for a 4090 (with 24 GB VRAM). [1] https://www.nvidia.com/en-us/data-center/h200/ https://www.nvidia.com/en-us/data-center/h200/
- tos1 6mo agoThat’s a pretty non-standard H200 configuration. In the regular HGX configurations, a node with 8xH200 has that much CPU DRAM. That makes the title of the paper somewhat arguable imo.