3 ms·
You can set how much of your system ram your gpu gets in windows and linux
by kjkjadksj 3y ago
You can set how much of your system ram your gpu gets in windows and linux
- xcv123 3y agoUseless for LLMs. Limited system memory bandwidth kills it. GPU will be mostly idle waiting for memory access. The H200 has 4.8 TB/s bandwidth. System memory is below 90 GB/s
- kjkjadksj 3y agoIf you have a use case for an h200 class gpu you are probably making enough money off of it to not care about $40k and the 900gb/s bandwidth mentioned in another comment would seem absolutely pedestrian in comparison. If these things were really that compelling of a performance per dollar ratio you wouldn’t be able to buy them. They’d be sold out because people would be filling datacenters with them.
- xcv123 3y agoM2 Ultra sits between a lower end Intel setup and an H200 for this purpose. I reject this all-or-nothing black and white childish framing of it.