5 ms·
One very exciting thing is the vram. 192GB!!! That seems like a minor bump compared to the new H100, but sometimes the quality of your training, or the speed o
by brucethemoose2 3y ago
One very exciting thing is the vram. 192GB!!!
That seems like a minor bump compared to the new H100, but sometimes the quality of your training, or the speed of your inference, is hugely effected by small thresholds of VRAM capacity per card. The MI300X can do things the H100 cannot, and judging by the sheer size of the silicon, do them reasonably quickly.
- dharma1 3y agoI think vram is one thing they can beat Nvidia on. Both on the pro and consumer GPUs. Just put 48gb into a consumer GPU and everyone would flock to AMD and help build their software side
- brucethemoose2 3y agoThey already have 48GB Pro GPUs. If they were going to do it on the consumer side, they would have done it already. And I would own a 7900 48GB instead of a 3090, and probably have debugged ROCm on several projects by now :/
- dharma1 3y agoExactly - users would flock to it. They have to compete with Nvidia somehow and I don't see what other edge they could have - massive disadvantage on the software side that has to be compensated for in some way
- brucethemoose2 3y agoWell local LLM running with rocm isnt exactly huge business. You are not wrong, but I can envision why decision makers wouldn't want to spoil the pro line.
- dharma1 3y agoThe real pros will use data centre GPUs like the MI300. I think local ML will become huge in games