3 ms·
Doing large language model inference on AMD MI300 GPUs is a no-brainer due to the increased RAM and lower cost, while Nvidia A100/H100 still dominates for train
by ipsum2 2y ago
Doing large language model inference on AMD MI300 GPUs is a no-brainer due to the increased RAM and lower cost, while Nvidia A100/H100 still dominates for training.
- LarsDu88 2y agoDoes AMD have a library like TensorRT that optimizes inference with operator fusions and kernel optimizations though? TensorRT can halve the inference compute for certain Nvidia inference workloads
- ipsum2 2y agoYeah their equivalent is MIGraphX. There's also support for Triton for custom kernels.