4 ms·
Is there anything special about H100/A100 GPUs that would make this acceleration work on them and not work on 30x0/40x0 GPUs
by vladgur 3y ago
Is there anything special about H100/A100 GPUs that would make this acceleration work on them and not work on 30x0/40x0 GPUs
- nkingsy 3y agohttps://github.com/NVIDIA/TensorRT/issues/982 https://github.com/NVIDIA/TensorRT/issues/982 Looks like tensorRT does work with 30x0, but I couldn't find much.
- imjonse 3y agoFrom the article I understand it's only for H100? Both the title and the charts imply that.
- minimaxir 3y agoNormally it's architectural changes such as FP8 support as noted in the submission, but in that case newer mid-range GPUs like the A10 and the L4 (https://www.nvidia.com/en-us/data-center/l4/ https://www.nvidia.com/en-us/data-center/l4/) should be supported but aren't.
- why_only_15 3y agoNVLink is the most notable (>600GB/s GPU-to-GPU communication) whereas the best case scenario on 30x0/40x0 GPUs is peer-to-peer PCIe (~30GB/s) and average case is routing PCIe through the host (~15GB/s)
- boxfire 3y ago3090 does 4x nvlink each way, ~100 GB/s bidirectional data
- why_only_15 3y agoOh huh I didn't know about that!
- ban-lan-gen 3y agoThey just want to sell some expensive hardwares?