2 ms·
Showing roughly 1.5-2.0x speed ups on a couple of example workloads under fp8. Not bad for what's basically just a software change. Just one more thing that wil
by nharada 3y ago
Showing roughly 1.5-2.0x speed ups on a couple of example workloads under fp8. Not bad for what's basically just a software change. Just one more thing that will make it hard for AMD to catch up.
- kkielhofner 3y agoProjects like this are the tip of the iceberg... While ROCm users celebrate finding just the right configuration to get PyTorch not to crash, Nvidia shops are deploying solutions in production with stuff like this, FasterTransformer, Triton Inference Server, etc, etc. People (fairly) complain about Nvidia price points but they put a lot of work in on the software side to get a lot of value out of their products. At the point you’re deploying with a fully tuned and optimized stack built on dozens of Nvidia software components that get 10x the perf of AMD you’re thrilled you only paid 5x as much for the hardware.