4 ms·

Thanks for detailed analysis, would be curios to see FP8 comparison too, given vllm has some custom kernels

• cheptsov 2y ago
  Unfortunately, neither VLLM nor TGI support FP8 on AMD yet. But once they do, we will look into it.