It should also be noted that H100 has 80 GB VRAM and MI300X has 192 GB. So, it can happily load the largest 8 bit model H100 can in 16 bit. Also, like others no

It should also be noted that H100 has 80 GB VRAM and MI300X has 192 GB. So, it can happily load the largest 8 bit model H100 can in 16 bit. Also, like others noted, vLLM is behind on FP8 support, its not a hardware issue.