I would say that some one modern GPU has adopted PCIe 5.0, the NVIDIA H100 - if you want more (FP32) 'compute' (and no HBM...) in one GPU your sourceable choice

I would say that some one modern GPU has adopted PCIe 5.0, the NVIDIA H100 - if you want more (FP32) 'compute' (and no HBM...) in one GPU your sourceable choice is the L40 which didn't get PCIe 5.0. This feels like another market/product segmentation (that and L40 didn't get nvlink).