HBM is very expensive though. I wonder if older generations are still being produced, but significantly cheaper these days? A P100 from 2016 (HBM2) has 732 GB

HBM is very expensive though. I wonder if older generations are still being produced, but significantly cheaper these days? A P100 from 2016 (HBM2) has 732 GB/s memory bandwidth [1], whereas the ddr-4 [2] produces 50GB/s. So the AMD APU would be more than 10x slower at LLM inference. Edit, even old GDDR5 cards like the P40 have bandwidth of 347GB/s. [3] [1] https://www.techpowerup.com/gpu-specs/tesla-p100-pcie-16-gb.c2888 https://www.techpowerup.com/gpu-specs/tesla-p100-pcie-16-gb.... [2] https://news.ycombinator.com/item?id=37163639 https://news.ycombinator.com/item?id=37163639 [3] https://www.techpowerup.com/gpu-specs/tesla-p40.c2878 https://www.techpowerup.com/gpu-specs/tesla-p40.c2878