LLM inference processors (GPUs) don't use DDR, it uses special, costly stacked HBM ram soldered to the board. I tested out running Llama on a 512GB machine, it

LLM inference processors (GPUs) don't use DDR, it uses special, costly stacked HBM ram soldered to the board. I tested out running Llama on a 512GB machine, it's rather slow and inefficient. Maybe 1-token/sec.