Indeed, GPUs are not much better. For example, the NVIDIA H100 SXM has roughly the same price per transistor as the A100 SXM. The gains between generations were

Indeed, GPUs are not much better. For example, the NVIDIA H100 SXM has roughly the same price per transistor as the A100 SXM. The gains between generations were roughly equally split between (1) better chip design, (2) higher clock speed / power consumption, and (3) more transistors. Cost per transistor did not meaningfully improve.