These won't be smaller I guess. Given we keep the number of parameters same. Pre LLM era (let's say 2020), the hardware used to look decently powerful for most

These won't be smaller I guess. Given we keep the number of parameters same. Pre LLM era (let's say 2020), the hardware used to look decently powerful for most use cases (disks in hundreds of GBs, dozen or two of RAM and quad or hex core processors) but with the advent of LLMs, even disk drives start to look pretty small let alone compute and memory.