3 ms·
> VRAM isn't moving nearly as fast as the models are progressing in size. And it's never going to. The cost will get ever greater to operate these at scale. It
by brucethemoose2 3y ago
> VRAM isn't moving nearly as fast as the models are progressing in size. And it's never going to. The cost will get ever greater to operate these at scale.
It is in at least 2025. AMD (and Intel, maybe) will have M-Pro-Esque APUs that can run a 70B model at very reasonable speeds.
I am pretty sure Intel is going to rock the VRAM boat on desktops as well. They literally have no market to lose, unlike AMD which infuriatingly still artificially segments their high VRAM cards.