4 ms·
He means a videocard with 512gb of memory or more.
by holoduke 3y ago
He means a videocard with 512gb of memory or more.
- delecti 3y agoIs 512gb a typo? The current biggest consumer card has 24GB, so we're probably 15 years from a 512GB card (judging from the increase of 4Gb to 24GB between 2012 and 2022).
- flangola7 3y agoThat's why 3-4 years would be impressive
- Legend2440 3y agoWith 4-bit quantization the requirements are more like 64gb. I expect we'll see more unified memory designs like Apple's 128GB M1 Ultra.
- zacmps 3y agoI doubt it to be honest, desktop GPUs use too much power (and hence produce too much heat) to be integrated in that fashion, and any kind of shared memory will be too high latency.
- nullc 3y agoThere are 'desktop' (well server) cpus with 64GB of HBM memory per socket now. And big LLMs can be run on lower memory bandwidth systems (like zen4 chips with 12x ddr5 per socket) at lower performance, but where 1-2TB of ram is no big deal.