3 ms·
I highly doubt that'll be the case. GPU memory and average RAM numbers have only increased by like 3-5x in the past 10 years. Even if we follow that growth rate
by moffkalast 4y ago
I highly doubt that'll be the case. GPU memory and average RAM numbers have only increased by like 3-5x in the past 10 years. Even if we follow that growth rate while making models a lot more compact it'll still be nowhere enough.
It currently takes like what, 400 or 500 GB of VRAM to load ChatGPT according to some estimates? You can't realistically compress that data much further without final performance degradation.
I'd expect something one tenth as good as it running locally in 5 years at best.