3 ms·
It enables the use of giant AI models on a personal computer. Might not run too fast though. But at least it's possible at all.
by Sharlin 2y ago
It enables the use of giant AI models on a personal computer. Might not run too fast though. But at least it's possible at all.
- tobyhinloopen 2y agoWhat is stopping us from running these models on a PC with 512GB RAM?
- Sharlin 2y agoYou have a point; technically they aren't impossible to run if you have enough system RAM (or hell, SSD/HDD space for that mater). But in practice neither running on the CPU, nor on the GPU by constantly paging data in and out of VRAM, is a very attractive option (~10x slowdown at least).
- tobyhinloopen 2y agoSo the only reason the mac is faster is because the RAM is accessible by its GPU, right? Not because the RAM is faster than regular RAM, because AFAIK it isn't far off from workstation RAM speeds.
- badc0ffee 2y agoThe RAM is faster. 8 DDR5 64GB sticks at 8800 MT/s would in theory give you a maximum of 563.2 GB/s. Whereas the M3 Ultra is 819 GB/s.
- instagib 2y agoCan anyone even get 8 sticks to run together at 8800?
- tobyhinloopen 2y agoI'd say "No". IIRC the current number is somewhere 5-6K