3 ms·
What is stopping us from running these models on a PC with 512GB RAM?
by tobyhinloopen 2y ago
What is stopping us from running these models on a PC with 512GB RAM?
- Sharlin 2y agoYou have a point; technically they aren't impossible to run if you have enough system RAM (or hell, SSD/HDD space for that mater). But in practice neither running on the CPU, nor on the GPU by constantly paging data in and out of VRAM, is a very attractive option (~10x slowdown at least).
- tobyhinloopen 2y agoSo the only reason the mac is faster is because the RAM is accessible by its GPU, right? Not because the RAM is faster than regular RAM, because AFAIK it isn't far off from workstation RAM speeds.
- badc0ffee 2y agoThe RAM is faster. 8 DDR5 64GB sticks at 8800 MT/s would in theory give you a maximum of 563.2 GB/s. Whereas the M3 Ultra is 819 GB/s.
- instagib 2y agoCan anyone even get 8 sticks to run together at 8800?
- tobyhinloopen 2y agoI'd say "No". IIRC the current number is somewhere 5-6K