4 ms·
Repair-friendly form factor for a "Unified Memory" platform. For $2000, you get 128 of system RAM, 96 of which is addressable as VRAM. Only ways of getting 96G
by h14h 2y ago
Repair-friendly form factor for a "Unified Memory" platform.
For $2000, you get 128 of system RAM, 96 of which is addressable as VRAM. Only ways of getting 96GB of VRAM in a desktop are to either:
1. Drop ~$5000 on a (very non-upgradeable) Mac Studio
2. Drop ~$20k on a dual RTX 6000 workstation
For running local LLMs, there's nothing on the market presently even remotely like this.
- deleted 2y ago[deleted]
- seanmcdirmid 2y agoYou can get a M3/M4 Max with 128 GB of RAM as well. The Studio will give you > 128 GB. I have a max with 64GB RAM, which is good enough for 70b models with a 3 bit quant. Even if I had more RAM to run larger models, my GPU would be the bottleneck.
- Dylan16807 2y ago> You can get a M3/M4 Max with 128 GB of RAM as well. To get an M4 Max so you can have 128GB, you need a macbook pro. The cheapest macbook pro with 128GB is $4700. M4 Max does have the benefit of more memory controllers, so it has twice as much memory bandwidth as Ryzen AI Max. But that's a lot of money to pay for it.
- orangecat 2y agoTo get an M4 Max so you can have 128GB, you need a macbook pro. By the time the Framework ships the Mac Studio will have been updated to the M4 Max. Although 128GB will still probably be around $3k.
- seanmcdirmid 2y agoYep. You can get an M3 Max refurbished, but will make some tradeoff with GPU performance.