5 ms·
Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They
by ksec 2y ago
Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task.
They now bump it to 512GB. Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.
- dwighttk 2y agoMaybe .1% of tasks need this RAM, why are they charging so much?
- regularfry 2y agoThe narrower the niche, the more you can charge.
- rewtraw 2y agobecause that's how much its worth
- internetter 2y agoIts not though. For consumer computers somewhere 1k-4k there's nothing better. But for the price of 512gb of RAM you could buy that + a crazy CPU + 2x 5090s by building your own. The market fit is "needs power; needs/wants macOS; has no budget" which is so incredibly niche. But in terms of raw compute output there's absolutely no chance this is providing bang for buck
- jeffhuys 2y agoDo you understand that it's UNIFIED RAM, so it doubles as vRAM? I would love to know what computer you can build for <10k with 0.5TB of VRAM.
- kjreact 2y ago2x 5090s would only give you 64GB of memory to work with re:LLM workloads, which is what people are talking about in this thread. The 512GB of system RAM you’re referring to would not be useful in this context. Apple’s unified memory architecture is the part you’re missing.
- DrBenCarson 2y agoHow much VRAM do you get on those 2x 5090s? How much would it cost to get up to 512gb?
- pier25 2y agoBecause the minority that needs that much RAM can't work without it. In the media composing world they use huge orchestral templates with hundreds and hundreds of tracks with millions of samples loaded into memory.
- A4ET8a8uTh0_v2 2y agoI think the answer is because they can ( there is a market for it ). The benefit to a crazy person like me that with this addition, I might be able to grab 128gb version at a lower price.
- znpy 2y agobecause they know there will be a large amount of people that don't know this much ram but they'll buy it anyway.
- cjbgkagh 2y agoI don't need 512GB of RAM but the moment I do I'm certain I'll have bigger things to worry about than a $10K price tag.
- almostgotcaught 2y agoThis is Pascal's wager written in terms of ... RAM. The original didn't make sense and neither does this iteration.
- cjbgkagh 2y agoI would still wait until I need it before buying it…
- agloe_dreams 2y agoBecause the .1% is who will buy it? I mean, yeah, supply and demand. High demand in a niche with no supply currently means large margins. I don't think anyone commercially offers nearly this much unified memory or NPU/GPUs with anything near 512GB of memory.
- madeofpalk 2y agoMaybe because .1% of tasks need this RAM, it attracts a .1% price tag
- Spooky23 2y agoWith all things semiconductor, low volume = higher cost (and margin). The people who need the crazy resource can tie it to some need that costs more. You’d spend like $10k running a machine with similar capabilities in AWS in a month.
- Sharlin 2y agoIt enables the use of giant AI models on a personal computer. Might not run too fast though. But at least it's possible at all.
- tobyhinloopen 2y agoWhat is stopping us from running these models on a PC with 512GB RAM?
- Sharlin 2y agoYou have a point; technically they aren't impossible to run if you have enough system RAM (or hell, SSD/HDD space for that mater). But in practice neither running on the CPU, nor on the GPU by constantly paging data in and out of VRAM, is a very attractive option (~10x slowdown at least).
- tobyhinloopen 2y agoSo the only reason the mac is faster is because the RAM is accessible by its GPU, right? Not because the RAM is faster than regular RAM, because AFAIK it isn't far off from workstation RAM speeds.
- badc0ffee 2y agoThe RAM is faster. 8 DDR5 64GB sticks at 8800 MT/s would in theory give you a maximum of 563.2 GB/s. Whereas the M3 Ultra is 819 GB/s.
- instagib 2y agoCan anyone even get 8 sticks to run together at 8800?
- tobyhinloopen 2y agoI'd say "No". IIRC the current number is somewhere 5-6K
- InTheArena 2y agoEvery single AI shop on the planet is trying to figure out if there is enough compute or not to make this a reasonable AI path. If the answer is yes, that 10k is a absolute bargain.
- internetter 2y agoNo AI shop is buying macs to use as a server. Apple should really release some server macOS distribution, maybe even rackable M-series chips. I believe they have one internally.
- jerjerjer 2y agoWhy would any business pay Apple Tax for a backend, server product?
- 827a 2y agoIs this actually true? Were people doing this with the 192gb of the M2 Ultra? I'm curious to learn how AI shops are actually doing model development if anyone has experience there. What I imagined was: Its all in the "cloud" (or, their own infra), and the local machine doesn't matter. If it did matter, the nvidia software stack is too important, especially given that a 512gb M3 Ultra config costs $10,000+.
- DrBenCarson 2y agoYou’re largely correct for training models Where this hardware shines is inference (aka developing products on top of the models themselves)
- 827a 2y agoTrue. But with Project Digits supposedly around the corner, which supposedly costs $3,000 and supports ConnectX and runs Blackwell; what's the over-under on just buying two of those at about half the price of one maxed M3 Ultra Mac Studio?
- HPsquared 2y agoLLMs easily use a lot of RAM, and these systems are MUCH, MUCH cheaper (though slower) than a GPU setup with the equivalent RAM. A 4-bit quantization of Llama-3.1 405b, for example, should fit nicely.
- segmondy 2y agoThe question will be how it will perform. I suspect Deepseek, Llama405B demonstrated the need for larger memory. Right now folks could build an epyc system with that much ram or more to run Deepseek at about 6 tokens/sec for a fraction of that cost. However not everyone is a tinker, so there's a market for this for those that don't want to be bothered. You say "AI Gold rush" like it's a bad thing, it's not.
- bloppe 2y agoBig question is: Does the $10k price already reflect Trump's tariffs on China? Or will the price rise further still..
- MR4D 2y agoRemember, that RAM is also VRAM, so 1/2 terabyte of VRAM ain’t cheap. By comparison, Apple is a downright bargain!
- tobyhinloopen 2y agoIt doesn't have the bandwidth of dedicated GPU VRAM.
- leetharris 2y agoYes it does. It is just short of a 4090's memory bandwidth. It's still far away from an H100 though.
- heeen2 2y agoWouldn't multiple rtx gpus also have more bandwidth