4 ms·
For enterprise markets, this is table stakes. A lot of datacenter customers will probably ignore this release altogether since there isn't a high-bandwidth opti
by bigyabai 2y ago
For enterprise markets, this is table stakes. A lot of datacenter customers will probably ignore this release altogether since there isn't a high-bandwidth option for systems interconnect.
- pavlov 2y agoThe Mac Studio isn’t meant for data centers anyway? It’s a small and silent desktop form factor — in every respect the opposite of a design you’d want to put in a rack. A long time ago Apple had a rackmount server called Xserve, but there’s no sign that they’re interested in updating that for the AI age.
- bigyabai 2y agoIt's the Ultra chip, the same one that goes into the rackmount Mac Pro. I don't think there's much confusion as to who this is for. > there’s no sign that they’re interested in updating that for the AI age. https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/
- pavlov 2y agoI genuinely forgot the Mac Pro still exists. It’s been so long since I even saw one. And I’ve had every previous Mac tower design since 1999: G4, G5, the excellent dual Xeon, the horrible black trash can… But Apple Silicon delivers so much punch in the Studio form factor, the old school Pro has become very niche. Edit - looks like the new M3 Ultra is only available in Mac Studio anyway? So the existence of the Pro is moot here.
- choilive 2y agonever understood the hate on the trash can. Isn't the mac studio basically the same idea as the trash can but even less upgradeable?
- pavlov 2y agoThe Mac Studio hit a sweet spot in 2023 that the trash can Mac Pro couldn't ten years earlier. It's mostly thanks to the high integration of Apple Silicon and improved device availability and speed of Thunderbolt. The 2013 Mac Pro was stuck forever with its original choice of Intel CPU and AMD GPU. And it was unfortunately prone to overheating due to these same components.
- wtallis 2y agoThe trash can also suffered from hitting the market right around when the industry gave up on making dual-GPU work.
- MBCook 2y agoYep. It was designed for CPU grunt, and came out right when people swapped to wanting tons of GPU grunt. The cooling solution wasn’t designed for huge GPUs. So it couldn’t really be upgraded in ways most people wanted.
- pjmlp 2y agoFolks that want to keep the customisation aspect of Mac Pro hardly see that. In fact a very famous podcaster is still holding out to his.
- TylerE 2y agoThe Studio also hits a sweet spot for home users like me that want tons of IO and no built in input devices.
- Alupis 2y agoOutside of extremely niche use cases, who is racking apple products in 2025?
- nordsieck 2y agoThere's MacMiniVault (nee MacMiniColo) https://www.macminivault.com/ https://www.macminivault.com/ Not sure if they count as niche or not.
- waveringana 2y agogithub for their macos runners (pretty sure theyre m1 minis)
- wpm 2y agoAWS
- kube-system 2y agoEvery provider who offers MacOS in the cloud.
- Alupis 2y agoSo MacOS is still not allowed to be virtualized per the EULA? Wow if that's true...
- kube-system 2y agoMacOS is permitted to be virtualized... as long as the host is a Mac. :)
- wtallis 2y agoThe rackmount Mac Pro is for A/V studios, not datacenters.
- phillco 2y agoDon't forget CI/CD farms for iOS builds, although I think it's much more cost effective to just make Minis or Studios work, despite their nonstandard formfactor
- kridsdale1 2y agoGoogle and Facebook have vast fleets of Minis in custom chassis for this purpose.
- alwillis 2y agoApple recently announced they’re building a new plant in Texas to produce servers. Yes, they need servers for their Private Compute Cloud used by Apple Intelligence, but it doesn’t only need to be for that. From https://www.apple.com/newsroom/2025/02/apple-will-spend-more-than-500-billion-usd-in-the-us-over-the-next-four-years/ https://www.apple.com/newsroom/2025/02/apple-will-spend-more... As part of its new U.S. investments, Apple will work with manufacturing partners to begin production of servers in Houston later this year. A 250,000-square-foot server manufacturing facility, slated to open in 2026, will create thousands of jobs.
- PaulHoule 2y agoThat article says you can connect them through the Thunderbolt 5 somehow to form clusters.
- burnerthrow008 2y agoI wonder if that’s something new, or just the same virtual network interface that’s been around since the TB1 days (a new network interface appears when you connect two Macs with a TB cable)
- PaulHoule 2y agoWell already it is faster than GigE... https://arstechnica.com/gadgets/2013/10/os-x-10-9-brings-fast-but-choppy-thunderbolt-networking/ https://arstechnica.com/gadgets/2013/10/os-x-10-9-brings-fas... Thunderbolt is PCIe-based and I could imagine it being extended to do what https://en.wikipedia.org/wiki/Compute_Express_Link https://en.wikipedia.org/wiki/Compute_Express_Link and https://en.wikipedia.org/wiki/InfiniBand https://en.wikipedia.org/wiki/InfiniBand
- jauntywundrkind 2y agoIts the same host-to-host usb network, I believe. I'm super interested in the clustering capability. At launch people said they were only getting like 11Gbps from their TB4 drive arrays, which was really way less than expected. Apple does kind of advertise that each TB port has its own controllers. Which gives me hope that whatever 1x port can do 6x can do 6x better. AMD's Strix Halo victory feels much more shallow today. Eventually 48GB or 64GB sticks will probably expand Strix Halo to 192 then 256GB. But Strix Halo is super super io starved, is basically a desktop of IO, with no way to easily host-to-host, and Apple absolutely understands that the use of a chip is bounded by what it can connect to. 6x TB5, if even half true, will be utterly outstanding. It's been so so so so cool to see Non-Transparent Bridging atop thunderbolt, so one host can act like a device. Since it's PCIe, that hypothetically would allow amazing RDMA over TB. USB4 mandates host to host networking, but I have no idea how it is implemented and I suspect it's no where near as close to the metal.
- 2y ago
- spiderfarmer 2y agoYou can use Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 or 5 Mac Studios.
- whimsicalism 2y agowhy would you ever want to do that remains an open question
- aurareturn 2y agoProbably some kind of local LLM server. 1TB of 1.6 TB/s memory if you link 2 together. $20k total. Half the price of a single Blackwell chip.
- whimsicalism 2y agowith a vanishingly small fraction of flops and a small fraction of memory bandwidth
- aurareturn 2y agoIt's good enough to run whatever local model you want. 2x 80core GPU is no joke. Linking them together gives it effectively 1.6 TB/s of bandwidth. 1TB of total memory. You can run the full Deepseek 671b q8 model at 40 tokens/s. Q4 model at 80 tokens/s. 37B active params at a time because R1 is MoE. Linking 2 of these together let's you run a model more capable (R1) than GPT4o at a comfortable speed at home. That was simply fantasy a year ago.
- burnerthrow008 2y ago> with a vanishingly small fraction of flops and a small fraction of memory bandwidth Is it though? Wikipedia says [1] an M3 Max can do 14 TFLOPS of FP32, so an M3 Ultra ought to do 28 TFLOPS. nVidia claims [2] a Blackwell GPU does 80 TFLOPs of FP32. So M3 Ultra is 1/3 the speed of a Blackwell. Calling that "a vanishingly small fraction" seems like a bit of an exaggeration. I mean, by that metric, a single Blackwell GPU only has "a vanishingly small fraction" of the memory of an M3 Ultra. And the M3 Ultra is only burning "a vanishingly small fraction" of a Blackwell's electrical power. nVidia likes throwing around numbers like "20 petaFLOPs" for FP4, but that's not real floating point... it's just 1990's-vintage uLaw/aLaw integer math. [1] https://en.wikipedia.org/wiki/Apple_silicon#Comparison_of_M-series_processors https://en.wikipedia.org/wiki/Apple_silicon#Comparison_of_M-... [2] https://resources.nvidia.com/en-us-blackwell-architecture/datasheet https://resources.nvidia.com/en-us-blackwell-architecture/da... Edit: Further, most (all?) of the TFLOPs numbers you see on nVidia datasheets for "Tensor FLOPs" have a little asterisk next to them saying they are "effective" TFLOPs using the sparsity feature, where half the elements of the matrix multiplication are zeroed.
- phonon 2y agoThunderbolt 5 can do bi-directional 80 Gbps....and Mac Studio Ultra has 6 ports...
- cibyr 2y agoThat's still not even competitive with 100G Ethernet on a per-port basis. An overall bandwidth of 480 Gbps pales in comparison with, for example, the 3200 Gbps you get with a P5 instance on EC2.
- nyrikki 2y agoTo add to this GPU servers like supermicro have a 400GBe port per GPU plus more for the CPU.
- kridsdale1 2y agoCost competitive though?
- phonon 2y agoA 3 year reservation of a P5 is over a million dollars though? Not sure how that's comparable....