6 ms·

I’d been wondering how they were going to handle expandable memory with the M chip, since the integrated memory seemed pretty central to the design - seems like

I’d been wondering how they were going to handle expandable memory with the M chip, since the integrated memory seemed pretty central to the design - seems like the answer is, “they’re not.” Be interested to see if PCI expansion is sufficient to satisfy the Max Pro market.

pwthornton 3y ago
The approach they took is certainly enough to satisfy the video and related markets. It won't help those who need truly staggering amounts of ram for their workloads. It may not be a big use case for the Intel Mac Pro, but it a use case nonetheless. For video editing, color grading, audio editing, 3D animation, etc. this new machine seems really strong. I am not sure if there is anything beyond that, however.
- WWLink 3y ago
  I think that was deliberate. I am kinda amused at how they keep on narrowing the scope of what the mac pro is intended to be used for lol.
  - gumby 3y ago
    They probably have very good data on how their machines are used, and are optimizing for the fat part of the market. This may not be a great machine for training models, which is what I happen care about (I couldn't care less about video). I wonder how big the model generation market actually is though.
    - rodgerd 3y ago
      I do wonder in these conversations how many people are actual users of these things - I'm not, I got a Studio last year to manage my photography addition better, and the Pro is pointless for that - but you'd assume that Apple have talked to major Pro users, and they've said things like "oh, no for training a model I just spin up terabyte instances on AWS, why would I do that on a workstation?", or whatever.
      - gumby 3y ago
        I assume the audience is the Disneys/Pixars of the world, and the people making ads, etc in 8K. They presumably need that I/O. For them the absurdly expensive monitor is a rounding error, and the extra local disk is worth it over running over a SAN. Perhaps also some computationally adjacent applications like CFD or weather prediction. I struggle to think of anyone else with such an application. There are applications that could benefit from the features but it wouldn’t be cost effective (e.g. something more systolic where it’s better to rent cloud service, even if it were nominally slower, because once you’re done you don’t have the hardware lying around).
- USB5 3y ago
  >3D animation Disagree. All the good GPU-based rendering engines need CUDA, and none of them are optimized for Apple silicon. Octane (the one in the demo) is trash, only good for fancy titles and that sort of thing.
  - haldean 3y ago
    Octane, Redshift, and eevee/Cycles all work on Metal, and Renderman and Mantra are CPU-only for now -- RM/Mantra have beta-versions that are GPU enabled, and Karma is nvidia-only and RM XPU doesn't work on macs at all yet, but I don't think many people are using either in production very much yet. I think the only GPU renderer people are using in production pipelines that's really locked to NVIDIA chips is Arnold, right?
    - USB5 3y ago
      I can't speak for cycles, but Redshift is very slow on OpenCL. I've never heard of using it on Metal. I know it was written for CUDA and didn't even support OpenCL for a long time. Do you know how it runs on Metal? I don't think V-Ray GPU will run on anything but Nvidia, and if it will then it's definitely slow.
      - haldean 3y ago
        Ah yeah forgot about V-Ray. I haven't tried Redshift on Metal, nope; all my work has been Renderman and Cycles recently. Curious to hear how well it works, people online seem to say it's pretty fast but hard to tell marketing copy from actual performance when it comes to renderers. FWIW Cycles is definitely slower on my M2 than it was on my 3080 but it's not a huge difference -- maybe 20% slower? I still have to let the render run overnight either way haha.
        USB5 3y ago
        Interesting. Which M2 machine are you using?
bee_rider 3y ago
Everything seems slightly parallel-universe in Apple world, looking at it from over here in x86/linux land. I can see why they’d decide to go in-package with their memory, it is really very fast. And 192GB of memory is not a huge amount of memory in server/HPC land, but it is still a decent chunk of space. You could load up a Mac Pro with a bunch of PCIe nvme drives or something, I wonder if it would really be that hard to adapt to that. I certainly wouldn’t turn down the chance to try, haha.
- AprilArcus 3y ago
  NVMe wouldn't give you the best latency, but a 16x PCIe card loaded with DRAM and addressable as a scratch disk doesn't sound bananas. I wonder why Apple didn't market something like that as a first party solution.
  - nickpeterson 3y ago
    I expect that expansion cards from Apple are going to be the way they differentiate the mac pro line. I'm betting that Apple is going to come out with discrete GPU's based on their integrated graphics and sell them as add-ons for the Pros for ML/AI workloads and likely to help with normal stuff like rendering in Autodesk/Adobe/Finalcut type apps. That said, it isn't the bet I'd make if I was tasked with purchasing for a company. I'd get the studios in a reasonably high config and then sell them every 2-3 years and replace with newer model. You'd pass the pro's in perf pretty readily from CPU bumps alone and likely recoupe a lot of your investment.
  - bee_rider 3y ago
    A RAM disk formatted as swap? Probably would work. Funny concept. Let’s lie to OS and tell it that this memory is actually a disk. Then, it will lie to the programs, and tell them that that disk is actually memory.
- adanto6840 3y ago
  I've got a PCIe RAID card loaded with NVMe SSDs on my 2019 MP. It's really great. Unfortunately, the latest macOS versions are not amenable to using other drives as a boot drive (IIRC may be related to secure enclave stuff?). Lots of (third party) macOS apps don't play real well with additional storage devices either, and the OS itself has some (relatively minor) hurdles to deal with, too. For project/data/"working directory"/general filesystem usage, it's great. That said, up to now it's probably provided me with ~ fastest possible drive speeds available for macOS. For business/dev purposes, I really can't complain (except about price & lack of modern Nvidia support - but alas), even though I'm already using M1 Max these days - the single-core speed is just too good.
tiffanyh 3y ago
Mac Pro = Mac Studio + Expansion Cards I have to imagine this will be a huge disappointment to some, because 192GB of shared memory is way less than the 1.5TB of RAM available on the "old" Mac Pro.
- peoplearepeople 3y ago
  Perhaps someone will come out with a new PCIe card with a load of RAM slots on it, and then writes a kernel driver to map the pcie card pages to appear as regular pages
  - Kon-Peki 3y ago
    That's an interesting idea. But do you actually need to go all the way to making the extra memory appear as a contiguous part of the system memory? I am thinking about CUDA unified memory and perhaps some parallels to your idea. The number of applications that are likely to use the extra memory is probably pretty small. So if you have some sort of framework that those developers can integrate into their software, you've probably done everything you need to do.
  - PlutoIsAPlanet 3y ago
    You could use them as some kind of swap or ram disk, but I don't believe as normal RAM would be possible due to how CPUs work.
  - daemontus 3y ago
    Someone already has :) It's called CXL. Both Intel and AMD are diving head first into it for servers. Apple would just need to support it...
kllrnohj 3y ago
The integrated memory design doesn't prevent doing it over a dimm slot. It seems more they just didn't want to deal with a ddr5 or some bespoke connector.
- qwytw 3y ago
  Also allowing users to upgrade RAM themselves would lower Apples margin and their computers would remain usable for much longer which would result in even less profits... Now Apple can release a 512 GB version in a year or two then a 1TB one etc. The technical issues are totally insignificant compared to this. Edit: having said this extra memory for Mac Pro seems cheap as f** by Apple standards. Just $800 for 64 -> 128GB. 8 -> 24GB for Mac mini/Air is $400 and you only get 48GB for $800 in a MBP.
sliken 3y ago
192GB ram does seem like enough for a large fraction of the potential market, especially @ 800GB/sec which makes it much more usable than similar amounts of ram on Intel/AMD desktops at 1/8th the bandwidth.
- StillBored 3y ago
  Except that my fairly modest upper midrange desktop has ~1.2TB/sec of memory bandwidth... Cause, I just added the GPU and system RAM bandwidth numbers together. Which is what needs to be kept in mind with much of this. Yes that is a lot of memory bandwidth and its hella useful for some subset of users, but its shared, and largely pointless for a lot of CPU bound tasks. But OTOH, may not be enough for many GPU bound ones. It also assumes that pretty much every other CPU manufacture on the planet are idiots for optimizing for latency and putting in large caches to compensate (aka the desktop parts from AMD/intel have only _two_ channels, vs the 8+ in the server/workstation parts) and price discriminating for the parts that have more CPU bandwidth. AKA, you can get amd machines in the same ballpark (or possibly faster depending on how fast you can get 24 channels of DDR5 to run). So, I'm not saying which is better because its likely workload dependent, but to claim its a blanket insurmountable advantage is questionable. Particularly since the price ranges we are talking about a similar machine is probably a 64 core threadripper plus a fat nvidia GPU or four and the shear core count and raw GPU compute is probably a win in most workloads.
  - USB5 3y ago
    I just built a workstation with a 32 core Threadripper Pro, 128 GB ECC RAM, Thunderbolt 4, and an RTX 4090 for $6,500.
    - sliken 3y ago
      Likely better on any workloads that are GPU heavy and fit in 24GB of vram. Two data points from Apple: a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200 b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600 Likely 1/4th the size, 1/4th the power consumption, and 4x the ram bandwidth. Have you by chance played with any LLMs? Just saw a post that someone managed 5 tokens/sec with the llama 65B model.

pwthornton 3y ago

The approach they took is certainly enough to satisfy the video and related markets. It won't help those who need truly staggering amounts of ram for their workloads. It may not be a big use case for the Intel Mac Pro, but it a use case nonetheless. For video editing, color grading, audio editing, 3D animation, etc. this new machine seems really strong. I am not sure if there is anything beyond that, however.

WWLink 3y ago
I think that was deliberate. I am kinda amused at how they keep on narrowing the scope of what the mac pro is intended to be used for lol.
- gumby 3y ago
  They probably have very good data on how their machines are used, and are optimizing for the fat part of the market. This may not be a great machine for training models, which is what I happen care about (I couldn't care less about video). I wonder how big the model generation market actually is though.
  - rodgerd 3y ago
    I do wonder in these conversations how many people are actual users of these things - I'm not, I got a Studio last year to manage my photography addition better, and the Pro is pointless for that - but you'd assume that Apple have talked to major Pro users, and they've said things like "oh, no for training a model I just spin up terabyte instances on AWS, why would I do that on a workstation?", or whatever.
    - gumby 3y ago
      I assume the audience is the Disneys/Pixars of the world, and the people making ads, etc in 8K. They presumably need that I/O. For them the absurdly expensive monitor is a rounding error, and the extra local disk is worth it over running over a SAN. Perhaps also some computationally adjacent applications like CFD or weather prediction. I struggle to think of anyone else with such an application. There are applications that could benefit from the features but it wouldn’t be cost effective (e.g. something more systolic where it’s better to rent cloud service, even if it were nominally slower, because once you’re done you don’t have the hardware lying around).
USB5 3y ago
>3D animation Disagree. All the good GPU-based rendering engines need CUDA, and none of them are optimized for Apple silicon. Octane (the one in the demo) is trash, only good for fancy titles and that sort of thing.
- haldean 3y ago
  Octane, Redshift, and eevee/Cycles all work on Metal, and Renderman and Mantra are CPU-only for now -- RM/Mantra have beta-versions that are GPU enabled, and Karma is nvidia-only and RM XPU doesn't work on macs at all yet, but I don't think many people are using either in production very much yet. I think the only GPU renderer people are using in production pipelines that's really locked to NVIDIA chips is Arnold, right?
  - USB5 3y ago
    I can't speak for cycles, but Redshift is very slow on OpenCL. I've never heard of using it on Metal. I know it was written for CUDA and didn't even support OpenCL for a long time. Do you know how it runs on Metal? I don't think V-Ray GPU will run on anything but Nvidia, and if it will then it's definitely slow.
    - haldean 3y ago
      Ah yeah forgot about V-Ray. I haven't tried Redshift on Metal, nope; all my work has been Renderman and Cycles recently. Curious to hear how well it works, people online seem to say it's pretty fast but hard to tell marketing copy from actual performance when it comes to renderers. FWIW Cycles is definitely slower on my M2 than it was on my 3080 but it's not a huge difference -- maybe 20% slower? I still have to let the render run overnight either way haha.
      - USB5 3y ago
        Interesting. Which M2 machine are you using?

bee_rider 3y ago

Everything seems slightly parallel-universe in Apple world, looking at it from over here in x86/linux land. I can see why they’d decide to go in-package with their memory, it is really very fast. And 192GB of memory is not a huge amount of memory in server/HPC land, but it is still a decent chunk of space. You could load up a Mac Pro with a bunch of PCIe nvme drives or something, I wonder if it would really be that hard to adapt to that. I certainly wouldn’t turn down the chance to try, haha.

AprilArcus 3y ago
NVMe wouldn't give you the best latency, but a 16x PCIe card loaded with DRAM and addressable as a scratch disk doesn't sound bananas. I wonder why Apple didn't market something like that as a first party solution.
- nickpeterson 3y ago
  I expect that expansion cards from Apple are going to be the way they differentiate the mac pro line. I'm betting that Apple is going to come out with discrete GPU's based on their integrated graphics and sell them as add-ons for the Pros for ML/AI workloads and likely to help with normal stuff like rendering in Autodesk/Adobe/Finalcut type apps. That said, it isn't the bet I'd make if I was tasked with purchasing for a company. I'd get the studios in a reasonably high config and then sell them every 2-3 years and replace with newer model. You'd pass the pro's in perf pretty readily from CPU bumps alone and likely recoupe a lot of your investment.
- bee_rider 3y ago
  A RAM disk formatted as swap? Probably would work. Funny concept. Let’s lie to OS and tell it that this memory is actually a disk. Then, it will lie to the programs, and tell them that that disk is actually memory.
adanto6840 3y ago
I've got a PCIe RAID card loaded with NVMe SSDs on my 2019 MP. It's really great. Unfortunately, the latest macOS versions are not amenable to using other drives as a boot drive (IIRC may be related to secure enclave stuff?). Lots of (third party) macOS apps don't play real well with additional storage devices either, and the OS itself has some (relatively minor) hurdles to deal with, too. For project/data/"working directory"/general filesystem usage, it's great. That said, up to now it's probably provided me with ~ fastest possible drive speeds available for macOS. For business/dev purposes, I really can't complain (except about price & lack of modern Nvidia support - but alas), even though I'm already using M1 Max these days - the single-core speed is just too good.

tiffanyh 3y ago

Mac Pro = Mac Studio + Expansion Cards I have to imagine this will be a huge disappointment to some, because 192GB of shared memory is way less than the 1.5TB of RAM available on the "old" Mac Pro.

peoplearepeople 3y ago
Perhaps someone will come out with a new PCIe card with a load of RAM slots on it, and then writes a kernel driver to map the pcie card pages to appear as regular pages
- Kon-Peki 3y ago
  That's an interesting idea. But do you actually need to go all the way to making the extra memory appear as a contiguous part of the system memory? I am thinking about CUDA unified memory and perhaps some parallels to your idea. The number of applications that are likely to use the extra memory is probably pretty small. So if you have some sort of framework that those developers can integrate into their software, you've probably done everything you need to do.
- PlutoIsAPlanet 3y ago
  You could use them as some kind of swap or ram disk, but I don't believe as normal RAM would be possible due to how CPUs work.
- daemontus 3y ago
  Someone already has :) It's called CXL. Both Intel and AMD are diving head first into it for servers. Apple would just need to support it...

kllrnohj 3y ago

The integrated memory design doesn't prevent doing it over a dimm slot. It seems more they just didn't want to deal with a ddr5 or some bespoke connector.

qwytw 3y ago
Also allowing users to upgrade RAM themselves would lower Apples margin and their computers would remain usable for much longer which would result in even less profits... Now Apple can release a 512 GB version in a year or two then a 1TB one etc. The technical issues are totally insignificant compared to this. Edit: having said this extra memory for Mac Pro seems cheap as f** by Apple standards. Just $800 for 64 -> 128GB. 8 -> 24GB for Mac mini/Air is $400 and you only get 48GB for $800 in a MBP.

sliken 3y ago

192GB ram does seem like enough for a large fraction of the potential market, especially @ 800GB/sec which makes it much more usable than similar amounts of ram on Intel/AMD desktops at 1/8th the bandwidth.

StillBored 3y ago
Except that my fairly modest upper midrange desktop has ~1.2TB/sec of memory bandwidth... Cause, I just added the GPU and system RAM bandwidth numbers together. Which is what needs to be kept in mind with much of this. Yes that is a lot of memory bandwidth and its hella useful for some subset of users, but its shared, and largely pointless for a lot of CPU bound tasks. But OTOH, may not be enough for many GPU bound ones. It also assumes that pretty much every other CPU manufacture on the planet are idiots for optimizing for latency and putting in large caches to compensate (aka the desktop parts from AMD/intel have only _two_ channels, vs the 8+ in the server/workstation parts) and price discriminating for the parts that have more CPU bandwidth. AKA, you can get amd machines in the same ballpark (or possibly faster depending on how fast you can get 24 channels of DDR5 to run). So, I'm not saying which is better because its likely workload dependent, but to claim its a blanket insurmountable advantage is questionable. Particularly since the price ranges we are talking about a similar machine is probably a 64 core threadripper plus a fat nvidia GPU or four and the shear core count and raw GPU compute is probably a win in most workloads.
- USB5 3y ago
  I just built a workstation with a 32 core Threadripper Pro, 128 GB ECC RAM, Thunderbolt 4, and an RTX 4090 for $6,500.
  - sliken 3y ago
    Likely better on any workloads that are GPU heavy and fit in 24GB of vram. Two data points from Apple: a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200 b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600 Likely 1/4th the size, 1/4th the power consumption, and 4x the ram bandwidth. Have you by chance played with any LLMs? Just saw a post that someone managed 5 tokens/sec with the llama 65B model.