5 ms·

The Ampere Altra Max Review: Pushing It to 128 Cores per Socket

91 points by pella 5y ago

Syonyk 5y ago
That was about as expected. Going to 128 cores for cache-resident compute workloads helps, it hurts for anything else. That is a seriously cache starved chip... only 16MB SLC? Even with 1MB of L2... that's still a cache starved chip. The new iPhone has almost as much cache on it! That said, I would absolutely love some sort of mid-range ARM desktop (that isn't Apple). There are the "small workstation" class ARM boxes (quite a few thousand dollars, 64 cores or so), there are the various small board ARM computers (4-6 cores, $100), and... there's really nothing in the middle. Give me 4 X2 cores, a cluster of A74s clocked hard, and put it in a NUC type case with at least 16GB of RAM, preferably more, and some NVMe.
- Teknoman117 5y ago
  Have you seen the honeycomb lx2? It has 16 cortex a72 cores @ 2.0 GHz, USB 3.0, quad 10G Ethernet, PCIe, and supports up to 64 GiB of DDR4-3200 in SODIMM format. Performance wise it’s around a x86 quad core from a few years ago. https://shop.solid-run.com/product/SRLX216S00D00GE064H09CH/ https://shop.solid-run.com/product/SRLX216S00D00GE064H09CH/ It $750, but it’s the only thing that comes to mind. There’s a $1000 variant that adds 40 GbE support.
  - mhh__ 5y ago
    That's a lotta money for not much oomph. Single threaded performance is probably going to hurt, not everything is a SPEC benchmark ;)
    - deleted 5y ago
      [deleted]
  - adrian_b 5y ago
    Unfortunately, the more than 6-year old Cortex-A72 cores are really obsolete. Especially when having many ARM cores, it is desirable to have cores that support at least the ARMv8.2-A ISA, which corrects the initial mistake of the 64-bit ARM ISA of omitting the atomic instructions. Actually the mistake was corrected in ARMv8.1-A, but all easily available ARM cores support either ARMv8.2-A (Cortex-A55, Cortex-A75 and newer, also NVIDIA Carmel) or ARMv8.0-A (Cortex-A73 and older). If you want to experiment with many ARM threads and how to organize the synchronization and communication between them, the results on ancient Cortex-A72 cores can be misleading. Sadly the vast majority of ARM vendors offer only very old ARM cores. Except for the chips intended for mobile phones, the only devices with not too old ARM cores are slow devices with quadruple Cortex-A55 cores. If you want more threads than that, the best that can be found would be an 8-core NVIDIA Xavier AGX or a 6-core NVIDIA Xavier NX. An 8-core Xavier might actually have a speed not much different than a 16 core Cortex-A72, even if the NVIDIA cores are relatively slow when compared with more recent ARM cores, i.e. Carmel is a little slower than Cortex-A75. Buying a Honeycomb lx2 makes sense only when your interest is in the multiple 10G Ethernet and you do not care about what kind of CPU is provided on the board, i.e. you are not interested in the board as a software development platform for future ARM devices.
    - microtonal 5y ago
      Sadly the vast majority of ARM vendors offer only very old ARM cores. Except for the chips intended for mobile phones, the only devices with not too old ARM cores are slow devices with quadruple Cortex-A55 cores. I fear that Asahi Linux on a Mac Mini (especially when the M1X comes out with possibly 6 or 8 high-performance cores) is going to be the best option for a modern Linux ARM workstation for a while.
      - Teknoman117 5y ago
        The largest hurdles in these sorts of projects tend to the GPU and video codecs. I wish them luck and hope they succeed in entirety, but I can only early adopt so many things. Hopefully Apple doesn't reinvent their GPU every single year.
  - JohnJamesRambo 5y ago
    Can anyone provide insight into who buys that sort of device? It seems kind of pricey for what you get.
    - smackeyacky 5y ago
      I'm keen on an ARM workstation. I do all my dev work on Linux and deploy everything to Linux hosts. AWS offers ARM hosts at half the cost of amd64 ones, you can cross compile but I would be happier if my build box was the same processor. What might work is some kind of local headless cloud machine running a pile of VMs until somebody builds something I can hang two big monitors off. Laptops are too constraining.
      - exikyut 5y ago
        I just thought for a moment it could be interesting to try and convince Amazon to do for their ARM kit what Google have done with their TPU chipset (https://cloud.google.com/edge-tpu https://cloud.google.com/edge-tpu, https://google.com/search?q=tpu+board https://google.com/search?q=tpu+board). Then I realized that - If such an initiative succeeded, the developer side of the fence would lose ground somewhere, because Amazon. - It would never actually happen in the first place, because ARM is so fast-moving now that everyone's doing everything they can to maintain their first-mover advantage, which of course includes being as secretive as possible. - In the ideal case where an engineer actually considered the situation, it would be all too easy (given the iteration that's happening right now) for them to counter-argue that it makes the most sense to just buy the Altera kit available on the retail market because it's technically the closest to what Amazon are using internally. The evolutionary focus is entirely on servers because of the appreciable overheads associated with that market that can be tapped into to drive the R&D process - especially given the chip shortages at the moment. Inn the same vein, there are not likely to be any consumer-class implementations (or new products) until the supply chain has had a chance to figure itself out, which is probably not going to happen for a while. :(
        smackeyacky 5y ago
        This is a shame for a lot of reasons. Chiefly that I would really like a graviton2 local server to get as close to replicating the metal my server will run on. Perhaps Amazon could just ship a PCI card with a system on it or something like that. Its interesting to me that all of a sudden amd64 seems like a legacy system. I can hear the fan in my old proliant dev server from here and while I am somewhat sentimental about it, its definitely time to move on, preferably withou Apple involved. Maybe its time to start a 1980s style workstation company again.
- Uberphallus 5y ago
  My thoughts exactly. Can be also useful for certain high volume workloads as you pass through data once and cache isn't all that useful. But otherwise it is severely deprived of cache for anything in the middle.
- rowanG077 5y ago
  well soon you can get a mac mini. It's doubtful you could get a better bang for your buck elsewhere.
- TradingPlaces 5y ago
  I think the upcoming Qualcomm-Nuvia chip will be what you want
- volta83 5y ago
  You can get an Ampere Altra Q32-17 with 32 cores at 45W TDP.
- dragontamer 5y ago
  > That was about as expected. Going to 128 cores for cache-resident compute workloads helps, it hurts for anything else. It looks like it has really good SPECjbb? I assume Java is non-cache resident? (Especially if SPECjbb is designed to emulate server-scale tasks). If anything, the SPECjbb results come full circle IMO: the large caches of Intel/AMD no longer help, and you're back to latency bound (Maybe??). Its very curious to me that this 16MB / 128core part is out performing the 32MB / 80core config on so many benchmarks. I think you're right in that this processor has very little cache (and no hyperthreading). But it seems to just get around that issue by having a crapton of cores. If things are waiting on memory, it just brute-forces more cores to do more things. I can't help but think that SMT2 or SMT4 would really help here. I have to imagine that most of those cores are waiting on RAM, given this architecture. Or maybe they ran the math and decided that 128 threads was enough to saturate the memory channels already? I dunno, memory latency is always a killer, especially since this architecture has so little cache. My instincts suggest that SMT would still help.
  - cogman10 5y ago
    IDK what SPECjbb does, but it really sort of depends on a lot of factors around how the JVM was setup. The JVM CAN be fairly cache friendly in the case where allocations are used relatively close to one another. The JVM will (typically) keep memory allocated at the same time in roughly the same memory region.
    - dragontamer 5y ago
      https://www.spec.org/jbb2015/docs/designdocument.pdf https://www.spec.org/jbb2015/docs/designdocument.pdf SPECjbb is a large, Java benchmark. The idea is to create a "realistic enterprise system", and then measure the number of requests before 100ms latency is violated (aka: critical-jOPs), as well as maximum throughput (aka: max-jOPs). The general model is on a hypothetical Supermarket: each time a POS system scans things, the Headquarters has to double-check the status of coupons, report back to the POS system and determine if coupons are still valid. These hypothetical messages are GZipped XML that's getting passed back and forth between distinct JVMs, across the Grizzly-based non-blocking HTTPS server (default). The benchmark artificially shoves more-and-more requests through the system until the 100ms response time is violated (ex: the system is beginning to show signs of being overburdened). Then it continues to shove more data into the system until the maximum throughput is measured. It involves many JVMs doing many different tasks and communicating over Java-APIs. Thread pools, IPC, sockets, XML, HTTPS, encryption, the whole shebang.
- LeifCarrotson 5y ago
  I'm curious what workloads are suitable. You do have eight channels of 25 GB/s DDR4-3200 bandwidth (theoretically 200 GB/s) and 128 lanes of 2 GB/s PCIe 4.0(theoretically another 256 GB/s) to feed the thing, but getting it to make efficient use of those peripherals is another question altogether.
MayeulC 5y ago
As we are witnessing multiple relatively small companies entering the server CPU market, it would be a really great thing if they could standardize on a compatible socket. This way, instead of designing a motherboard for a single CPU vendor in a small market, integrators could target all of them at once. Intel reportedly keeps changing sockets to please OEMs who want to sell motherboards; I would be happy to see something change in that landscape. Otherwise, I'm extremely pleased to see more options are starting to become viable (again).
- cameron_b 5y ago
  If you abstract that ask just a bit there is already an answer, although it's more of an industrial answer. the COM-Express Spec is something of this Universal socket [0] And while all of that link above looks like a creamy beige box cringefest, the COM-HPC[1] interface is being used for some heavy-hitting systems including Xeon and Ampere Altra [2] So, yea, you can get this in something more interoperable. 0 - https://www.picmg.org/openstandards/com-express/ https://www.picmg.org/openstandards/com-express/ 1 - https://www.picmg.org/openstandards/com-hpc/ https://www.picmg.org/openstandards/com-hpc/ 2 - https://www.ipi.wiki/pages/com-hpc-altra https://www.ipi.wiki/pages/com-hpc-altra
  - dragontamer 5y ago
    > If you abstract that ask just a bit there is already an answer, although it's more of an industrial answer. the COM-Express Spec is something of this Universal socket [0] So are PCIe slots, except GPUs seem to have won on that front (RIP Xeon Phi). Moving forward: PCIe 5.0 is having more-and-more atomic instructions execute over the PCIe 5.0 specification. Its looking like PCIe 5.0 will have cache-cohesive, in particular, the Compute Express Link (CXL). So PCIe is looking more-and-more like a "standardized socket", rather than an I/O mechanism.
jeffbee 5y ago
Does that LLVM comparison make sense? It seems to target the native platform, which doesn't seem like it would make for a valid comparison between x86 and arm.
- tyingq 5y ago
  Maybe for the relative 1 socket / 2 socket differences.

91 points by pella 5y ago

That was about as expected. Going to 128 cores for cache-resident compute workloads helps, it hurts for anything else. That is a seriously cache starved chip... only 16MB SLC? Even with 1MB of L2... that's still a cache starved chip. The new iPhone has almost as much cache on it! That said, I would absolutely love some sort of mid-range ARM desktop (that isn't Apple). There are the "small workstation" class ARM boxes (quite a few thousand dollars, 64 cores or so), there are the various small board ARM computers (4-6 cores, $100), and... there's really nothing in the middle. Give me 4 X2 cores, a cluster of A74s clocked hard, and put it in a NUC type case with at least 16GB of RAM, preferably more, and some NVMe.

Teknoman117 5y ago
Have you seen the honeycomb lx2? It has 16 cortex a72 cores @ 2.0 GHz, USB 3.0, quad 10G Ethernet, PCIe, and supports up to 64 GiB of DDR4-3200 in SODIMM format. Performance wise it’s around a x86 quad core from a few years ago. https://shop.solid-run.com/product/SRLX216S00D00GE064H09CH/ https://shop.solid-run.com/product/SRLX216S00D00GE064H09CH/ It $750, but it’s the only thing that comes to mind. There’s a $1000 variant that adds 40 GbE support.
- mhh__ 5y ago
  That's a lotta money for not much oomph. Single threaded performance is probably going to hurt, not everything is a SPEC benchmark ;)
  - deleted 5y ago
    [deleted]
- adrian_b 5y ago
  Unfortunately, the more than 6-year old Cortex-A72 cores are really obsolete. Especially when having many ARM cores, it is desirable to have cores that support at least the ARMv8.2-A ISA, which corrects the initial mistake of the 64-bit ARM ISA of omitting the atomic instructions. Actually the mistake was corrected in ARMv8.1-A, but all easily available ARM cores support either ARMv8.2-A (Cortex-A55, Cortex-A75 and newer, also NVIDIA Carmel) or ARMv8.0-A (Cortex-A73 and older). If you want to experiment with many ARM threads and how to organize the synchronization and communication between them, the results on ancient Cortex-A72 cores can be misleading. Sadly the vast majority of ARM vendors offer only very old ARM cores. Except for the chips intended for mobile phones, the only devices with not too old ARM cores are slow devices with quadruple Cortex-A55 cores. If you want more threads than that, the best that can be found would be an 8-core NVIDIA Xavier AGX or a 6-core NVIDIA Xavier NX. An 8-core Xavier might actually have a speed not much different than a 16 core Cortex-A72, even if the NVIDIA cores are relatively slow when compared with more recent ARM cores, i.e. Carmel is a little slower than Cortex-A75. Buying a Honeycomb lx2 makes sense only when your interest is in the multiple 10G Ethernet and you do not care about what kind of CPU is provided on the board, i.e. you are not interested in the board as a software development platform for future ARM devices.
  - microtonal 5y ago
    Sadly the vast majority of ARM vendors offer only very old ARM cores. Except for the chips intended for mobile phones, the only devices with not too old ARM cores are slow devices with quadruple Cortex-A55 cores. I fear that Asahi Linux on a Mac Mini (especially when the M1X comes out with possibly 6 or 8 high-performance cores) is going to be the best option for a modern Linux ARM workstation for a while.
    - Teknoman117 5y ago
      The largest hurdles in these sorts of projects tend to the GPU and video codecs. I wish them luck and hope they succeed in entirety, but I can only early adopt so many things. Hopefully Apple doesn't reinvent their GPU every single year.
- JohnJamesRambo 5y ago
  Can anyone provide insight into who buys that sort of device? It seems kind of pricey for what you get.
  - smackeyacky 5y ago
    I'm keen on an ARM workstation. I do all my dev work on Linux and deploy everything to Linux hosts. AWS offers ARM hosts at half the cost of amd64 ones, you can cross compile but I would be happier if my build box was the same processor. What might work is some kind of local headless cloud machine running a pile of VMs until somebody builds something I can hang two big monitors off. Laptops are too constraining.
    - exikyut 5y ago
      I just thought for a moment it could be interesting to try and convince Amazon to do for their ARM kit what Google have done with their TPU chipset (https://cloud.google.com/edge-tpu https://cloud.google.com/edge-tpu, https://google.com/search?q=tpu+board https://google.com/search?q=tpu+board). Then I realized that - If such an initiative succeeded, the developer side of the fence would lose ground somewhere, because Amazon. - It would never actually happen in the first place, because ARM is so fast-moving now that everyone's doing everything they can to maintain their first-mover advantage, which of course includes being as secretive as possible. - In the ideal case where an engineer actually considered the situation, it would be all too easy (given the iteration that's happening right now) for them to counter-argue that it makes the most sense to just buy the Altera kit available on the retail market because it's technically the closest to what Amazon are using internally. The evolutionary focus is entirely on servers because of the appreciable overheads associated with that market that can be tapped into to drive the R&D process - especially given the chip shortages at the moment. Inn the same vein, there are not likely to be any consumer-class implementations (or new products) until the supply chain has had a chance to figure itself out, which is probably not going to happen for a while. :(
      - smackeyacky 5y ago
        This is a shame for a lot of reasons. Chiefly that I would really like a graviton2 local server to get as close to replicating the metal my server will run on. Perhaps Amazon could just ship a PCI card with a system on it or something like that. Its interesting to me that all of a sudden amd64 seems like a legacy system. I can hear the fan in my old proliant dev server from here and while I am somewhat sentimental about it, its definitely time to move on, preferably withou Apple involved. Maybe its time to start a 1980s style workstation company again.
Uberphallus 5y ago
My thoughts exactly. Can be also useful for certain high volume workloads as you pass through data once and cache isn't all that useful. But otherwise it is severely deprived of cache for anything in the middle.
rowanG077 5y ago
well soon you can get a mac mini. It's doubtful you could get a better bang for your buck elsewhere.
TradingPlaces 5y ago
I think the upcoming Qualcomm-Nuvia chip will be what you want
volta83 5y ago
You can get an Ampere Altra Q32-17 with 32 cores at 45W TDP.
dragontamer 5y ago
> That was about as expected. Going to 128 cores for cache-resident compute workloads helps, it hurts for anything else. It looks like it has really good SPECjbb? I assume Java is non-cache resident? (Especially if SPECjbb is designed to emulate server-scale tasks). If anything, the SPECjbb results come full circle IMO: the large caches of Intel/AMD no longer help, and you're back to latency bound (Maybe??). Its very curious to me that this 16MB / 128core part is out performing the 32MB / 80core config on so many benchmarks. I think you're right in that this processor has very little cache (and no hyperthreading). But it seems to just get around that issue by having a crapton of cores. If things are waiting on memory, it just brute-forces more cores to do more things. I can't help but think that SMT2 or SMT4 would really help here. I have to imagine that most of those cores are waiting on RAM, given this architecture. Or maybe they ran the math and decided that 128 threads was enough to saturate the memory channels already? I dunno, memory latency is always a killer, especially since this architecture has so little cache. My instincts suggest that SMT would still help.
- cogman10 5y ago
  IDK what SPECjbb does, but it really sort of depends on a lot of factors around how the JVM was setup. The JVM CAN be fairly cache friendly in the case where allocations are used relatively close to one another. The JVM will (typically) keep memory allocated at the same time in roughly the same memory region.
  - dragontamer 5y ago
    https://www.spec.org/jbb2015/docs/designdocument.pdf https://www.spec.org/jbb2015/docs/designdocument.pdf SPECjbb is a large, Java benchmark. The idea is to create a "realistic enterprise system", and then measure the number of requests before 100ms latency is violated (aka: critical-jOPs), as well as maximum throughput (aka: max-jOPs). The general model is on a hypothetical Supermarket: each time a POS system scans things, the Headquarters has to double-check the status of coupons, report back to the POS system and determine if coupons are still valid. These hypothetical messages are GZipped XML that's getting passed back and forth between distinct JVMs, across the Grizzly-based non-blocking HTTPS server (default). The benchmark artificially shoves more-and-more requests through the system until the 100ms response time is violated (ex: the system is beginning to show signs of being overburdened). Then it continues to shove more data into the system until the maximum throughput is measured. It involves many JVMs doing many different tasks and communicating over Java-APIs. Thread pools, IPC, sockets, XML, HTTPS, encryption, the whole shebang.
LeifCarrotson 5y ago
I'm curious what workloads are suitable. You do have eight channels of 25 GB/s DDR4-3200 bandwidth (theoretically 200 GB/s) and 128 lanes of 2 GB/s PCIe 4.0(theoretically another 256 GB/s) to feed the thing, but getting it to make efficient use of those peripherals is another question altogether.

MayeulC 5y ago

As we are witnessing multiple relatively small companies entering the server CPU market, it would be a really great thing if they could standardize on a compatible socket. This way, instead of designing a motherboard for a single CPU vendor in a small market, integrators could target all of them at once. Intel reportedly keeps changing sockets to please OEMs who want to sell motherboards; I would be happy to see something change in that landscape. Otherwise, I'm extremely pleased to see more options are starting to become viable (again).

cameron_b 5y ago
If you abstract that ask just a bit there is already an answer, although it's more of an industrial answer. the COM-Express Spec is something of this Universal socket [0] And while all of that link above looks like a creamy beige box cringefest, the COM-HPC[1] interface is being used for some heavy-hitting systems including Xeon and Ampere Altra [2] So, yea, you can get this in something more interoperable. 0 - https://www.picmg.org/openstandards/com-express/ https://www.picmg.org/openstandards/com-express/ 1 - https://www.picmg.org/openstandards/com-hpc/ https://www.picmg.org/openstandards/com-hpc/ 2 - https://www.ipi.wiki/pages/com-hpc-altra https://www.ipi.wiki/pages/com-hpc-altra
- dragontamer 5y ago
  > If you abstract that ask just a bit there is already an answer, although it's more of an industrial answer. the COM-Express Spec is something of this Universal socket [0] So are PCIe slots, except GPUs seem to have won on that front (RIP Xeon Phi). Moving forward: PCIe 5.0 is having more-and-more atomic instructions execute over the PCIe 5.0 specification. Its looking like PCIe 5.0 will have cache-cohesive, in particular, the Compute Express Link (CXL). So PCIe is looking more-and-more like a "standardized socket", rather than an I/O mechanism.

jeffbee 5y ago

Does that LLVM comparison make sense? It seems to target the native platform, which doesn't seem like it would make for a valid comparison between x86 and arm.

tyingq 5y ago
Maybe for the relative 1 socket / 2 socket differences.