3 ms·

What I really want is an ARM server that can accelerate our JVM workloads as much as M1/M2 does, without regards to power density. Why? In colo you’re usually

What I really want is an ARM server that can accelerate our JVM workloads as much as M1/M2 does, without regards to power density. Why? In colo you’re usually not as concerned about density. We’re renting a chunk of rack space for a bottom basement price of $200/month and we can easily stick a router and compute / storage nodes with 64 TB of disk and boat loads of memory in that rented space; plenty for our needs without coming anywhere close to the power allocation. When we were purchasing a few servers for our colo, we ultimately just went with AMD because the price per core was there. It ends up being less performant than an m2, maybe m1 ish. Ultimately we’re a small market though. The ‘sexy’ thing to do is waive your AWS invoice size around at other hipsters during cocktail parties; not to own your own hosting even though you’ve paid for it serval times over. I don’t see this changing anytime soon.

yolo3000 4y ago
How does M1/M2 accelerate JVM apps, and in general how do you optimise your JVM apps to make full use of the resources?
- crop_rotation 4y ago
  I doubt M1/M2 do anything special for the JVM, it's just that they are overall better performing for the same power. In a server setting though I don't see them having the same advantages.
- pshirshov 4y ago
  It has lot higher memory throughput, so mid-range M1 chip produces results comparable to my 3970x Threadripper on some of my workloads (scala compilation) while it consumes just a fraction of the power the Threadripper requires.
lockhouse 4y ago
With Asahi Linux, I could actually see people shoving M2 Mac Minis in a Server rack in the future.
- tambre 4y ago
  Does it have ECC?
  - jlokier 4y ago
    No. See https://news.ycombinator.com/item?id=26058061 https://news.ycombinator.com/item?id=26058061 about the M1. If ECC were added between M1 and M2, Apple would almost certainly have announced it as a feature.
cjbgkagh 4y ago
I’d love to see a cluster of M2 SOCs placed together on a single PCB with built in networking. Maybe direct PCIe to PCIe communication between the SOCs in a hypercube topology. Having multiple CPUs spread out around the box will help with cooling which is my limiting factor for power density. My tasks are data parallel and CPU bound with low ram and disk requirements. I could probably design such a PCB myself but the cost of doing so will unlikely be paid back unless my compute needs go up by another order of magnitude. I am many years away from that being viable. For now I’m sticking to AMD chips. Unless someone else wants this as well for which I could amortize the costs.
- moondev 4y ago
  It would seem this devkit is even more ideal for you? 80 cores and buy as little memory as you want. Not to mention it does not need clustering and runs any aarch64 distro of your choosing
  - cjbgkagh 4y ago
    I checked out the benchmarks and it seems that the 80 core version would be ~6.5x M2 and would indeed be cheaper on the assumption that the only way to get M2 chips is to first buy a Mac. But if (an I know it isn’t going to happen) M2 was at the Jetson Nano price of $150 then it starts to become attractive again. It’s 2.3x faster than Ryzen 7900X, I’m not sure if the dev kit has a price performance advantage over that and I’ve seen the Ampere Altra chips on their own for much more so I’m thinking the dev kit has a discount which probably comes with a purchase limit.
    - joespeed 4y ago
      The dev kit has no purchase limit. Same is true for the dev platform workstation 32, 64, 80, 128 core (and a 96 faster cores option if you ask nicely). https://www.tomshardware.com/news/ampere-64-core-arm-workstation-runs-windows https://www.tomshardware.com/news/ampere-64-core-arm-worksta... For those keeping score the Ampere Altra 128 core 3GHz Supermicro telco edge server being used in 5G base stations benchmarks equal to 100 RPi4 and 22% more energy efficient. https://youtu.be/UT5UbSJOyog https://youtu.be/UT5UbSJOyog
smoldesu 4y ago
Have you tried Ampere? It's pretty good as far as ARM servers go, and even has acceleration for hip stuff like Tensorflow and Pytorch. I personally haven't experimented with Java apps on it, but it should have a high enough IPC to compete with other similarly dense ARM options. And of course, Nvidia has their 72x2 core Grace chip coming soon. It will be very interesting to see how that stacks up against the ARM incumbents.
- exabrial 4y ago
  Actually pretty excited to NVidia offering, but we spent our budget :(
subarctic 4y ago
i could see it changing a bit if oxide.computer is successful in shipping lots of units, since they've already got major cool factor with their podcasts. I'm curious about these cocktail parties you speak of - over here, in-person tech meetups have been pretty rare since covid
- kungito 4y ago
  Even post covid?
cameron_b 4y ago
I'm a proponent of ARM, but out of curiosity, why would Oracle's hardware not be your first choice in accelerating JVM workloads?
- pshirshov 4y ago
  Probably because of the memory throughput?
  - exabrial 4y ago
    I think that is what’s we’re seeing.
- oneplane 4y ago
  Because it's not as performant per watt as an M2 (and in most cases not even an M1), and it also isn't as performance per rack unit. In a full-depth 1u tray you can fit a crapload of M1 Mac minis which perform better than anything oracle can stick in that 1U. There is a difference in the type of workload and the ultimate total size (i.e. if you have a single 2-rack workload then the M1 and M2 aren't an option at all -- those only make sense in parallel instantiated workloads and container-based loads). After experimenting with a shipment of M1's (before they had to be commissioned for an office that was still in the process of being built and furnished) I'm pretty confident that similar JVM performance isn't currently available on anything else if your max allocation per instance is 8-core or lower. As soon as your cores-per-workload is higher the cost/density swings back in favour of Intel again because the Ultra in the studio doesn't fit in 1U and energy/cooling COP becomes on par with classic servers again. Maybe on M2 that's not the case, but when I ran the experiment the M1 was all that was available. Maybe once the FPGAs for Intel get public bitstreams it might be as efficient on Intel again, but right now, it's not (and also not on AMD), and thus also not on anything Oracle. Note: didn't test Ampere, but would love it.
- deleted 4y ago
  [deleted]
- exabrial 4y ago
  We also have ‘normal’ workloads like databases. Not confident in the ports Oracle would provide

yolo3000 4y ago

How does M1/M2 accelerate JVM apps, and in general how do you optimise your JVM apps to make full use of the resources?

crop_rotation 4y ago
I doubt M1/M2 do anything special for the JVM, it's just that they are overall better performing for the same power. In a server setting though I don't see them having the same advantages.
pshirshov 4y ago
It has lot higher memory throughput, so mid-range M1 chip produces results comparable to my 3970x Threadripper on some of my workloads (scala compilation) while it consumes just a fraction of the power the Threadripper requires.

lockhouse 4y ago

With Asahi Linux, I could actually see people shoving M2 Mac Minis in a Server rack in the future.

tambre 4y ago
Does it have ECC?
- jlokier 4y ago
  No. See https://news.ycombinator.com/item?id=26058061 https://news.ycombinator.com/item?id=26058061 about the M1. If ECC were added between M1 and M2, Apple would almost certainly have announced it as a feature.

cjbgkagh 4y ago

I’d love to see a cluster of M2 SOCs placed together on a single PCB with built in networking. Maybe direct PCIe to PCIe communication between the SOCs in a hypercube topology. Having multiple CPUs spread out around the box will help with cooling which is my limiting factor for power density. My tasks are data parallel and CPU bound with low ram and disk requirements. I could probably design such a PCB myself but the cost of doing so will unlikely be paid back unless my compute needs go up by another order of magnitude. I am many years away from that being viable. For now I’m sticking to AMD chips. Unless someone else wants this as well for which I could amortize the costs.

moondev 4y ago
It would seem this devkit is even more ideal for you? 80 cores and buy as little memory as you want. Not to mention it does not need clustering and runs any aarch64 distro of your choosing
- cjbgkagh 4y ago
  I checked out the benchmarks and it seems that the 80 core version would be ~6.5x M2 and would indeed be cheaper on the assumption that the only way to get M2 chips is to first buy a Mac. But if (an I know it isn’t going to happen) M2 was at the Jetson Nano price of $150 then it starts to become attractive again. It’s 2.3x faster than Ryzen 7900X, I’m not sure if the dev kit has a price performance advantage over that and I’ve seen the Ampere Altra chips on their own for much more so I’m thinking the dev kit has a discount which probably comes with a purchase limit.
  - joespeed 4y ago
    The dev kit has no purchase limit. Same is true for the dev platform workstation 32, 64, 80, 128 core (and a 96 faster cores option if you ask nicely). https://www.tomshardware.com/news/ampere-64-core-arm-workstation-runs-windows https://www.tomshardware.com/news/ampere-64-core-arm-worksta... For those keeping score the Ampere Altra 128 core 3GHz Supermicro telco edge server being used in 5G base stations benchmarks equal to 100 RPi4 and 22% more energy efficient. https://youtu.be/UT5UbSJOyog https://youtu.be/UT5UbSJOyog

smoldesu 4y ago

Have you tried Ampere? It's pretty good as far as ARM servers go, and even has acceleration for hip stuff like Tensorflow and Pytorch. I personally haven't experimented with Java apps on it, but it should have a high enough IPC to compete with other similarly dense ARM options. And of course, Nvidia has their 72x2 core Grace chip coming soon. It will be very interesting to see how that stacks up against the ARM incumbents.

exabrial 4y ago
Actually pretty excited to NVidia offering, but we spent our budget :(

subarctic 4y ago

i could see it changing a bit if oxide.computer is successful in shipping lots of units, since they've already got major cool factor with their podcasts. I'm curious about these cocktail parties you speak of - over here, in-person tech meetups have been pretty rare since covid

kungito 4y ago
Even post covid?

cameron_b 4y ago

I'm a proponent of ARM, but out of curiosity, why would Oracle's hardware not be your first choice in accelerating JVM workloads?

pshirshov 4y ago
Probably because of the memory throughput?
- exabrial 4y ago
  I think that is what’s we’re seeing.
oneplane 4y ago
Because it's not as performant per watt as an M2 (and in most cases not even an M1), and it also isn't as performance per rack unit. In a full-depth 1u tray you can fit a crapload of M1 Mac minis which perform better than anything oracle can stick in that 1U. There is a difference in the type of workload and the ultimate total size (i.e. if you have a single 2-rack workload then the M1 and M2 aren't an option at all -- those only make sense in parallel instantiated workloads and container-based loads). After experimenting with a shipment of M1's (before they had to be commissioned for an office that was still in the process of being built and furnished) I'm pretty confident that similar JVM performance isn't currently available on anything else if your max allocation per instance is 8-core or lower. As soon as your cores-per-workload is higher the cost/density swings back in favour of Intel again because the Ultra in the studio doesn't fit in 1U and energy/cooling COP becomes on par with classic servers again. Maybe on M2 that's not the case, but when I ran the experiment the M1 was all that was available. Maybe once the FPGAs for Intel get public bitstreams it might be as efficient on Intel again, but right now, it's not (and also not on AMD), and thus also not on anything Oracle. Note: didn't test Ampere, but would love it.
deleted 4y ago
[deleted]
exabrial 4y ago
We also have ‘normal’ workloads like databases. Not confident in the ports Oracle would provide