6 ms·

Longer term this will also likely accelerate servers running on ARM. Writing software on ARM laptops that are deployed to production servers running on x86 serv

Longer term this will also likely accelerate servers running on ARM. Writing software on ARM laptops that are deployed to production servers running on x86 servers will start to cause a host of new challenges. The switch to running ARM in production will have many advantages for developers and will likely be very attractive to cloud providers (AWS, Azure) as the costs of electricity for these servers may be significantly less.

krrrh 6y ago
The transition to ARM at AWS appears to be well underway. https://aws.amazon.com/ec2/graviton/ https://aws.amazon.com/ec2/graviton/
jeffbee 6y ago
If there was an actual energy efficiency advantage -- i.e. less power consumed for the same amount of work -- Google would already be 100% on ARM. Why do you think they would leave that on the table? I realize the situation changes every time a new CPU comes out, but I have never personally seen a real workload where ARM won on energy efficiency and had reasonable performance. Tests like [1] and [2] showing x86 having an orders-of-magnitude lead on database performance vs. AWS Gravitron2 should give you serious pause. 1: https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON200&sha=c954e23&p=2 https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON... 2: https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON200&sha=0c9b11b&p=2 https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON... If you're wondering why ARM needs to have both competitive performance and energy efficiency, see Urs Holzle's comments on wimpy vs. brawny[3]. 3: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36448.pdf https://static.googleusercontent.com/media/research.google.c...
- nojito 6y ago
  >Google would already be 100% on ARM Google is laggard when it comes to server tech. Amazon and others have leapfrogged them.
  - throwaway1777 6y ago
    Citation needed
  - jeffbee 6y ago
    That's pretty silly. What makes you say this? There are large groups at Google responsible for buying every computer there is and evaluating the TCO thereof. With operating expenses exceeding two billion dollars per week, they have a larger incentive to optimize their first-party power efficiency than anyone else in the business. I'm fairly certain their first-party workloads (search etc) are the largest workloads in the world.
    - baybal2 6y ago
      I don't think they would switch even if they could in the near future. The poster above says that Amazons "leapfrogs." They question is "leapfrogs where?" The fact that ARM cores cost 100+ times less than Intel, and are n times more power efficient was well known for the whole eternity. What people don't get is that you get the whole platform on x86, and ARM is a very, very DIY thing, even if you are a multinational with lots of money on RnD.
      - jeffbee 6y ago
        > I don't think they would switch even if they could in the near future. They've already brandished their ability to port their whole product from x86 to POWER[1], and deploy POWER at scale if they need to[2]. My personal interpretation of these announcements is they are made with the purpose of keeping their Intel sales representatives in order, but the fact that you don't also see them or anyone else brandishing their AArch64 port should tell you something. 1: https://www.cnet.com/news/google-acquires-a-taste-for-ibms-power8-processors/ https://www.cnet.com/news/google-acquires-a-taste-for-ibms-p... 2: https://www.forbes.com/sites/patrickmoorhead/2018/03/19/headed-into-its-fifth-year-openpower-has-momentum-into-the-power9-generation/#5c2a7e0978a8 https://www.forbes.com/sites/patrickmoorhead/2018/03/19/head...
    - scarface74 6y ago
- my123 6y ago
  (disclaimer: I work at AWS) Hello, There definitely is that power use advantage. Those tests are pathological cases because of bugs, and aren't representative of the performance of the hardware. (for that one I suspect that ARMv8.1-A atomics weren't used in compiler options...)
  - jeffbee 6y ago
    Well, the listed GCC options do not specify microarchitecture for either ARM or x86. So it's probably a k8-compatible binary on the top line, too. I'm also not sure why atomics would be important in a single-client mysql benchmark. Either way, the risk that your toolchain doesn't automatically do the right thing is one of the costs that keeps people from jumping architectures.
    - my123 6y ago
      GCC 10 onwards are finally able to get and use Arm atomics at runtime on arm64. > I'm also not sure why atomics would be important in a single-client mysql benchmark They are because exclusive load & store operations are quite expensive on Arm, even if you actually just run an ST workload.
      - jeffbee 6y ago
        Let me ask you then, since you seem plugged in: if the win is there, why can't I buy RDS on Gravitron?
        datasage 6y ago
        I presume this has to do with support for ARM with the database software. PostgreSQL appears to have had ARM support for sometime. Mysql only added it with 8.0. As far as the other database options are concerned, ARM isn't even supported yet.
        imtringued 6y ago
        Ok, so why would I use an architecture that barely has any software support?
- baybal2 6y ago
  > If there was an actual energy efficiency advantage -- i.e. less power consumed for the same amount of work -- Google would already be 100% on ARM. I don't think they would do that. It is not wise to venture into microelectronics business, let alone CPUs.
- bob1029 6y ago
  I found this to be an excellent explanation for what is going on in these x86 CPUs that makes them so powerful: https://youtu.be/Nb2tebYAaOA https://youtu.be/Nb2tebYAaOA (Starting from ~6:30)
  - baybal2 6y ago
    The interviewer guy is weird
  - imtringued 6y ago
    Add timestamps to your link instead of messing around with parenthesis. https://youtu.be/Nb2tebYAaOA?t=390 https://youtu.be/Nb2tebYAaOA?t=390 Right click -> "Copy video URL at current time"
- ksec 6y ago
  >If there was an actual energy efficiency advantage -- i.e. less power consumed for the same amount of work -- Google would already be 100% on ARM. Energy Efficiency Wins on Server Workload is a very recent thing. To the point it is new as it basically start with N1 / AWS Graviton2. And even that is not ALL workload. Not all software are optimised on ARM, compared to decades of optimisation on x86. Not to mention Compiler options and EPYC was running on bare Metal with NVMe SSD compared to Graviton 2 running on Amazon EBS. And most importantly, you would be paying for 64 Thread on Amazon for the price of 64 Core Graviton 2. i.e It should be tested against a 32 Core EPYC 2 with SMT. I still doubt G2 would win in the fair test, but it would be close, and it would be cheaper. And that is the point.
- klelatti 6y ago
  But up until recently Intel have had a manufacturing process advantage that has made it difficult / impossible for the likes of Google to source competitive high performance cores. That advantage has slipped away. EPYC is an innovative product and comparing it with Graviton clearly shows that AMD has done a great job and that Graviton is not quite fully competitive yet (but not to the extent that those benchmarks seem to indicate, as others have commented). I think its possible to overstate the energy efficiency gains from using ARM but all the indications are that ARM cores with fully competitive performance will emerge and that they will have some efficiency advantage - after all why would AWS be investing in ARM if not?
deleted 6y ago
[deleted]
ohazi 6y ago
> Writing software on ARM laptops that are deployed to production servers running on x86 servers will start to cause a host of new challenges. We already know what this is like, and the challenges are usually in the other direction (write/test on x86, attempt to deploy on ARM). This is due to things like word alignment and memory ordering requirements. For the most part, the "extra complexity" in x86 allows you to ignore a lot of the stuff you need to pay more attention to on ARM.
- scarface74 6y ago
  Millions of iOS developers develop on x86 and deploy to ARM devices every day. The iOS simulator compiles apps to x86 and they run on top of an x86 version of the iOS frameworks.
  - ohazi 6y ago
    The "x86 version of the iOS frameworks" is where those differences would be apparent. Apple may have done a good job with their simulator, but that's just it -- Apple did this work so that their iOS developers wouldn't have to. As an iOS developer, you don't need to think about memory barriers if you don't want to, but Apple's simulator and frameworks absolutely do. That's the kind of work that awaits systems programmers who want to port high performance x86 server software to ARM. x86 hardware is physically more permissive than ARM. Accesses that must be aligned on ARM can be left unaligned on x86, and they'll still work correctly. But x86 isn't going to explode either if you do happen align them. ARM is weakly ordered, x86 is strongly ordered, so you generally need more fences on ARM than you do on x86, but leaving these fences in place doesn't break things on x86. My point was that code that works correctly on ARM usually works correctly without modification on x86. The opposite is generally not the case, even if the iOS simulator hides this for you.
    - vbezhenar 6y ago
      People use Java and Go to write server software. They'll make things aligned properly, so it won't be a problem. The real problem is memory concurrency model. You will hit subtle concurrency bugs with ARM which were not manifested on x86. And those bugs will be present in any language with threads and shared data. They are hard to debug and frustrating, because they'll lead to rare deadlocks, and resource leaks.
hocuspocus 6y ago
Now the question is, who's going to make those chips? While the big cloud providers are certainly able to internalize design of their hardware, you still need to ship millions of servers to smaller players.
- jcheng 6y ago
  Ampere Altra and Marvell ThunderX are targeting this market. Qualcomm, Broadcom, and Nvidia tried earlier and gave up, but I wonder if we will see them enter back in.
formerly_proven 6y ago
Ultimately moving data is the prime power consumer in a data center, so unless you are somehow drastically reducing the amount of data movement, you are not going to get drastic energy savings. This remains true even inside systems and CPUs, fast and wide buses and caches require lots of power, the main power cost in wide SIMD (AVX-512 vs. AVX2) isn't the computation itself, it's getting the data to and from the ALUs.

krrrh 6y ago

The transition to ARM at AWS appears to be well underway. https://aws.amazon.com/ec2/graviton/ https://aws.amazon.com/ec2/graviton/

jeffbee 6y ago

If there was an actual energy efficiency advantage -- i.e. less power consumed for the same amount of work -- Google would already be 100% on ARM. Why do you think they would leave that on the table? I realize the situation changes every time a new CPU comes out, but I have never personally seen a real workload where ARM won on energy efficiency and had reasonable performance. Tests like [1] and [2] showing x86 having an orders-of-magnitude lead on database performance vs. AWS Gravitron2 should give you serious pause. 1: https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON200&sha=c954e23&p=2 https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON... 2: https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON200&sha=0c9b11b&p=2 https://openbenchmarking.org/embed.php?i=2005220-NI-GRAVITON... If you're wondering why ARM needs to have both competitive performance and energy efficiency, see Urs Holzle's comments on wimpy vs. brawny[3]. 3: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36448.pdf https://static.googleusercontent.com/media/research.google.c...

nojito 6y ago
>Google would already be 100% on ARM Google is laggard when it comes to server tech. Amazon and others have leapfrogged them.
- throwaway1777 6y ago
  Citation needed
- jeffbee 6y ago
  That's pretty silly. What makes you say this? There are large groups at Google responsible for buying every computer there is and evaluating the TCO thereof. With operating expenses exceeding two billion dollars per week, they have a larger incentive to optimize their first-party power efficiency than anyone else in the business. I'm fairly certain their first-party workloads (search etc) are the largest workloads in the world.
  - baybal2 6y ago
    I don't think they would switch even if they could in the near future. The poster above says that Amazons "leapfrogs." They question is "leapfrogs where?" The fact that ARM cores cost 100+ times less than Intel, and are n times more power efficient was well known for the whole eternity. What people don't get is that you get the whole platform on x86, and ARM is a very, very DIY thing, even if you are a multinational with lots of money on RnD.
    - jeffbee 6y ago
      > I don't think they would switch even if they could in the near future. They've already brandished their ability to port their whole product from x86 to POWER[1], and deploy POWER at scale if they need to[2]. My personal interpretation of these announcements is they are made with the purpose of keeping their Intel sales representatives in order, but the fact that you don't also see them or anyone else brandishing their AArch64 port should tell you something. 1: https://www.cnet.com/news/google-acquires-a-taste-for-ibms-power8-processors/ https://www.cnet.com/news/google-acquires-a-taste-for-ibms-p... 2: https://www.forbes.com/sites/patrickmoorhead/2018/03/19/headed-into-its-fifth-year-openpower-has-momentum-into-the-power9-generation/#5c2a7e0978a8 https://www.forbes.com/sites/patrickmoorhead/2018/03/19/head...
  - scarface74 6y ago
my123 6y ago
(disclaimer: I work at AWS) Hello, There definitely is that power use advantage. Those tests are pathological cases because of bugs, and aren't representative of the performance of the hardware. (for that one I suspect that ARMv8.1-A atomics weren't used in compiler options...)
- jeffbee 6y ago
  Well, the listed GCC options do not specify microarchitecture for either ARM or x86. So it's probably a k8-compatible binary on the top line, too. I'm also not sure why atomics would be important in a single-client mysql benchmark. Either way, the risk that your toolchain doesn't automatically do the right thing is one of the costs that keeps people from jumping architectures.
  - my123 6y ago
    GCC 10 onwards are finally able to get and use Arm atomics at runtime on arm64. > I'm also not sure why atomics would be important in a single-client mysql benchmark They are because exclusive load & store operations are quite expensive on Arm, even if you actually just run an ST workload.
    - jeffbee 6y ago
      Let me ask you then, since you seem plugged in: if the win is there, why can't I buy RDS on Gravitron?
      - datasage 6y ago
        I presume this has to do with support for ARM with the database software. PostgreSQL appears to have had ARM support for sometime. Mysql only added it with 8.0. As far as the other database options are concerned, ARM isn't even supported yet.
        imtringued 6y ago
        Ok, so why would I use an architecture that barely has any software support?
baybal2 6y ago
> If there was an actual energy efficiency advantage -- i.e. less power consumed for the same amount of work -- Google would already be 100% on ARM. I don't think they would do that. It is not wise to venture into microelectronics business, let alone CPUs.
bob1029 6y ago
I found this to be an excellent explanation for what is going on in these x86 CPUs that makes them so powerful: https://youtu.be/Nb2tebYAaOA https://youtu.be/Nb2tebYAaOA (Starting from ~6:30)
- baybal2 6y ago
  The interviewer guy is weird
- imtringued 6y ago
  Add timestamps to your link instead of messing around with parenthesis. https://youtu.be/Nb2tebYAaOA?t=390 https://youtu.be/Nb2tebYAaOA?t=390 Right click -> "Copy video URL at current time"
ksec 6y ago
>If there was an actual energy efficiency advantage -- i.e. less power consumed for the same amount of work -- Google would already be 100% on ARM. Energy Efficiency Wins on Server Workload is a very recent thing. To the point it is new as it basically start with N1 / AWS Graviton2. And even that is not ALL workload. Not all software are optimised on ARM, compared to decades of optimisation on x86. Not to mention Compiler options and EPYC was running on bare Metal with NVMe SSD compared to Graviton 2 running on Amazon EBS. And most importantly, you would be paying for 64 Thread on Amazon for the price of 64 Core Graviton 2. i.e It should be tested against a 32 Core EPYC 2 with SMT. I still doubt G2 would win in the fair test, but it would be close, and it would be cheaper. And that is the point.
klelatti 6y ago
But up until recently Intel have had a manufacturing process advantage that has made it difficult / impossible for the likes of Google to source competitive high performance cores. That advantage has slipped away. EPYC is an innovative product and comparing it with Graviton clearly shows that AMD has done a great job and that Graviton is not quite fully competitive yet (but not to the extent that those benchmarks seem to indicate, as others have commented). I think its possible to overstate the energy efficiency gains from using ARM but all the indications are that ARM cores with fully competitive performance will emerge and that they will have some efficiency advantage - after all why would AWS be investing in ARM if not?

deleted 6y ago

[deleted]

ohazi 6y ago

> Writing software on ARM laptops that are deployed to production servers running on x86 servers will start to cause a host of new challenges. We already know what this is like, and the challenges are usually in the other direction (write/test on x86, attempt to deploy on ARM). This is due to things like word alignment and memory ordering requirements. For the most part, the "extra complexity" in x86 allows you to ignore a lot of the stuff you need to pay more attention to on ARM.

scarface74 6y ago
Millions of iOS developers develop on x86 and deploy to ARM devices every day. The iOS simulator compiles apps to x86 and they run on top of an x86 version of the iOS frameworks.
- ohazi 6y ago
  The "x86 version of the iOS frameworks" is where those differences would be apparent. Apple may have done a good job with their simulator, but that's just it -- Apple did this work so that their iOS developers wouldn't have to. As an iOS developer, you don't need to think about memory barriers if you don't want to, but Apple's simulator and frameworks absolutely do. That's the kind of work that awaits systems programmers who want to port high performance x86 server software to ARM. x86 hardware is physically more permissive than ARM. Accesses that must be aligned on ARM can be left unaligned on x86, and they'll still work correctly. But x86 isn't going to explode either if you do happen align them. ARM is weakly ordered, x86 is strongly ordered, so you generally need more fences on ARM than you do on x86, but leaving these fences in place doesn't break things on x86. My point was that code that works correctly on ARM usually works correctly without modification on x86. The opposite is generally not the case, even if the iOS simulator hides this for you.
  - vbezhenar 6y ago
    People use Java and Go to write server software. They'll make things aligned properly, so it won't be a problem. The real problem is memory concurrency model. You will hit subtle concurrency bugs with ARM which were not manifested on x86. And those bugs will be present in any language with threads and shared data. They are hard to debug and frustrating, because they'll lead to rare deadlocks, and resource leaks.

hocuspocus 6y ago

Now the question is, who's going to make those chips? While the big cloud providers are certainly able to internalize design of their hardware, you still need to ship millions of servers to smaller players.

jcheng 6y ago
Ampere Altra and Marvell ThunderX are targeting this market. Qualcomm, Broadcom, and Nvidia tried earlier and gave up, but I wonder if we will see them enter back in.

formerly_proven 6y ago

Ultimately moving data is the prime power consumer in a data center, so unless you are somehow drastically reducing the amount of data movement, you are not going to get drastic energy savings. This remains true even inside systems and CPUs, fast and wide buses and caches require lots of power, the main power cost in wide SIMD (AVX-512 vs. AVX2) isn't the computation itself, it's getting the data to and from the ALUs.