18 ms·

AWS Graviton2

188 points by yarapavan 7y ago

yarapavan 7y ago
* [James Hamilton] believe there is a high probability we are now looking at what will become the first high volume ARM Server. More speeds and feeds: >30B transistors in 7nm process 64KB icache, 64KB dcache, and 1MB L2 cache 2TB/s internal, full-mesh fabric Each vCPU is a full non-shared core (not SMT) Dual SIMD pipelines/core including ML optimized int8 and fp16 Fully cache coherent L1 cache 100% encrypted DRAM 8 DRAM channels at 3200 Mhz * ARM Servers have been inevitable for a long time but it’s great to finally see them here and in customers hands in large numbers.
- kortilla 7y ago
  Is there somewhere to buy something like this for use at home?
  - downrightmike 7y ago
    Probably not until end of life for these chips
  - wmf 7y ago
    Ampere QuickSilver, but don't expect an 80-core server to be cheap.
- nine_k 7y ago
  What really stands out for me is "100% encrypted DRAM". How efficient is this? Can different cores have different encryption keys, so that different VMs under a hypervisor can't benefit from breaking the hypervisor's protections?
  - praseodym 7y ago
    I can’t speak for the Graviton2 CPUs, but AMD Epyc CPUs have RAM encryption with per-VM keys for increased isolation: https://developer.amd.com/sev/ https://developer.amd.com/sev/
  - wwarner 7y ago
    Same question from me. At what point does dedicated hosting become more efficient than encrypted everything?
    - wmf 7y ago
      Pretty much never. With dedicated servers you don't have anyone to split the Nitro overhead with.
  - amluto 7y ago
    Some Intel chips can encrypt memory with different keys for different VMs. This sounds great for marketing but adds basically no security whatsoever. The feature is called MKTME. What’s going on here is that “different keys for different VMs” does not actually improve isolation without a considerable amount of hardware or microcode enforcement. AMD has this type of tracking of which VM is which. Intel does not. I don’t know what AMD does. In any case, exception makes little difference. Cores aren’t bound 1:1 to VMs, so the core can access any VM’s data if it wants. And actually clearing the key on a context switch would require flushing caches and require that there is no cache shared between cores. The performance hit would be extreme.
    - nine_k 7y ago
      What is the point of sharing cache between VMs? Pinning a VM to a set of cores when encryption is enabled would make sense, and could be a feature cloud users would be willing to pay for.
      - amluto 7y ago
        Basically every recent CPU has a big cache shared between all cores. So, unless you pin a VM to one socket and you do something to encrypt cache coherency traffic between sockets per VM, you lose. The underlying issue here is that encryption is fast but not fast enough. So no one encrypts cache — instead, plaintext is cached and data is encrypted on its way to DRAM. So the actual isolation is in the access controls that the CPUs apply to which process or VM can access which pages, and this has little to do with encryption. It’s worth noting that Intel has been very bad lately at protecting cache contents from side channels, while AMD has done just fine. You can turn fancy encryption on, but those side channels leak plaintext.
    - thu2111 7y ago
      In fairness to Intel they also have SGX which has encrypted RAM and also a lot of isolation logic, as well as working RA, recovery, versioned sealed data and a lot of other things that AMD's equivalent just doesn't do well or at all.
  - Koshkin 7y ago
    That seems like another blow to high-performance computing on VMs.
otterley 7y ago
(I work for AWS. Opinions are my own and not necessarily those of my employer.) I've been doing some initial M6g tests in my lab, and while I'm not able to disclose benchmarks, I can say that my real-world experience so far reflects what's been claimed elsewhere. Graviton2 is going to be a game changer. It's not like the usual experience with ARM where you have to trade off performance for price, and decide whether migrating is worth the recompilation effort. In my lab, performance of the workloads I've tried so far is uniformly better than on the equivalent M5 configuration running on the Intel processor. You're not sacrificing anything by running on Graviton2. If your workloads are based on scripting languages, Java, or Go, or you can recompile your C/C++ code, you're going to want to use these instances if you can. The pricing is going to make it irresistible. Basically, unless you're running COTS (commercial off-the-shelf software), it's a no-brainer.
- rbranson 7y ago
  I am extremely excited about this development. Less so getting everything that assumes x86 in the stack supporting a “parameterized” platform. (Hi Michael!)
  - otterley 7y ago
    Hi Rick! I know it's only a part of the toolchain needed to support the migration, but multi-arch Docker image support for Amazon ECR is definitely on our immediate roadmap.
  - floatboth 7y ago
    heh, I've had a very fun experience with spotinst.com — my a1 spot instance went down and that service couldn't restore it because it did not label the AMI as arm64. Reported that to them, got a couple acknowledgements but haven't heard back in a while, so presumably this is still not fixed.
- chroem- 7y ago
  Does the chip still manage to have superior power efficiency versus x86 even at these performance levels?
gok 7y ago
There are some interesting implications for widely deployed processors that are literally never publicly seen because they spend their whole life in a highly locked down data center. I wonder if things like Meltdown could have ever been discovered if researchers could only poke at the chips via EC2.
- QuinnyPig 7y ago
  I believe the “metal” variants expose the processor extensions you’d need to discover Meltdown.
  - ec109685 7y ago
    That’s a good point. And AWS allows this type of security research as well freely: https://twitter.com/TeriRadichel/status/1101228943128969218 https://twitter.com/TeriRadichel/status/1101228943128969218
  - floatboth 7y ago
    Also, AWS processors use off-the-shelf Arm Cortex/Neoverse cores, and stuff like Spectre is core-level.
neonate 7y ago
https://web.archive.org/web/20200125180037/https://perspectives.mvdirona.com/2020/01/aws-graviton2/ https://web.archive.org/web/20200125180037/https://perspecti...
miohtama 7y ago
This is good news. Are Linux server distributions for ARM64s yet on-par with their PC counterparts? Getting base layer software is not going to be an issue?
- QuinnyPig 7y ago
  Been using Ubuntu on one for a few weeks; the only things I missed was a few Docker containers that weren’t built, and aws-vault didn’t have an ARM binary. I built my own, and aws-vault shopped a new release with ARM support 20 minutes after I whined about it on Twitter. Everything else has been flawless.
Dunedan 7y ago
> Here’s comparative data between M6g and M5, the previous generation instance type Instead of comparing the 7nm Graviton2 processor against an 14nm Intel processor, I'd like to see its performance compared to an AMD Epyc 2 processor, which would be a more apples-to-apples comparison as both are "7nm" parts. Unfortunately Epyc 2 processors aren't available from AWS yet (but are already announced: https://aws.amazon.com/de/blogs/aws/in-the-works-new-amd-powered-compute-optimized-ec2-instances-c5a-c5ad/ https://aws.amazon.com/de/blogs/aws/in-the-works-new-amd-pow...).
- Rapzid 7y ago
  This is what I'm curious about as epyc currently represents the state of the art x86 right? What's the TCO comparison when factoring in performance, density, and power?
- reitzensteinm 7y ago
  You're putting way too much emphasis on process tech. TSMC 7nm is roughly equivalent to Intel 10nm, so it's only a generation behind. Intel 10nm products, where available, have also not exactly been lighting the world on fire with their performance.
  - tpurves 7y ago
    No. Process tech is profoundly correlated to cpu performance. It’s literallly the definition of Moore’s law. Intel has no 10nm server processors. Intel’s near 5 year delay in getting past 14nm has opened a huge window of opportunity to competitors both AMD and amazon/arm. It is highly worth comparing these competitors apple to apple vs Intel’s oranges.
    - reitzensteinm 7y ago
      "No." yourself :) Smaller processes used to mean higher frequency switching, lower power and increased density. With the death of Dennard scaling, we mostly just get the latter. This means that the benefit is now largely economic; you get largely the same chips, you can just pack them more tightly on the wafer. If you're one node behind, you still price the chips at a price the market will bear, they just cost you a bit more to produce. And maybe not even then; mature last generation nodes perform pretty damn well against immature next generation nodes once you take yield and performance in to account. Intel's 14nm transition yielded Broadwell Xeon [1], barely any improvement over the Haswell chips. Haswell itself, however, gave us a ~50% performance boost on the same process node. This is the difference between a new process node and a new architecture in today's world. The reason Intel is in trouble is because of the 10nm fiasco, but not because of the lack of a die shrink. Their shrinks have been working like a well oiled machine for decades, and there was no contingency in place for a large delay. All post Skylake chips were being developed tightly against their 10nm libraries, with no possibility of a back port. It's not the lack of a Haswell->Broadwell analogous die shrink that's hurting Intel, but a Haswell->Broadwell->Skylake die shrink + new architecture. How do you know this is true? Because Intel switched gears and is now decoupling future architectures from die shrinks. If they did this earlier, you'd be seeing Ice Lake (or maybe Tiger Lake) on 14nm++ as an answer to Zen 2, and it would be a pretty good chip. Instead they're doing whatever minor tweaks they can to so many variations of Skylake I'm not sure I could list all the codenames from memory. Zen 2 is a seriously formidable chip, but most of the benefit came from cleaning up nasty edge cases in performance, like cross core communication being slower than a spill to DRAM. You can't disentangle the shrink from the architecture, because they happened simultaneously. [1] https://www.anandtech.com/show/10158/the-intel-xeon-e5-v4-review https://www.anandtech.com/show/10158/the-intel-xeon-e5-v4-re... [2] https://www.anandtech.com/show/8423/intel-xeon-e5-version-3-up-to-18-haswell-ep-cores- https://www.anandtech.com/show/8423/intel-xeon-e5-version-3-...
pranith 7y ago
I wonder when and how Azure and Google Cloud will compete with AWS in this market. They could buy ARM processors available in the market, but I doubt they will be able to get them as cheap as AWS who builds their own.
- justicezyx 7y ago
  I led the support for multi arch images for Borg. Google had the software stack ready for internal workload longtime ago, PowerPC was used https://www.forbes.com/sites/patrickmoorhead/2018/03/19/headed-into-its-fifth-year-openpower-has-momentum-into-the-power9-generation/ https://www.forbes.com/sites/patrickmoorhead/2018/03/19/head...
  - pranith 7y ago
    I am sure the software is multi-arch ready but I wonder if they are evaluating ARM servers either for use internally or to launch in the cloud...
- gundmc 7y ago
  Does Microsoft make any of their own silicon? It feels like Google has been directing their in-house designs on ML/TPUs while Amazon went all in on ARM. It will be interesting to see how those bets pay off.
  - floatboth 7y ago
    No, but Microsoft bought some off the shelf Ampere and Cavium/Marvell servers. But they keep them for internal use only for now :( Huawei makes their own silicon and servers with that silicon — also only internal, not available on huaweicloud :( The only other player is Scaleway who bought first gen Cavium ThunderX's way back when. And Packet of course but that's bare metal only, no cheap small VPSes.
    - pranith 7y ago
      If they are good enough for internal use, they should be good enough for public use. Not really sure what is stopping them from exposing it to the public... I am sure there is _some_ demand for it.
sdan 7y ago
This is great news. Except that a ton of software doesn't support ARM. When I was trying to shift all my current infrastructure onto a couple of RPI's, many of the Docker containers didn't support ARM (qeumu and buildx aren't reliable) and other software didn't support ARM either. Unless there's a good way to go from AMD to ARM, I'm not entirely sure how great Graviton or other competitors will get.
- JunkDNA 7y ago
  Back in the late 90’s and early 00’s, there were a ton of cpu platforms around: SGI MIPS, DEC Alpha’s, Intel, Sun SPARC, etc... while I will admit it was a colossal pain working somewhere that had all of those, it was often possible to recompile from source to get things to run. I’m not suggesting it’s trivial, but given the incredible investment in ARM in the mobile space, the wind is at least at your back today. It certainly has got to be much easier than it was in the days of being the only person in the world trying to recompile an obscure open source scientific computing package for DEC Alpha. Commercial software is a different beast, but even there, the incentive will be high to do a port if lots of people start migrating to this.
  - dehrmann 7y ago
    Windows NT 4 supported x86, Alpha, MIPS, and PowerPC. Yikes.
    - wbl 7y ago
      All have eight bit bytes with 2's complement, same availible word sizes and float formats (with some complexity on the Alpha due to VAX compat). C code will mostly not care beyond endianness. The PDP is the strange one.
- ksec 7y ago
  Assuming Amazon intend to upstream their work. All the testing of open source stack Amazon uses internally will support ARM. That is all of their Hosted Open Sources offering. This will kickstart all software support. AWS ARM offers cost advantage which proprietary software now has an incentives or their customer will request ARM support. All these will create a positive feedback loop into the ecosystem.
gautamcgoel 7y ago
This is great, but I'd be really excited if we could go out and buy the chips ourselves instead of having to pay the Amazon tax and run our code on untrusted systems in the cloud. Of course, Amazon has little incentive to sell the chips, since it gives them a competitive advantage against other cloud providers.
ijl 7y ago
What services are people using to run continuous integration for ARM? I see Travis CI has an alpha. Azure Pipelines doesn't host ARM instances I think.
- otterley 7y ago
  AWS CodeBuild supports ARM builds: https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codebuild-adds-support-for-arm--gpu--and-x-large-compute-typ/ https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codeb...
- oldmanhorton 7y ago
  Azure Pipelines doesn't host arm yet, but you can run a self hosted pool on a Gravitron2 VM pretty easily.
- jedbrown 7y ago
  GitLab-CI can use any host (and can be used with GitHub, though the PR integration is not nearly as nice as when used with a GitLab MR).
dehrmann 7y ago
Out of curiosity, what's the state of Jazelle on modern ARM? Would it help server-side ARM, or has the world (and JITs) moved on?
- pm215 7y ago
  Jazelle is dead. The v8 version (or maybe even v7; I forget) of the 32-bit architecture basically mandated that only 'trivial' Jazelle (which is the not-actually-there version) could be implemented, and 64-bit has never had anything like it. It was at best a technology of its time (when phone Java implementations were mostly interpreted, not JITs). It would be useless to a modern Java implementation.
- thu2111 7y ago
  Jazelle accelerated bytecode interpreting (a bit) but modern JVMs spend nearly all their time running compiled code, so it doesn't really help and was abandoned. There are CPU HW features that Intel doesn't have which can benefit JVM workloads but they're all pretty obscure and aren't really Java specific.
dehrmann 7y ago
Are there any security fears with virtualization on ARM (think Meltdown and Spectre)? I'd think it's been less studied than Intel's x86-64 chips.
Koshkin 7y ago
Annapurna, the goddess of job security.
xwowsersx 7y ago
> believing that massive client volumes fund the R&D stream that feeds most server-side innovation. What does he mean here?
- deleted 7y ago
  [deleted]
- wmf 7y ago
  Intel/AMD design laptop/desktop (client) cores then put those cores into server processors. Because vastly more PCs are sold, they effectively subsidize server processors. Arm has a similar advantage, designing cores for phones/tablets and repurposing/extending them for servers.
  - xwowsersx 7y ago
    Ohh I see, thank you!
dman 7y ago
When can I buy something similar for a homelab?
- ipsum2 7y ago
  You can buy Cavium Thunderx2s off ebay. They're last gen ARM server chips. The performance won't be as good, but if its just for playing around with, they're more than adequate.
Can_Not 7y ago
I'm curious about what languages or types of projects are already running on ARM servers in the cloud (and actually benefiting!)
taf2 7y ago
Damn has anyone tried ruby on of these CPU’s is it really 20% perf improvement on nginx? These sound too good to be true
DSingularity 7y ago
Amazon will dominate cloud computing with these server CPUs. Who can compete with vertical integration at the sheer scale of AWS? AWS usage patterns tell them exactly what to accelerate with silicon. A process that has been largely driven by Intel will be replaced by a process driven by the customer workloads themselves. These processors will only get better with time.
- pdelgallego 7y ago
  That is basically Amazon scale playbook 101 (aka flywheel), if there is an efficiency that can benefit the customer (e.g. lowering prices), they will chase it. It doesn't matter if that means designing Graviton2 or challenging Fedex by trying to build the biggest delivery network in the USA.
fulafel 7y ago
I guess the real benchmark is whether they'll put it to use with Lambda, Fargate etc.
- ksec 7y ago
  I think Amazon mentioned they intend to use their own chip on All of AWS except their IaaS / EC2 offering, where you still get to choose Server running on x86. That is why it was mentioned as the fall of x86 on Servers.
  - fulafel 7y ago
    Sounds interesting, where did they say it?
    - ksec 7y ago
      And AWS' initial strategy is to move its internal services to Graviton2-based infrastructure. Graviton2 required significant investment, but AWS can garner returns and improve its operating margins due to the ability to cut out middlemen involved with procuring processors, power savings due to Arm and efficiency gains from optimizing its own infrastructure. AWS services like Amazon Elastic Load Balancing, Amazon ElastiCache, and Amazon Elastic Map Reduce have tested the AWS Graviton2 instances and plan to move them into production in 2020. Normally I try to find Primary sources rather than secondary like Zdnet [1]. But I think those exact wording was quite widely reported at the time. They say they are not Anti-Intel or AMD. Which is true. ( They are only Anti x86. ) And they say the same to UPS and Fedex at the time. [1] https://www.zdnet.com/article/aws-graviton2-what-it-means-for-arm-in-the-data-center-cloud-enterprise-aws/ https://www.zdnet.com/article/aws-graviton2-what-it-means-fo...

188 points by yarapavan 7y ago

* [James Hamilton] believe there is a high probability we are now looking at what will become the first high volume ARM Server. More speeds and feeds: >30B transistors in 7nm process 64KB icache, 64KB dcache, and 1MB L2 cache 2TB/s internal, full-mesh fabric Each vCPU is a full non-shared core (not SMT) Dual SIMD pipelines/core including ML optimized int8 and fp16 Fully cache coherent L1 cache 100% encrypted DRAM 8 DRAM channels at 3200 Mhz * ARM Servers have been inevitable for a long time but it’s great to finally see them here and in customers hands in large numbers.

kortilla 7y ago
Is there somewhere to buy something like this for use at home?
- downrightmike 7y ago
  Probably not until end of life for these chips
- wmf 7y ago
  Ampere QuickSilver, but don't expect an 80-core server to be cheap.
nine_k 7y ago
What really stands out for me is "100% encrypted DRAM". How efficient is this? Can different cores have different encryption keys, so that different VMs under a hypervisor can't benefit from breaking the hypervisor's protections?
- praseodym 7y ago
  I can’t speak for the Graviton2 CPUs, but AMD Epyc CPUs have RAM encryption with per-VM keys for increased isolation: https://developer.amd.com/sev/ https://developer.amd.com/sev/
- wwarner 7y ago
  Same question from me. At what point does dedicated hosting become more efficient than encrypted everything?
  - wmf 7y ago
    Pretty much never. With dedicated servers you don't have anyone to split the Nitro overhead with.
- amluto 7y ago
  Some Intel chips can encrypt memory with different keys for different VMs. This sounds great for marketing but adds basically no security whatsoever. The feature is called MKTME. What’s going on here is that “different keys for different VMs” does not actually improve isolation without a considerable amount of hardware or microcode enforcement. AMD has this type of tracking of which VM is which. Intel does not. I don’t know what AMD does. In any case, exception makes little difference. Cores aren’t bound 1:1 to VMs, so the core can access any VM’s data if it wants. And actually clearing the key on a context switch would require flushing caches and require that there is no cache shared between cores. The performance hit would be extreme.
  - nine_k 7y ago
    What is the point of sharing cache between VMs? Pinning a VM to a set of cores when encryption is enabled would make sense, and could be a feature cloud users would be willing to pay for.
    - amluto 7y ago
      Basically every recent CPU has a big cache shared between all cores. So, unless you pin a VM to one socket and you do something to encrypt cache coherency traffic between sockets per VM, you lose. The underlying issue here is that encryption is fast but not fast enough. So no one encrypts cache — instead, plaintext is cached and data is encrypted on its way to DRAM. So the actual isolation is in the access controls that the CPUs apply to which process or VM can access which pages, and this has little to do with encryption. It’s worth noting that Intel has been very bad lately at protecting cache contents from side channels, while AMD has done just fine. You can turn fancy encryption on, but those side channels leak plaintext.
  - thu2111 7y ago
    In fairness to Intel they also have SGX which has encrypted RAM and also a lot of isolation logic, as well as working RA, recovery, versioned sealed data and a lot of other things that AMD's equivalent just doesn't do well or at all.
- Koshkin 7y ago
  That seems like another blow to high-performance computing on VMs.

otterley 7y ago

(I work for AWS. Opinions are my own and not necessarily those of my employer.) I've been doing some initial M6g tests in my lab, and while I'm not able to disclose benchmarks, I can say that my real-world experience so far reflects what's been claimed elsewhere. Graviton2 is going to be a game changer. It's not like the usual experience with ARM where you have to trade off performance for price, and decide whether migrating is worth the recompilation effort. In my lab, performance of the workloads I've tried so far is uniformly better than on the equivalent M5 configuration running on the Intel processor. You're not sacrificing anything by running on Graviton2. If your workloads are based on scripting languages, Java, or Go, or you can recompile your C/C++ code, you're going to want to use these instances if you can. The pricing is going to make it irresistible. Basically, unless you're running COTS (commercial off-the-shelf software), it's a no-brainer.

rbranson 7y ago
I am extremely excited about this development. Less so getting everything that assumes x86 in the stack supporting a “parameterized” platform. (Hi Michael!)
- otterley 7y ago
  Hi Rick! I know it's only a part of the toolchain needed to support the migration, but multi-arch Docker image support for Amazon ECR is definitely on our immediate roadmap.
- floatboth 7y ago
  heh, I've had a very fun experience with spotinst.com — my a1 spot instance went down and that service couldn't restore it because it did not label the AMI as arm64. Reported that to them, got a couple acknowledgements but haven't heard back in a while, so presumably this is still not fixed.
chroem- 7y ago
Does the chip still manage to have superior power efficiency versus x86 even at these performance levels?

gok 7y ago

There are some interesting implications for widely deployed processors that are literally never publicly seen because they spend their whole life in a highly locked down data center. I wonder if things like Meltdown could have ever been discovered if researchers could only poke at the chips via EC2.

QuinnyPig 7y ago
I believe the “metal” variants expose the processor extensions you’d need to discover Meltdown.
- ec109685 7y ago
  That’s a good point. And AWS allows this type of security research as well freely: https://twitter.com/TeriRadichel/status/1101228943128969218 https://twitter.com/TeriRadichel/status/1101228943128969218
- floatboth 7y ago
  Also, AWS processors use off-the-shelf Arm Cortex/Neoverse cores, and stuff like Spectre is core-level.

neonate 7y ago

https://web.archive.org/web/20200125180037/https://perspectives.mvdirona.com/2020/01/aws-graviton2/ https://web.archive.org/web/20200125180037/https://perspecti...

miohtama 7y ago

This is good news. Are Linux server distributions for ARM64s yet on-par with their PC counterparts? Getting base layer software is not going to be an issue?

QuinnyPig 7y ago
Been using Ubuntu on one for a few weeks; the only things I missed was a few Docker containers that weren’t built, and aws-vault didn’t have an ARM binary. I built my own, and aws-vault shopped a new release with ARM support 20 minutes after I whined about it on Twitter. Everything else has been flawless.

Dunedan 7y ago

> Here’s comparative data between M6g and M5, the previous generation instance type Instead of comparing the 7nm Graviton2 processor against an 14nm Intel processor, I'd like to see its performance compared to an AMD Epyc 2 processor, which would be a more apples-to-apples comparison as both are "7nm" parts. Unfortunately Epyc 2 processors aren't available from AWS yet (but are already announced: https://aws.amazon.com/de/blogs/aws/in-the-works-new-amd-powered-compute-optimized-ec2-instances-c5a-c5ad/ https://aws.amazon.com/de/blogs/aws/in-the-works-new-amd-pow...).

Rapzid 7y ago
This is what I'm curious about as epyc currently represents the state of the art x86 right? What's the TCO comparison when factoring in performance, density, and power?
reitzensteinm 7y ago
You're putting way too much emphasis on process tech. TSMC 7nm is roughly equivalent to Intel 10nm, so it's only a generation behind. Intel 10nm products, where available, have also not exactly been lighting the world on fire with their performance.
- tpurves 7y ago
  No. Process tech is profoundly correlated to cpu performance. It’s literallly the definition of Moore’s law. Intel has no 10nm server processors. Intel’s near 5 year delay in getting past 14nm has opened a huge window of opportunity to competitors both AMD and amazon/arm. It is highly worth comparing these competitors apple to apple vs Intel’s oranges.
  - reitzensteinm 7y ago
    "No." yourself :) Smaller processes used to mean higher frequency switching, lower power and increased density. With the death of Dennard scaling, we mostly just get the latter. This means that the benefit is now largely economic; you get largely the same chips, you can just pack them more tightly on the wafer. If you're one node behind, you still price the chips at a price the market will bear, they just cost you a bit more to produce. And maybe not even then; mature last generation nodes perform pretty damn well against immature next generation nodes once you take yield and performance in to account. Intel's 14nm transition yielded Broadwell Xeon [1], barely any improvement over the Haswell chips. Haswell itself, however, gave us a ~50% performance boost on the same process node. This is the difference between a new process node and a new architecture in today's world. The reason Intel is in trouble is because of the 10nm fiasco, but not because of the lack of a die shrink. Their shrinks have been working like a well oiled machine for decades, and there was no contingency in place for a large delay. All post Skylake chips were being developed tightly against their 10nm libraries, with no possibility of a back port. It's not the lack of a Haswell->Broadwell analogous die shrink that's hurting Intel, but a Haswell->Broadwell->Skylake die shrink + new architecture. How do you know this is true? Because Intel switched gears and is now decoupling future architectures from die shrinks. If they did this earlier, you'd be seeing Ice Lake (or maybe Tiger Lake) on 14nm++ as an answer to Zen 2, and it would be a pretty good chip. Instead they're doing whatever minor tweaks they can to so many variations of Skylake I'm not sure I could list all the codenames from memory. Zen 2 is a seriously formidable chip, but most of the benefit came from cleaning up nasty edge cases in performance, like cross core communication being slower than a spill to DRAM. You can't disentangle the shrink from the architecture, because they happened simultaneously. [1] https://www.anandtech.com/show/10158/the-intel-xeon-e5-v4-review https://www.anandtech.com/show/10158/the-intel-xeon-e5-v4-re... [2] https://www.anandtech.com/show/8423/intel-xeon-e5-version-3-up-to-18-haswell-ep-cores- https://www.anandtech.com/show/8423/intel-xeon-e5-version-3-...

pranith 7y ago

I wonder when and how Azure and Google Cloud will compete with AWS in this market. They could buy ARM processors available in the market, but I doubt they will be able to get them as cheap as AWS who builds their own.

justicezyx 7y ago
I led the support for multi arch images for Borg. Google had the software stack ready for internal workload longtime ago, PowerPC was used https://www.forbes.com/sites/patrickmoorhead/2018/03/19/headed-into-its-fifth-year-openpower-has-momentum-into-the-power9-generation/ https://www.forbes.com/sites/patrickmoorhead/2018/03/19/head...
- pranith 7y ago
  I am sure the software is multi-arch ready but I wonder if they are evaluating ARM servers either for use internally or to launch in the cloud...
gundmc 7y ago
Does Microsoft make any of their own silicon? It feels like Google has been directing their in-house designs on ML/TPUs while Amazon went all in on ARM. It will be interesting to see how those bets pay off.
- floatboth 7y ago
  No, but Microsoft bought some off the shelf Ampere and Cavium/Marvell servers. But they keep them for internal use only for now :( Huawei makes their own silicon and servers with that silicon — also only internal, not available on huaweicloud :( The only other player is Scaleway who bought first gen Cavium ThunderX's way back when. And Packet of course but that's bare metal only, no cheap small VPSes.
  - pranith 7y ago
    If they are good enough for internal use, they should be good enough for public use. Not really sure what is stopping them from exposing it to the public... I am sure there is _some_ demand for it.

sdan 7y ago

This is great news. Except that a ton of software doesn't support ARM. When I was trying to shift all my current infrastructure onto a couple of RPI's, many of the Docker containers didn't support ARM (qeumu and buildx aren't reliable) and other software didn't support ARM either. Unless there's a good way to go from AMD to ARM, I'm not entirely sure how great Graviton or other competitors will get.

JunkDNA 7y ago
Back in the late 90’s and early 00’s, there were a ton of cpu platforms around: SGI MIPS, DEC Alpha’s, Intel, Sun SPARC, etc... while I will admit it was a colossal pain working somewhere that had all of those, it was often possible to recompile from source to get things to run. I’m not suggesting it’s trivial, but given the incredible investment in ARM in the mobile space, the wind is at least at your back today. It certainly has got to be much easier than it was in the days of being the only person in the world trying to recompile an obscure open source scientific computing package for DEC Alpha. Commercial software is a different beast, but even there, the incentive will be high to do a port if lots of people start migrating to this.
- dehrmann 7y ago
  Windows NT 4 supported x86, Alpha, MIPS, and PowerPC. Yikes.
  - wbl 7y ago
    All have eight bit bytes with 2's complement, same availible word sizes and float formats (with some complexity on the Alpha due to VAX compat). C code will mostly not care beyond endianness. The PDP is the strange one.
ksec 7y ago
Assuming Amazon intend to upstream their work. All the testing of open source stack Amazon uses internally will support ARM. That is all of their Hosted Open Sources offering. This will kickstart all software support. AWS ARM offers cost advantage which proprietary software now has an incentives or their customer will request ARM support. All these will create a positive feedback loop into the ecosystem.

gautamcgoel 7y ago

This is great, but I'd be really excited if we could go out and buy the chips ourselves instead of having to pay the Amazon tax and run our code on untrusted systems in the cloud. Of course, Amazon has little incentive to sell the chips, since it gives them a competitive advantage against other cloud providers.

ijl 7y ago

What services are people using to run continuous integration for ARM? I see Travis CI has an alpha. Azure Pipelines doesn't host ARM instances I think.

otterley 7y ago
AWS CodeBuild supports ARM builds: https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codebuild-adds-support-for-arm--gpu--and-x-large-compute-typ/ https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codeb...
oldmanhorton 7y ago
Azure Pipelines doesn't host arm yet, but you can run a self hosted pool on a Gravitron2 VM pretty easily.
jedbrown 7y ago
GitLab-CI can use any host (and can be used with GitHub, though the PR integration is not nearly as nice as when used with a GitLab MR).

dehrmann 7y ago

Out of curiosity, what's the state of Jazelle on modern ARM? Would it help server-side ARM, or has the world (and JITs) moved on?

pm215 7y ago
Jazelle is dead. The v8 version (or maybe even v7; I forget) of the 32-bit architecture basically mandated that only 'trivial' Jazelle (which is the not-actually-there version) could be implemented, and 64-bit has never had anything like it. It was at best a technology of its time (when phone Java implementations were mostly interpreted, not JITs). It would be useless to a modern Java implementation.
thu2111 7y ago
Jazelle accelerated bytecode interpreting (a bit) but modern JVMs spend nearly all their time running compiled code, so it doesn't really help and was abandoned. There are CPU HW features that Intel doesn't have which can benefit JVM workloads but they're all pretty obscure and aren't really Java specific.

dehrmann 7y ago

Are there any security fears with virtualization on ARM (think Meltdown and Spectre)? I'd think it's been less studied than Intel's x86-64 chips.

Koshkin 7y ago

Annapurna, the goddess of job security.

xwowsersx 7y ago

> believing that massive client volumes fund the R&D stream that feeds most server-side innovation. What does he mean here?

deleted 7y ago
[deleted]
wmf 7y ago
Intel/AMD design laptop/desktop (client) cores then put those cores into server processors. Because vastly more PCs are sold, they effectively subsidize server processors. Arm has a similar advantage, designing cores for phones/tablets and repurposing/extending them for servers.
- xwowsersx 7y ago
  Ohh I see, thank you!

dman 7y ago

When can I buy something similar for a homelab?

ipsum2 7y ago
You can buy Cavium Thunderx2s off ebay. They're last gen ARM server chips. The performance won't be as good, but if its just for playing around with, they're more than adequate.

Can_Not 7y ago

I'm curious about what languages or types of projects are already running on ARM servers in the cloud (and actually benefiting!)

taf2 7y ago

Damn has anyone tried ruby on of these CPU’s is it really 20% perf improvement on nginx? These sound too good to be true

DSingularity 7y ago

Amazon will dominate cloud computing with these server CPUs. Who can compete with vertical integration at the sheer scale of AWS? AWS usage patterns tell them exactly what to accelerate with silicon. A process that has been largely driven by Intel will be replaced by a process driven by the customer workloads themselves. These processors will only get better with time.

pdelgallego 7y ago
That is basically Amazon scale playbook 101 (aka flywheel), if there is an efficiency that can benefit the customer (e.g. lowering prices), they will chase it. It doesn't matter if that means designing Graviton2 or challenging Fedex by trying to build the biggest delivery network in the USA.

fulafel 7y ago

I guess the real benchmark is whether they'll put it to use with Lambda, Fargate etc.

ksec 7y ago
I think Amazon mentioned they intend to use their own chip on All of AWS except their IaaS / EC2 offering, where you still get to choose Server running on x86. That is why it was mentioned as the fall of x86 on Servers.
- fulafel 7y ago
  Sounds interesting, where did they say it?
  - ksec 7y ago
    And AWS' initial strategy is to move its internal services to Graviton2-based infrastructure. Graviton2 required significant investment, but AWS can garner returns and improve its operating margins due to the ability to cut out middlemen involved with procuring processors, power savings due to Arm and efficiency gains from optimizing its own infrastructure. AWS services like Amazon Elastic Load Balancing, Amazon ElastiCache, and Amazon Elastic Map Reduce have tested the AWS Graviton2 instances and plan to move them into production in 2020. Normally I try to find Primary sources rather than secondary like Zdnet [1]. But I think those exact wording was quite widely reported at the time. They say they are not Anti-Intel or AMD. Which is true. ( They are only Anti x86. ) And they say the same to UPS and Fedex at the time. [1] https://www.zdnet.com/article/aws-graviton2-what-it-means-for-arm-in-the-data-center-cloud-enterprise-aws/ https://www.zdnet.com/article/aws-graviton2-what-it-means-fo...