6 ms·

Building server software on Graviton ARM creates a vendor lock-in to Amazon, with very high costs of switching elsewhere. Despite using A64 ISA and ARM’s cores,

Building server software on Graviton ARM creates a vendor lock-in to Amazon, with very high costs of switching elsewhere. Despite using A64 ISA and ARM’s cores, they are Amazon’s proprietary chips no one else has access to. Migrating elsewhere gonna be very expensive. I wouldn’t be surprised if they sponsor their Graviton offering taking profits elsewhere. This might make it seem like a good deal for customers, but I don’t think it is, at least not in the long run. This doesn’t mean Graviton is useless. For services running Amazon’s code as opposed to customer’s code (like these PAAS things billed per transaction) the lock-in is already in place, custom processors aren’t gonna make it any worse.

dragontamer 6y ago
I'm not necessarily disagreeing with you, but... maybe elaborating in a contrary manner? Graviton ARM is certainly vendor lock-in to Amazon. But a Graviton ARM is just a bog-standard Neoverse N1 core. Which means the core is going to show similar characteristics as the Ampere Altra (also a bog-standard Neoverse N1 core). There's more to a chip than its core. But... from a performance-portability and ISA perspective... you'd expect performance-portability between Graviton ARM and Ampere Altra. Now Ampere Altra is like 2x80 core, while Graviton ARM is... a bunch of different configurations. So its still not perfect compatibility. But a single-threaded program probably couldn't tell the difference between the two platforms. I'd expect that migrating between Graviton and Ampere Altra is going to be easier than Intel Skylake -> AMD Zen.
- Const-me 6y ago
  > you'd expect performance-portability between Graviton ARM and Ampere Altra I agree, that would what I would expect too. Still, are there many public clouds built of these Ampere Altra-s? Maybe we gonna have them widespread soon, but until then I wouldn’t want to build stuff that only runs on Amazon or my own servers with only a few on the market and not yet globally available on retail. Also, AFAIK on ARM the parts where CPUs integrate with the rest of the hardware are custom. The important thing for servers, disk and network I/O differs across ARM chips of the same ISA. Linux kernel abstracts it away i.e. stuff is likely to work, but I’m not so sure about performance portability.
  - dragontamer 6y ago
    > Also, AFAIK on ARM the parts where CPUs integrate with the rest of the hardware are custom. The important thing for servers, disk and network I/O differs across ARM chips of the same ISA. Linux kernel abstracts it away i.e. stuff is likely to work, but I’m not so sure about performance portability. Indeed. But Intel Xeon + Intel Ethernet integrates tightly and drops the Ethernet data directly into L3 cache (bypassing DRAM entirely). As such, I/O performance portability between x86 servers (in particular: Intel Xeon vs AMD EPYC) suffers from similar I/O issues. Even if you have AMD EPYC + Intel Ethernet, you lose the direct-to-L3 DMA, and will have slightly weaker performance characteristics compared to Intel Xeon + Intel Ethernet. Or Intel Xeon + Optane optimizations, which also do not exist on AMD EPYC + Optane. So these I/O performance differences between platforms are already on the status-quo, and should be expected if you're migrating between platforms. A degree of testing and tuning is always needed when changing platforms. -------- >Still, are there many public clouds built of these Ampere Altra-s? Maybe we gonna have them widespread soon, but until then I wouldn’t want to build stuff that only runs on Amazon or my own servers with only a few on the market and not yet globally available on retail. A fair point. Still, since Neoverse N1 is a premade core available to purchase from ARM, many different companies have the ability to buy it for themselves. Current rumors look like Microsoft/Oracle are just planning to use Ampere Altra. But like all other standard ARM cores, any company can buy the N1 design and make their own chip.
    - yaantc 6y ago
      > > Also, AFAIK on ARM the parts where CPUs integrate with the rest of the hardware are custom. The important thing for servers, disk and network I/O differs across ARM chips of the same ISA. Linux kernel abstracts it away i.e. stuff is likely to work, but I’m not so sure about performance portability. > Indeed. But Intel Xeon + Intel Ethernet integrates tightly and drops the Ethernet data directly into L3 cache (bypassing DRAM entirely). This will be less of a problem on ARM servers as direct access to the LLC from a hardware master is a standard feature of ARM's "Dynamic Shared Unit" or DSU, which is the shared part of a cluster providing the LLC and coherency support. Connect a hardware function to the DSU ACP (accelerator coherency port) and the hardware can control, for all write accesses, whether to "stash" data into the LLC or even the L2 or L1 of a specific core. The hardware can also control allocate on miss vs not. So any high performance IP can benefit from it. And if I understand correctly, the DSU is required with modern ARM cores. As most (besides Apple) tend to use ARM cores now, you have this in the package. More details here in the DSU tech manual: https://developer.arm.com/documentation/100453/0002/functional-description/introduction/about-the-dsu https://developer.arm.com/documentation/100453/0002/function...
- Zardoz84 6y ago
  > I'd expect that migrating between Graviton and Ampere Altra is going to be easier than Intel Skylake -> AMD Zen. Could you explain what migration problems are between Skylake and Zen, beyond AVX-512 ?
treve 6y ago
Maybe I'm missing something, but don't the vast majority of applications don't care about what architecture they run on? The main difference for us was lower bills.
- Slartie 6y ago
  Even if the applications don't care, there's still the (Docker) container, which cares very much, and which seems to be the vehicle of choice to package and deliver many cloud-based applications today. Being able to actually run the exact same containers on your dev machine which are going to be running on the servers later is definitely a big plus.
  - acdha 6y ago
    Docker has had multiarch support for a while and most of the containers I’ve looked at support both. That’s not to say this won’t be a concern but it’s at the level of “check a box in CI” to solve and between Apple and Amazon there’ll be quite a few users doing that.
- MaxBarraclough 6y ago
  > Maybe I'm missing something, but don't the vast majority of applications don't care about what architecture they run on? There can be issues with moving to AArch64, for instance your Python code may depend on Python 'wheels' which in turn depend on C libraries that don't play nice with AArch64. I once encountered an issue like this, although I've now forgotten the details. If your software is pure Java I'd say the odds are pretty good that things will 'just work', but you'd still want to do testing.
  - matwood 6y ago
    Sure, but you're talking about short term problems. RPi, Graviton, Apple Silicon, etc... are making AArch64 a required mainstream target.
    - MaxBarraclough 6y ago
rapsey 6y ago
Why would it be lock in. If you can compile for arm you can compile for x86.
- optimiz3 6y ago
  Memory model, execution units, simd instructions...
  - oblio 6y ago
    The real question is, can you compile for ARM and move the binary around as easily as you can for x86? I'm reasonably sure that you can take a binary compiled with GCC on a P4 back in the day and run it on the latest Zen 3 CPU.
    - easton 6y ago
      As far as I can tell, yes. Docker images compiled for arm64 work fine on the Macs with M1 chips without rebuilding. And as another commenter said, you can compile a binary on a Raspberry Pi 4 and move it to a EC2 graviton instance and it just works.
    - cozzyd 6y ago
      it will probably be a similar situation to x86, with various vendors implementing various instructions in some processors that won't be supported by all. I guess the difference is that there may be many more variants than in x86, but performance-critical code can always use runtime dispatch mechanisms to adapt.
      - oblio 6y ago
        It's true that there are extensions to x86, but 99,99% of software out there (the one you'd commonly install on Windows or find in Linux distribution repos) doesn't use those instructions or maybe just detects the features and then uses it. I don't recall encountering a "Intel-locked" or "AMD-locked" application in more than 20 years of using x86. Ok, maybe ICC, but that one kind of makes sense :-)
jorblumesea 6y ago
Really, you could make the argument for any AWS service and generally using a cloud service provider. You get into the cloud, use their glue (lambda, kinesis, sqs etc) and suddenly migrating services somewhere else is a multi-year project. Do you think that vendor lock in has stopped people in the past (and future)? Thinking about those kinds of things are long term and many companies think short term.
- ralph84 6y ago
  Heck, Amazon themselves got locked-in to Oracle for the first 25 years of Amazon's existence. Vendor lock-in for your IT stack doesn't prevent you from becoming a successful business.
  - PaulDavisThe1st 6y ago
    True, true (and heh, it was me who pushed for Oracle, oops) But ... the difference is that Oracle wasn't a platform in the sense that (e.g.) AWS is. Oracle as a corporation could vanish, but as long as you can keep running a compatible OS on compatible hardware, you can keep using Oracle. If AWS pulls the plug on you, either as an overall customer or ends a particular API/service, what do you do then?
    - outside1234 6y ago
      You post a nasty note on Parler! Oh wait.
deaddodo 6y ago
> they are Amazon’s proprietary chips no one else has access to. Any ARM licensee (IP or architecture) has access to them. They're just NeoVerse N1 cores and can be synthesized on Samsung or TSMC processes.
chasil 6y ago
As I understand it, ARM's new willingness to allow custom op-codes is dependent upon the customer preventing fragmentation of the ARM instruction set. In theory, your software could run faster, or slower, depending upon Amazon's use of their extensions within their C library, or associated libraries in their software stack. Maybe the wildest thing that I've heard is Fujitsu not implementing either 32-bit or Thumb on their new supercomputer. Is that a special case? "But why doesn’t Apple document this and let us use these instructions directly? As mentioned earlier, this is something ARM Ltd. would like to avoid. If custom instructions are widely used it could fragment the ARM ecosystem." https://medium.com/swlh/apples-m1-secret-coprocessor-6599492fc1e1 https://medium.com/swlh/apples-m1-secret-coprocessor-6599492...
- billiam 6y ago
  It's interesting that if you step back and look at what Amazon has been most willing to just blow up and destroy, it is the idea of intellectual property of any kind. It comes out clearly in their business practices. This muscle memory may make it hard for ARM to have a long term stable relationship with a company like ARM.
  - oblio 6y ago
    What do you mean? Also, I think there's a typo in your last phrase.
- stephencanon 6y ago
  > Maybe the wildest thing that I've heard is Fujitsu not implementing either 32-bit or Thumb on their new supercomputer. Is that a special case? What's wild about this? Apple dropped support for 32b (arm and thumb) years ago with A11. Supporting it makes even less sense in an HPC design than it does in a phone CPU.
gchamonlive 6y ago
I think OP was talking about managed services, like lambda, Ecs and beanstalk internal control, EC2 internal management system, that is systems that are transparent for the user. AWS could very well run their platform systems entirely on graviton. After all, serverless and cloud is in essence someone else's server. AWS might as well run all their paas software on in-house architecture
- ogre_codes 6y ago
  While there is vendor lock in with those services, it also has nothing to do with what CPU you are running. At that layer, CPU is completely abstract.
  - gchamonlive 6y ago
    Maybe I wasn't clear enough. I am talking about code that runs behind the scenes. Management processes, schedulers, server allocation procedures, everything that runs on the aws side of things, transparent for the client.
timthorn 6y ago
Ubuntu 64 looks the same on Graviton as on a Raspberry Pi. You can take a binary you've compiled on the RPi, scp it to the Graviton instance and it will just run. That works the other way round too, which is great for speedy Pi software builds without having to set up a cross-compile environment.
- erosenbe0 6y ago
  Yep just Aarch64. Probably can use qemu too. Cross compilation is no big deal these days. Embedded devs cross compile to ARM all day every day. The tooling will be there when it needs to be.
echelon 6y ago
> Building server software on Graviton ARM creates a vendor lock-in to Amazon Amazon already has lock-in. Lambda, SQS, etc. They've already won. You might be able to steer your org away from this, but Amazon's gravity is strong.
pjmlp 6y ago
My Java and .NET applications don't care most of the time in what hardware they are running, and many of other languages managed languages I use also do not, even if AOT compiled to native code. That is the beauty of having proper defined numeric types and memory model, instead of the C and derived approaches of whatever the CPU gives, with whatever memory model.

dragontamer 6y ago

I'm not necessarily disagreeing with you, but... maybe elaborating in a contrary manner? Graviton ARM is certainly vendor lock-in to Amazon. But a Graviton ARM is just a bog-standard Neoverse N1 core. Which means the core is going to show similar characteristics as the Ampere Altra (also a bog-standard Neoverse N1 core). There's more to a chip than its core. But... from a performance-portability and ISA perspective... you'd expect performance-portability between Graviton ARM and Ampere Altra. Now Ampere Altra is like 2x80 core, while Graviton ARM is... a bunch of different configurations. So its still not perfect compatibility. But a single-threaded program probably couldn't tell the difference between the two platforms. I'd expect that migrating between Graviton and Ampere Altra is going to be easier than Intel Skylake -> AMD Zen.

Const-me 6y ago
> you'd expect performance-portability between Graviton ARM and Ampere Altra I agree, that would what I would expect too. Still, are there many public clouds built of these Ampere Altra-s? Maybe we gonna have them widespread soon, but until then I wouldn’t want to build stuff that only runs on Amazon or my own servers with only a few on the market and not yet globally available on retail. Also, AFAIK on ARM the parts where CPUs integrate with the rest of the hardware are custom. The important thing for servers, disk and network I/O differs across ARM chips of the same ISA. Linux kernel abstracts it away i.e. stuff is likely to work, but I’m not so sure about performance portability.
- dragontamer 6y ago
  > Also, AFAIK on ARM the parts where CPUs integrate with the rest of the hardware are custom. The important thing for servers, disk and network I/O differs across ARM chips of the same ISA. Linux kernel abstracts it away i.e. stuff is likely to work, but I’m not so sure about performance portability. Indeed. But Intel Xeon + Intel Ethernet integrates tightly and drops the Ethernet data directly into L3 cache (bypassing DRAM entirely). As such, I/O performance portability between x86 servers (in particular: Intel Xeon vs AMD EPYC) suffers from similar I/O issues. Even if you have AMD EPYC + Intel Ethernet, you lose the direct-to-L3 DMA, and will have slightly weaker performance characteristics compared to Intel Xeon + Intel Ethernet. Or Intel Xeon + Optane optimizations, which also do not exist on AMD EPYC + Optane. So these I/O performance differences between platforms are already on the status-quo, and should be expected if you're migrating between platforms. A degree of testing and tuning is always needed when changing platforms. -------- >Still, are there many public clouds built of these Ampere Altra-s? Maybe we gonna have them widespread soon, but until then I wouldn’t want to build stuff that only runs on Amazon or my own servers with only a few on the market and not yet globally available on retail. A fair point. Still, since Neoverse N1 is a premade core available to purchase from ARM, many different companies have the ability to buy it for themselves. Current rumors look like Microsoft/Oracle are just planning to use Ampere Altra. But like all other standard ARM cores, any company can buy the N1 design and make their own chip.
  - yaantc 6y ago
    > > Also, AFAIK on ARM the parts where CPUs integrate with the rest of the hardware are custom. The important thing for servers, disk and network I/O differs across ARM chips of the same ISA. Linux kernel abstracts it away i.e. stuff is likely to work, but I’m not so sure about performance portability. > Indeed. But Intel Xeon + Intel Ethernet integrates tightly and drops the Ethernet data directly into L3 cache (bypassing DRAM entirely). This will be less of a problem on ARM servers as direct access to the LLC from a hardware master is a standard feature of ARM's "Dynamic Shared Unit" or DSU, which is the shared part of a cluster providing the LLC and coherency support. Connect a hardware function to the DSU ACP (accelerator coherency port) and the hardware can control, for all write accesses, whether to "stash" data into the LLC or even the L2 or L1 of a specific core. The hardware can also control allocate on miss vs not. So any high performance IP can benefit from it. And if I understand correctly, the DSU is required with modern ARM cores. As most (besides Apple) tend to use ARM cores now, you have this in the package. More details here in the DSU tech manual: https://developer.arm.com/documentation/100453/0002/functional-description/introduction/about-the-dsu https://developer.arm.com/documentation/100453/0002/function...
Zardoz84 6y ago
> I'd expect that migrating between Graviton and Ampere Altra is going to be easier than Intel Skylake -> AMD Zen. Could you explain what migration problems are between Skylake and Zen, beyond AVX-512 ?

treve 6y ago

Maybe I'm missing something, but don't the vast majority of applications don't care about what architecture they run on? The main difference for us was lower bills.

Slartie 6y ago
Even if the applications don't care, there's still the (Docker) container, which cares very much, and which seems to be the vehicle of choice to package and deliver many cloud-based applications today. Being able to actually run the exact same containers on your dev machine which are going to be running on the servers later is definitely a big plus.
- acdha 6y ago
  Docker has had multiarch support for a while and most of the containers I’ve looked at support both. That’s not to say this won’t be a concern but it’s at the level of “check a box in CI” to solve and between Apple and Amazon there’ll be quite a few users doing that.
MaxBarraclough 6y ago
> Maybe I'm missing something, but don't the vast majority of applications don't care about what architecture they run on? There can be issues with moving to AArch64, for instance your Python code may depend on Python 'wheels' which in turn depend on C libraries that don't play nice with AArch64. I once encountered an issue like this, although I've now forgotten the details. If your software is pure Java I'd say the odds are pretty good that things will 'just work', but you'd still want to do testing.
- matwood 6y ago
  Sure, but you're talking about short term problems. RPi, Graviton, Apple Silicon, etc... are making AArch64 a required mainstream target.
  - MaxBarraclough 6y ago

rapsey 6y ago

Why would it be lock in. If you can compile for arm you can compile for x86.

optimiz3 6y ago
Memory model, execution units, simd instructions...
- oblio 6y ago
  The real question is, can you compile for ARM and move the binary around as easily as you can for x86? I'm reasonably sure that you can take a binary compiled with GCC on a P4 back in the day and run it on the latest Zen 3 CPU.
  - easton 6y ago
    As far as I can tell, yes. Docker images compiled for arm64 work fine on the Macs with M1 chips without rebuilding. And as another commenter said, you can compile a binary on a Raspberry Pi 4 and move it to a EC2 graviton instance and it just works.
  - cozzyd 6y ago
    it will probably be a similar situation to x86, with various vendors implementing various instructions in some processors that won't be supported by all. I guess the difference is that there may be many more variants than in x86, but performance-critical code can always use runtime dispatch mechanisms to adapt.
    - oblio 6y ago
      It's true that there are extensions to x86, but 99,99% of software out there (the one you'd commonly install on Windows or find in Linux distribution repos) doesn't use those instructions or maybe just detects the features and then uses it. I don't recall encountering a "Intel-locked" or "AMD-locked" application in more than 20 years of using x86. Ok, maybe ICC, but that one kind of makes sense :-)

jorblumesea 6y ago

Really, you could make the argument for any AWS service and generally using a cloud service provider. You get into the cloud, use their glue (lambda, kinesis, sqs etc) and suddenly migrating services somewhere else is a multi-year project. Do you think that vendor lock in has stopped people in the past (and future)? Thinking about those kinds of things are long term and many companies think short term.

ralph84 6y ago
Heck, Amazon themselves got locked-in to Oracle for the first 25 years of Amazon's existence. Vendor lock-in for your IT stack doesn't prevent you from becoming a successful business.
- PaulDavisThe1st 6y ago
  True, true (and heh, it was me who pushed for Oracle, oops) But ... the difference is that Oracle wasn't a platform in the sense that (e.g.) AWS is. Oracle as a corporation could vanish, but as long as you can keep running a compatible OS on compatible hardware, you can keep using Oracle. If AWS pulls the plug on you, either as an overall customer or ends a particular API/service, what do you do then?
  - outside1234 6y ago
    You post a nasty note on Parler! Oh wait.

deaddodo 6y ago

> they are Amazon’s proprietary chips no one else has access to. Any ARM licensee (IP or architecture) has access to them. They're just NeoVerse N1 cores and can be synthesized on Samsung or TSMC processes.

chasil 6y ago

As I understand it, ARM's new willingness to allow custom op-codes is dependent upon the customer preventing fragmentation of the ARM instruction set. In theory, your software could run faster, or slower, depending upon Amazon's use of their extensions within their C library, or associated libraries in their software stack. Maybe the wildest thing that I've heard is Fujitsu not implementing either 32-bit or Thumb on their new supercomputer. Is that a special case? "But why doesn’t Apple document this and let us use these instructions directly? As mentioned earlier, this is something ARM Ltd. would like to avoid. If custom instructions are widely used it could fragment the ARM ecosystem." https://medium.com/swlh/apples-m1-secret-coprocessor-6599492fc1e1 https://medium.com/swlh/apples-m1-secret-coprocessor-6599492...

billiam 6y ago
It's interesting that if you step back and look at what Amazon has been most willing to just blow up and destroy, it is the idea of intellectual property of any kind. It comes out clearly in their business practices. This muscle memory may make it hard for ARM to have a long term stable relationship with a company like ARM.
- oblio 6y ago
  What do you mean? Also, I think there's a typo in your last phrase.
stephencanon 6y ago
> Maybe the wildest thing that I've heard is Fujitsu not implementing either 32-bit or Thumb on their new supercomputer. Is that a special case? What's wild about this? Apple dropped support for 32b (arm and thumb) years ago with A11. Supporting it makes even less sense in an HPC design than it does in a phone CPU.

gchamonlive 6y ago

I think OP was talking about managed services, like lambda, Ecs and beanstalk internal control, EC2 internal management system, that is systems that are transparent for the user. AWS could very well run their platform systems entirely on graviton. After all, serverless and cloud is in essence someone else's server. AWS might as well run all their paas software on in-house architecture

ogre_codes 6y ago
While there is vendor lock in with those services, it also has nothing to do with what CPU you are running. At that layer, CPU is completely abstract.
- gchamonlive 6y ago
  Maybe I wasn't clear enough. I am talking about code that runs behind the scenes. Management processes, schedulers, server allocation procedures, everything that runs on the aws side of things, transparent for the client.

timthorn 6y ago

Ubuntu 64 looks the same on Graviton as on a Raspberry Pi. You can take a binary you've compiled on the RPi, scp it to the Graviton instance and it will just run. That works the other way round too, which is great for speedy Pi software builds without having to set up a cross-compile environment.

erosenbe0 6y ago
Yep just Aarch64. Probably can use qemu too. Cross compilation is no big deal these days. Embedded devs cross compile to ARM all day every day. The tooling will be there when it needs to be.

echelon 6y ago

> Building server software on Graviton ARM creates a vendor lock-in to Amazon Amazon already has lock-in. Lambda, SQS, etc. They've already won. You might be able to steer your org away from this, but Amazon's gravity is strong.

pjmlp 6y ago

My Java and .NET applications don't care most of the time in what hardware they are running, and many of other languages managed languages I use also do not, even if AOT compiled to native code. That is the beauty of having proper defined numeric types and memory model, instead of the C and derived approaches of whatever the CPU gives, with whatever memory model.