13 ms·

AWS launches ARM-powered Lambdas

270 points by orf 5y ago

OJFord 5y ago
This seems pretty huge ('19% better performance at 20% lower cost') considering many (most?) Lambda workloads will be arch-agnostic, not even needing updated scripts/installed libs etc. Then many more will only need a trivial re-compile. I'm sure they don't need me to tell them they'll be inundated with demand...
- orf 5y ago
  It will probably take a while to ramp up - it's explicitly opt-in and I expect there's a huge amount of existing code that creates lambdas without being able to specify the architecture. For starters example Terraform and Cloudformation don't support it yet. And if you're using a 3rd party layer you need to wait for them to publish a layer that explicitly supports that architecture.
  - k__ 5y ago
    According to AWS, CFN rollout is happening right now.
    - orf 5y ago
      Sure but then you have the CDK, etc. I’d be interested to know how many people create lambdas in the console and not via some Other tooling, which needs to be updated to support this.
      - k__ 5y ago
        CDK support is out too.
      - edhzsz 5y ago
        Not saying that you are wrong, but CDK is ready now: https://github.com/aws/aws-cdk/releases/tag/v1.125.0 https://github.com/aws/aws-cdk/releases/tag/v1.125.0
  - OJFord 5y ago
    Good point. I'd forgotten memory was the only existing toggle (i.e. no variation of x86 CPUs as on EC2, that 'g' or whatever could just be added to).
  - smoldesu 5y ago
    Precisely. Once you start to consider how other ISAs factor into that (like perhaps RISC-V on the horizon) I really wonder if ARM has a chance to take off in the first place. A ~50% price/performance boost isn't negligible, but it's also not very compelling. Especially for larger companies who'd need to invest more time/money into transitioning their backend.
    - rewma 5y ago
      > A ~50% price/performance boost isn't negligible, but it's also not very compelling. My take is that a 50% discount in price/performance and a 20% discount in price per execution unit that's only a toggle flip away is something that everyone loves and is eager to jump onboard. The only exception is when your workload stubbornly stays within free tier limits, which in that case you have no compelling reason to do the work.
    - staticassertion 5y ago
      50% price/performance is very compelling. If you talk to a large company and say "you can cut your compute bill in half with a single re-deploy" they will be very happy.
  - paranoidrobot 5y ago
    Terraform issue tracking support for architectures parameter: https://github.com/hashicorp/terraform-provider-aws/issues/21090 https://github.com/hashicorp/terraform-provider-aws/issues/2... There's a pull request that went up about 40 minutes prior to my comment. So it looks like it's coming, soon.
    - reilly3000 5y ago
      Pulumi has theirs up already: https://www.pulumi.com/blog/aws-lambda-functions-powered-by-graviton2/ https://www.pulumi.com/blog/aws-lambda-functions-powered-by-...
- rewma 5y ago
  > This seems pretty huge ('19% better performance at 20% lower cost') I believe the performance boost will not be meaningful and the only important factor is cost per GB*s, given that most Lambda workloads are IO bound or simple event handlers that barely take 1s to run anyway. Nevertheless this is exciting stuff, specially given that NodeJS might be lambda's most popular runtime and thus we might only be a toggle flip away from a 20% discount.
  - ryanmarsh 5y ago
    My Lambda workloads are CPU bound so this is great but I get it, most spend their time waiting for IO.
  - diskzero 5y ago
    I wrote a lambda that encoded a timeline comprised of audio, video and GLSL shaders using SwiftShader and ffmpeg. Time to give it try again!
  - bonobocop 5y ago
    Apparently Python rules the roost for Lambda: https://mobile.twitter.com/julian_wood/status/1442775542374277120/photo/1 https://mobile.twitter.com/julian_wood/status/14427755423742... But NodeJS is second!
    - zxcvbn4038 5y ago
      Golang is the top dog for Lambda IMO. The AWS Lambda SDK for Golang includes the entire lambda runtime so you can deploy a single 20mb binary w/ an Alpine linux base (speaking of the ECR backed lambdas) and that’s it. With NodeJS you either have to use the AWS provided centos based images which weigh in at several hundred MB or do some multistage magic to build a minimal Alpine image including the lambda runtime and AWS SDK which altogether still weighs in almost a hundred MB.
      - bonobocop 5y ago
        TBH, as the Dockerfile based Lambda layers have a 20GB file size limit, I'm not sure the size matters as much. It's certainly easier to figure out what's going on in a smaller container though! I've had to debug some nasty situation with layers in Python Lambdas before and it's not fun...
nicois 5y ago
Hopefully ECS/fargate will also be supported soon. I tried shifting our CI workers to ARM but it resulted in not being able to use them to build ECS images, which was not great.
- tcoff91 5y ago
  can't you cross-compile x86_64 binaries on an arm machine?
  - remexre 5y ago
    Cross-compilation is iffy at best -- too many developers of build scripts don't think about it, and end up not supporting it (e.g. Makefile doing codegen using a binary built during the run for the target arch) or having subtle and mysterious bugs (./configure saves things for the build arch, not the target arch)
- slownews45 5y ago
  I'm using fargate very happily (after price reductions). For ubuntu / python-slim etc docker images - what changes are needed to let them target ARM (if any).
- Kipters 5y ago
  Ditto. My main project & CI are ready to be (cross) compiled to ARM, I'm just waiting for Graviton-powered Fargate.
braincode 5y ago
So good AWS listens to customers: https://twitter.com/braincode/status/1382940634093219840 https://twitter.com/braincode/status/1382940634093219840 Now, the SAM-CLI local developer experience is broken in Rust, those would be my next asks for AWS: https://github.com/umccr/s3-rust-noodles-bam/blob/s3-server/FLUKES.md https://github.com/umccr/s3-rust-noodles-bam/blob/s3-server/...
- dcu 5y ago
  yes, it's awesome. I also asked about this here: https://news.ycombinator.com/item?id=25267921 https://news.ycombinator.com/item?id=25267921 hopefully we'll see ML chips on Lambda soon too
DeepYogurt 5y ago
Doesn't exposing the hardware under the lambda kinda defeat the point of the lambda?
- koolba 5y ago
  Not at all. If you want to run assembly code that targets a specific architecture you need to know what that architecture is going to be. At it’s heart Lambda is just a Linux container running a CGI app.
  - aynyc 5y ago
    Why would you run assembly on lambda?
    - nightfly 5y ago
      SIMD is a thing
    - stevemk14ebr 5y ago
      Everything is assembly, that's how your CPU works. If you run C code, rust, go whatever you have to Target an architecture so the compiler can generate the correct assembly instructions. Even python or js interpreters must be recompiled per each architecture you wish to run those languages in. Lambda doesn't change that, as the comment above says its just a container
      - aynyc 5y ago
        The whole point of container is to abstract away the underlying OS & hardware. If you need CPU architecture specific code, running it on lambda seem a bit pointless to me.
        monocasa 5y ago
        Containers are more about abstracting away kernel state tables, but not deeper env like processor arch. You can see this in how there was a lot of work into deploying aarch64 containers over the past year to support M1 docker. And lambda is more about changing the process lifecycle from traditional always using resources. Sort of the inetd for hyperscalers.
TeeMassive 5y ago
Too bad this won't offset the 200%+ additional cost of choosing AWS over self-hosting.
- nightfly 5y ago
  There are time where AWS makes sense, just like there are times where self hosting makes sense.
  - TeeMassive 5y ago
    Well Reddit is barely profitable if not at all and it's not like they invested for innovation like Amazon is doing.
- tehbeard 5y ago
  The cost that with self hosting you instead sink into paying for SREs? Or getting paged at 2am?
- acdha 5y ago
  If you're not trolling, what function-as-a-service system are you self-hosting? How much time are you spending on ops?
- orf 5y ago
  10,000,000 300ms invocations with 1gb of memory is under 30 dollars. That's dirt cheap, over a month of CPU time that you can burst to tens of thousands of concurrent executions without caring. Not saying AWS doesn't come with a premium, but lambda isn't exactly the poster-child for expensive compute.
41209 5y ago
This is awesome, saving that much money can be great for startups. Oddly I can’t see lambdas being enough of a cost to justify it to hobbyist such as myself. But this is a great sign of things to come, so much energy is consumed by data centers. Then again, I wonder how much code will randomly break, plus AWS’s dependency management is literally bundle it up locally and upload a zip. What happens if a ARM package can’t be built locally.
- OJFord 5y ago
  > AWS’s dependency management is literally bundle it up locally and upload a zip. That's true and not great IMO either, I also wish the zip wasn't necessary - in the 'bootstrap' binary case just let me upload the binary! - but in case you're not aware of layers (just guessing from 'dependencies') it can be improved a bit: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-l... > What happens if a ARM package can’t be built locally. An Arm package can only not be built locally if you decline to use tools that'd allow you to do so?
  - HatchedLake721 5y ago
    Zip is not necessary anymore https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/ https://aws.amazon.com/blogs/aws/new-for-aws-lambda-containe...
    - OJFord 5y ago
      Ah I'd forgotten about that, thanks! I haven't created any since then other than via Serverless, which.. a large part of my distaste for it is the opaqueness and not knowing what it's doing, but I assume it uses zip method.
  - zxcvbn4038 5y ago
    I wish cloud formation allowed for more then 4096 bytes of inlined code. The underlying API allows for 50MB, sometimes it’s really nice to be able to deploy from cloudformation without all of the overhead for S3 or ECR.
bashtoni 5y ago
Sadly no support for Go right now.
- pugz 5y ago
  It works with Go: https://awsteele.com/blog/2021/09/29/graviton2-arm-comes-to-lambda.html https://awsteele.com/blog/2021/09/29/graviton2-arm-comes-to-...
  - bashtoni 5y ago
    Yep, it turns out the go1.x runtime is no longer the preferred way to using Lambda functions written in Go.
  - dalrympm 5y ago
    Thanks for this! I was hoping it was going to be this easy for a SAM based Go app but didn't have it in me to wade through the AWS docs. This post is $.
- OJFord 5y ago
  You can run an arbitrary binary. Keywords for searching docs are 'bootstrap' (the entrypoint name) and 'custom runtime'. In this way you can also run whatever version of the provided language runtimes you want, not limited to the official ones if you want newer python syntax or node stdlib features than are available, for example.
ThinkBeat 5y ago
There seems to have been a rain of ARM permutations lately most of them proprietary? Apple as their M1 and M2 (coming). MS has their Sq1, Sq2 Tesla has the FSD Chip AWS Graviton, Nvidia Snapdragon, Broadcom I am sure there are many more. I know Apple has not publicly documented their chips. I dont know for sure about the others. Will all of this make building compilers magnitudes more complex? If you have a codebase and you want it to optimize for M2, with all of its goodies like the GPUs and neural whatnot, then I need the same codebase to run as fast as possible on the Graviton cpus that sounds difficult? Maybe there is a subset of Opcodes shared across all or most of them but is that reasonable to limit compilers to them? I am sure I am just confused but it seems to me to be a bit of a mess. Can GCC be able to keep up?
- seabrookmx 5y ago
  That subset of opcodes is the ARM instruction set. So yes GCC will have no problem. Even the SIMD extensions are under ARM's purview. The neural processors, GPU's, TPU's etc. aren't part of the CPU anyways (they're part of the SoC). This fragmentation has some downsides but having a good compiler isn't one of them. It's more likely all the different ways these SoC's handle booting, connecting to peripherals, etc. This will create a huge burden on something like the Linux kernel if it attempts to support them all (for evidence, check out the recent work on Linux M1 support.. fascinating stuff).
- codemac 5y ago
  ARM itself is proprietary, including the ISA and several micro-architectures that implement the ISA. I don't think it's shocking that folks who licensed a proprietary ISA, and then paid a stiff license fee to implement their own microarch off of a proprietary reference microarch, would keep their chips proprietary. risc-v is the only hope in this arena, and luckily David Patterson is generally signal enough that something might be a good idea.
  - sneak 5y ago
    Is it possible to actually have a proprietary instruction set? Can't I design a new uarch that implements the ISA and is compatible like Cyrix and AMD did back in the day?
ThinkBeat 5y ago
Ideally on Serverless infrastructure the system would "move" jobs around so they run on the optimal platform for what they are doing continuously? (Based on price or based on raw compute etc) I know there are many reasons this will be difficult in real life, but it sounds like where it should be heading.
- rewma 5y ago
  > Ideally on Serverless infrastructure the system would "move" jobs around so they run on the optimal platform for what they are doing continuously? (Based on price or based on raw compute etc) AWS Lambda supports about half a dozen runtimes, some based on interpreted languages, and some based on compiled languages. Thus you can't simply move golang or rust code between completely different architectures and expect things to work. Also, one of AWS Lambda's usecases is having your runtime call precompiled binaries. I'd be pissed if my Lambda's ceased to work because AWS decided to run my trusty python-calling-C++ lib Lambdas in ARM just because they want to push people towards graviton.
estebarb 5y ago
I'm hoping that the next stop is Lambda on GPUs
JoshTriplett 5y ago
Awesome, I've wanted this for a while. Now if only I could run a custom kernel...
StratusBen 5y ago
I would love to know what the margin boost is to AWS as a result of Graviton. It's cheaper to customers but I have to imagine better margins to them - so kind of a win/win. It's just amazing how they're pushing it out to every AWS service at this point. We transitioned over our RDS instances to Graviton this last weekend and have a nice performance boost with ~10% savings. Disclosure as I'm Co-Founder and CEO of Vantage - but we'll likely be adding Graviton recommendations to our suite of cost saving recommendations on https://www.vantage.sh/ https://www.vantage.sh/ to give per resource views of potential savings (i.e. you can save X% on this Lambda function based upon the costs we saw by switching to Graviton, etc.)
- Twirrim 5y ago
  From folks I know at AWS, services are all being heavily pushed to use Graviton to run their services, to the point of having to justify why they couldn't. The cost savings internally are just as significant.
  - fridif 5y ago
    I am now going to go down the rabbit hole of arm specific bugs in the libraries I use because so far all of the applications I dev (arm ubuntu) have never encountered any bugs. Yay free money!
  - LukvonStrom 5y ago
    I think they will transition all services that do not expose infrastructure (think i.e. Cognito, Pinpoint, VPC, Api Gateway, CloudFormation, ...) ASAP.
- discodave 5y ago
  With Nitro they disclosed that every EC2 machine deployed in the last 5+ years has at least one ARM chip because each server has a nitro card. So reading the tea leaves... the first generation of EC2 ARM (a1) could have been almost as simple as two nitro cards hooked together. Graviton2 is likely a bigger chip than what they're sticking on Nitro cards, but if Graviton1 was just the overhead for each Intel machine...
tyingq 5y ago
I wonder if the ARM-powered Lambdas have a different vCPU scaling. Supposedly, you get between 2-6 vCPUs for x86-64 Lambdas, depending on how much memory you configure for the Lambda. But, the x86-64 vCPU is not a "real core". It's a hyperthread (SMT), or 1/2 a core. So I'm curious if the scale is still 2-6 vCPUs for Graviton, where a vCPU == A real core...since there's no SMT on Graviton.
- ryanmarsh 5y ago
  This is an important question and I hope they clarify it.
- mjb 5y ago
  Lambda does not use hyperthreads/SMT on either x86 or ARM.
  - tyingq 5y ago
    Interesting. Just tried running "cat /proc/cpuinfo" on an X86 Lambda, and I get: siblings : 2 cpu cores : 2 Which would imply no hyperthreads, to the degree that their hypervisor is telling the truth. So Lambdas are then one of the few places where an x86 core is a core. Thanks for the info.
    - my123 5y ago
      It’s too hard/not really feasible today to guarantee SMT safety between VMs. As Lambda is a many-VMs platform…
      - tyingq 5y ago
        Yes, I had assumed SMT because they say they give you between 2-6 vCPUs. I assumed that meant 2,4, or 6 hyperthreads...where they could keep you from sharing a core with neighbors. But, seems you get real cores.
    - don-code 5y ago
      I highly recommend Andy Jassy's infrastructure keynote from re:Invent 2020. Among many other things, he discusses why Amazon never gives you a single hyperthread (it's mostly about cache coherency, IIRC).
imwillofficial 5y ago
Working on an internal AWS app today. Joked about using graviton. Now this happened. I’m pretty sure I’m Rasputin.
brian_herman 5y ago
I want to see an arm based computer that you can put together that has pci-e slots. It would be nice to build a computer in the arm ecosystem.
- wang_li 5y ago
  Depending on what your idea of a computer is, the reference carrier board for the Raspberry Pi 4 compute module has a pcie x1 slot. The jetson family of developer kits have pcie connectivity. The 4GB Jetson Nano has an m.2 E key slot which has a pcie x1 lane, with the right adapter available from Amazon or alibaba or where ever, you can plug in a pcie card of your choice. The Xavier NX has two m.2 slots, which again, with adapters can connect to pcie cards. Though I’m not sure of the use case as the jetson line has fairly capable gpus built in.
  - my123 5y ago
    And the bigger AGX Xavier has a PCIe 4.0 x8 slot at the top, a PCIe 4.0 x4 M2 connector sandwiched in for NVMe and a x1 M2 one for a Wi-Fi adapter. Was among the earliest launched PCIe 4.0 platforms.
- CameronNemo 5y ago
  https://www.solid-run.com/arm-servers-networking-platforms/honeycomb-workstation/ https://www.solid-run.com/arm-servers-networking-platforms/h...
015a 5y ago
This is the biggest use-case for ARM on the short-term, and it isn't close. Lambdas are small, isolated, stateless, and highly abstract. All of these things mean, most users will probably just change one line, and immediately see a 20-40% cost reduction. Practically speaking, there is no reason for x86 lambdas to exist anymore (outside of, "we ship a go binary compiled to x86, need to change one line in the CI to compile to ARM, ok done"). More broadly speaking; as I type this on an M1 Macbook Air, a device with no fan, capable of matching the single-core performance of the brand new i7 11700KF and Ryzen 7 5900x, talking about how new AWS ARM processors will reduce lambda costs by up-to 40% while increasing performance by 20% over x86... we need consumer ARM chips, and we need broader non-Mac desktop application support. Seems like the writing is on the wall, and developers can either keep up with the curve or be left behind. Right now, very specifically: Looking at you Autodesk. We're coming up on a year of M1. Where's the non-Rosetta release of, say, Fusion 360?
- nikita2206 5y ago
  ARM is cheaper on AWS per the unit of work done, in the general case. But there should be no assumption that ARM is that snappy little brother that will launch your lambdas in no time while x86 is still starting up, this is the kind of idea that I get when reading the comments mentioning ARM. It won’t launch your virtualized environment that noticeably faster. It’ll just do more work for less money, and 40% is probably not the figure that you’ll see in real life.
  - philliphaydon 5y ago
    I don't think it will have any affect on Lambda's, but with TeamCity spinning up cloud agents. A t3.medium instance with Ubuntu takes about 1m 20s to begin a build from starting. The t4g.medium with Ubuntu takes around 37 seconds.
    - pletnes 5y ago
      The latency reduction might be worth more than the money saved on AWS, is what I’m thinking.
qbasic_forever 5y ago
This is awesome. In my benchmarking on some EC2 instances the Graviton 2 is a great processor for web workloads (frontend web apps). It really is 20% more performance for 20% less cost. Where in the world is GCE and Azure with ARM processors to compete? It's been how many years now and nothing to show yet? Customers aren't going to wait much longer...
- theevilsharpie 5y ago
  > Where in the world is GCE and Azure with ARM processors to compete? I can't speak for Azure, but Google Cloud looks like they've gone all-in with AMD's EPYC platform for their premium option, and offers the E2 instance family for those that want cheap compute and don't care what the underlying platform is. > Customers aren't going to wait much longer... I can't imagine anyone outside of hobbyists and tiny startups changing their entire hosting platform due to it not having an ARM offering.
  - klelatti 5y ago
    > I can't imagine anyone outside of hobbyists and tiny startups changing their entire hosting platform due to it not having an ARM offering. Don’t the economics work the other way though? The bigger the workload the bigger the saving and the easier it will be to justify moving. Probably a degree of flexibility in pricing to keep the biggest customers though!
- ksec 5y ago
  Microsoft is partnering ( as well as being an investor ) with Ampere. And Google is going with AMD EPYC offer vCPU as single CPU Core instead of a thread. I mean Amazon has been preparing for this since 2015 when they acquired Annapurna Labs. Considering TSMC somehow increased their planned capacity for 5nm by 50%. There may be a chance you will will see that going to Microsoft of Google coming in 2023.
  - ksec 5y ago
    Just happened Ampere has released [1] their "listed" price. [1] https://news.ycombinator.com/item?id=28706131 https://news.ycombinator.com/item?id=28706131
srwx 5y ago
Drats, not available on edge lambdas yet.
DarthNebo 5y ago
Waiting on GPU lambdas to be launched for a lot of ML projects!
- bvm 5y ago
  so...my use case for GPU lambdas is for inference. Say 500 inferences/day ~1 sec/inference, has to be real time as the output is user facing. Can they really spin up quick enough to actually respond in real time? I don't really want to have to keep them warm as I'll end up paying through the nose for it.
  - DarthNebo 5y ago
    For some of my use cases the real-time response isn't essential, just needs one of containers for training. I figured I can just go about ECS, but thought if these were wrapped as lambdas it would just be a matter of toggling the base-image/lambda type.
joelthelion 5y ago
Neat, but when can we get GPU lambdas?
vishnugupta 5y ago
I don't see an option to switch over existing lambdas. I hope they will soon enable this.
- ColdHeat 5y ago
  It seems doable. You can edit it under runtime settings: https://aws.amazon.com/blogs/aws/aws-lambda-functions-powered-by-aws-graviton2-processor-run-your-functions-on-arm-and-get-up-to-34-better-price-performance/ https://aws.amazon.com/blogs/aws/aws-lambda-functions-powere....
  - vishnugupta 5y ago
    Fantastic!! Thanks. Note to self: RTFM.
yewenjie 5y ago
I would really love to see no cold-starts in AWS lambdas next, as already offered by Cloudflare workers.
apatheticonion 5y ago
I have been using a Graviton 2 based EC2 to host some production web server and database workloads. I love the savings! Publishing my docker images to my registry as ARM images from my x86 based CI pipeline is a little weird though. Would be nice if docker baked in all the QEMU magic and let me use `docker build --target-arch arm64`.
- qbasic_forever 5y ago
  Use docker buildx, it's the Buildkit engine which fully supports cross platform builds. It's essentially just what you mention, add a CLI flag and it spit out an ARM image even on your Intel machine. See: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/ https://www.docker.com/blog/multi-arch-build-and-images-the-...
Aardvark62 5y ago
I changed a few very simple Lambda routines running Node.js v14 from "x86_64" to "arm64". Not complicated modules that run relatively quickly doing some very basic calculations. I was seeing double the running time. One routine that was reporting a billed duration of 2049 ms with "x86_64" reported 4243 ms with "arm64". Switched it back to "x86_64" and the runtime halved as expected. So at least with Node.js, I see no benefit with the arm64 architecture. YMMV.

270 points by orf 5y ago

This seems pretty huge ('19% better performance at 20% lower cost') considering many (most?) Lambda workloads will be arch-agnostic, not even needing updated scripts/installed libs etc. Then many more will only need a trivial re-compile. I'm sure they don't need me to tell them they'll be inundated with demand...

orf 5y ago
It will probably take a while to ramp up - it's explicitly opt-in and I expect there's a huge amount of existing code that creates lambdas without being able to specify the architecture. For starters example Terraform and Cloudformation don't support it yet. And if you're using a 3rd party layer you need to wait for them to publish a layer that explicitly supports that architecture.
- k__ 5y ago
  According to AWS, CFN rollout is happening right now.
  - orf 5y ago
    Sure but then you have the CDK, etc. I’d be interested to know how many people create lambdas in the console and not via some Other tooling, which needs to be updated to support this.
    - k__ 5y ago
      CDK support is out too.
    - edhzsz 5y ago
      Not saying that you are wrong, but CDK is ready now: https://github.com/aws/aws-cdk/releases/tag/v1.125.0 https://github.com/aws/aws-cdk/releases/tag/v1.125.0
- OJFord 5y ago
  Good point. I'd forgotten memory was the only existing toggle (i.e. no variation of x86 CPUs as on EC2, that 'g' or whatever could just be added to).
- smoldesu 5y ago
  Precisely. Once you start to consider how other ISAs factor into that (like perhaps RISC-V on the horizon) I really wonder if ARM has a chance to take off in the first place. A ~50% price/performance boost isn't negligible, but it's also not very compelling. Especially for larger companies who'd need to invest more time/money into transitioning their backend.
  - rewma 5y ago
    > A ~50% price/performance boost isn't negligible, but it's also not very compelling. My take is that a 50% discount in price/performance and a 20% discount in price per execution unit that's only a toggle flip away is something that everyone loves and is eager to jump onboard. The only exception is when your workload stubbornly stays within free tier limits, which in that case you have no compelling reason to do the work.
  - staticassertion 5y ago
    50% price/performance is very compelling. If you talk to a large company and say "you can cut your compute bill in half with a single re-deploy" they will be very happy.
- paranoidrobot 5y ago
  Terraform issue tracking support for architectures parameter: https://github.com/hashicorp/terraform-provider-aws/issues/21090 https://github.com/hashicorp/terraform-provider-aws/issues/2... There's a pull request that went up about 40 minutes prior to my comment. So it looks like it's coming, soon.
  - reilly3000 5y ago
    Pulumi has theirs up already: https://www.pulumi.com/blog/aws-lambda-functions-powered-by-graviton2/ https://www.pulumi.com/blog/aws-lambda-functions-powered-by-...
rewma 5y ago
> This seems pretty huge ('19% better performance at 20% lower cost') I believe the performance boost will not be meaningful and the only important factor is cost per GB*s, given that most Lambda workloads are IO bound or simple event handlers that barely take 1s to run anyway. Nevertheless this is exciting stuff, specially given that NodeJS might be lambda's most popular runtime and thus we might only be a toggle flip away from a 20% discount.
- ryanmarsh 5y ago
  My Lambda workloads are CPU bound so this is great but I get it, most spend their time waiting for IO.
- diskzero 5y ago
  I wrote a lambda that encoded a timeline comprised of audio, video and GLSL shaders using SwiftShader and ffmpeg. Time to give it try again!
- bonobocop 5y ago
  Apparently Python rules the roost for Lambda: https://mobile.twitter.com/julian_wood/status/1442775542374277120/photo/1 https://mobile.twitter.com/julian_wood/status/14427755423742... But NodeJS is second!
  - zxcvbn4038 5y ago
    Golang is the top dog for Lambda IMO. The AWS Lambda SDK for Golang includes the entire lambda runtime so you can deploy a single 20mb binary w/ an Alpine linux base (speaking of the ECR backed lambdas) and that’s it. With NodeJS you either have to use the AWS provided centos based images which weigh in at several hundred MB or do some multistage magic to build a minimal Alpine image including the lambda runtime and AWS SDK which altogether still weighs in almost a hundred MB.
    - bonobocop 5y ago
      TBH, as the Dockerfile based Lambda layers have a 20GB file size limit, I'm not sure the size matters as much. It's certainly easier to figure out what's going on in a smaller container though! I've had to debug some nasty situation with layers in Python Lambdas before and it's not fun...

nicois 5y ago

Hopefully ECS/fargate will also be supported soon. I tried shifting our CI workers to ARM but it resulted in not being able to use them to build ECS images, which was not great.

tcoff91 5y ago
can't you cross-compile x86_64 binaries on an arm machine?
- remexre 5y ago
  Cross-compilation is iffy at best -- too many developers of build scripts don't think about it, and end up not supporting it (e.g. Makefile doing codegen using a binary built during the run for the target arch) or having subtle and mysterious bugs (./configure saves things for the build arch, not the target arch)
slownews45 5y ago
I'm using fargate very happily (after price reductions). For ubuntu / python-slim etc docker images - what changes are needed to let them target ARM (if any).
Kipters 5y ago
Ditto. My main project & CI are ready to be (cross) compiled to ARM, I'm just waiting for Graviton-powered Fargate.

braincode 5y ago

So good AWS listens to customers: https://twitter.com/braincode/status/1382940634093219840 https://twitter.com/braincode/status/1382940634093219840 Now, the SAM-CLI local developer experience is broken in Rust, those would be my next asks for AWS: https://github.com/umccr/s3-rust-noodles-bam/blob/s3-server/FLUKES.md https://github.com/umccr/s3-rust-noodles-bam/blob/s3-server/...

dcu 5y ago
yes, it's awesome. I also asked about this here: https://news.ycombinator.com/item?id=25267921 https://news.ycombinator.com/item?id=25267921 hopefully we'll see ML chips on Lambda soon too

DeepYogurt 5y ago

Doesn't exposing the hardware under the lambda kinda defeat the point of the lambda?

koolba 5y ago
Not at all. If you want to run assembly code that targets a specific architecture you need to know what that architecture is going to be. At it’s heart Lambda is just a Linux container running a CGI app.
- aynyc 5y ago
  Why would you run assembly on lambda?
  - nightfly 5y ago
    SIMD is a thing
  - stevemk14ebr 5y ago
    Everything is assembly, that's how your CPU works. If you run C code, rust, go whatever you have to Target an architecture so the compiler can generate the correct assembly instructions. Even python or js interpreters must be recompiled per each architecture you wish to run those languages in. Lambda doesn't change that, as the comment above says its just a container
    - aynyc 5y ago
      The whole point of container is to abstract away the underlying OS & hardware. If you need CPU architecture specific code, running it on lambda seem a bit pointless to me.
      - monocasa 5y ago
        Containers are more about abstracting away kernel state tables, but not deeper env like processor arch. You can see this in how there was a lot of work into deploying aarch64 containers over the past year to support M1 docker. And lambda is more about changing the process lifecycle from traditional always using resources. Sort of the inetd for hyperscalers.

TeeMassive 5y ago

Too bad this won't offset the 200%+ additional cost of choosing AWS over self-hosting.

nightfly 5y ago
There are time where AWS makes sense, just like there are times where self hosting makes sense.
- TeeMassive 5y ago
  Well Reddit is barely profitable if not at all and it's not like they invested for innovation like Amazon is doing.
tehbeard 5y ago
The cost that with self hosting you instead sink into paying for SREs? Or getting paged at 2am?
acdha 5y ago
If you're not trolling, what function-as-a-service system are you self-hosting? How much time are you spending on ops?
orf 5y ago
10,000,000 300ms invocations with 1gb of memory is under 30 dollars. That's dirt cheap, over a month of CPU time that you can burst to tens of thousands of concurrent executions without caring. Not saying AWS doesn't come with a premium, but lambda isn't exactly the poster-child for expensive compute.

41209 5y ago

This is awesome, saving that much money can be great for startups. Oddly I can’t see lambdas being enough of a cost to justify it to hobbyist such as myself. But this is a great sign of things to come, so much energy is consumed by data centers. Then again, I wonder how much code will randomly break, plus AWS’s dependency management is literally bundle it up locally and upload a zip. What happens if a ARM package can’t be built locally.

OJFord 5y ago
> AWS’s dependency management is literally bundle it up locally and upload a zip. That's true and not great IMO either, I also wish the zip wasn't necessary - in the 'bootstrap' binary case just let me upload the binary! - but in case you're not aware of layers (just guessing from 'dependencies') it can be improved a bit: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-l... > What happens if a ARM package can’t be built locally. An Arm package can only not be built locally if you decline to use tools that'd allow you to do so?
- HatchedLake721 5y ago
  Zip is not necessary anymore https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/ https://aws.amazon.com/blogs/aws/new-for-aws-lambda-containe...
  - OJFord 5y ago
    Ah I'd forgotten about that, thanks! I haven't created any since then other than via Serverless, which.. a large part of my distaste for it is the opaqueness and not knowing what it's doing, but I assume it uses zip method.
- zxcvbn4038 5y ago
  I wish cloud formation allowed for more then 4096 bytes of inlined code. The underlying API allows for 50MB, sometimes it’s really nice to be able to deploy from cloudformation without all of the overhead for S3 or ECR.

bashtoni 5y ago

Sadly no support for Go right now.

pugz 5y ago
It works with Go: https://awsteele.com/blog/2021/09/29/graviton2-arm-comes-to-lambda.html https://awsteele.com/blog/2021/09/29/graviton2-arm-comes-to-...
- bashtoni 5y ago
  Yep, it turns out the go1.x runtime is no longer the preferred way to using Lambda functions written in Go.
- dalrympm 5y ago
  Thanks for this! I was hoping it was going to be this easy for a SAM based Go app but didn't have it in me to wade through the AWS docs. This post is $.
OJFord 5y ago
You can run an arbitrary binary. Keywords for searching docs are 'bootstrap' (the entrypoint name) and 'custom runtime'. In this way you can also run whatever version of the provided language runtimes you want, not limited to the official ones if you want newer python syntax or node stdlib features than are available, for example.

ThinkBeat 5y ago

There seems to have been a rain of ARM permutations lately most of them proprietary? Apple as their M1 and M2 (coming). MS has their Sq1, Sq2 Tesla has the FSD Chip AWS Graviton, Nvidia Snapdragon, Broadcom I am sure there are many more. I know Apple has not publicly documented their chips. I dont know for sure about the others. Will all of this make building compilers magnitudes more complex? If you have a codebase and you want it to optimize for M2, with all of its goodies like the GPUs and neural whatnot, then I need the same codebase to run as fast as possible on the Graviton cpus that sounds difficult? Maybe there is a subset of Opcodes shared across all or most of them but is that reasonable to limit compilers to them? I am sure I am just confused but it seems to me to be a bit of a mess. Can GCC be able to keep up?

seabrookmx 5y ago
That subset of opcodes is the ARM instruction set. So yes GCC will have no problem. Even the SIMD extensions are under ARM's purview. The neural processors, GPU's, TPU's etc. aren't part of the CPU anyways (they're part of the SoC). This fragmentation has some downsides but having a good compiler isn't one of them. It's more likely all the different ways these SoC's handle booting, connecting to peripherals, etc. This will create a huge burden on something like the Linux kernel if it attempts to support them all (for evidence, check out the recent work on Linux M1 support.. fascinating stuff).
codemac 5y ago
ARM itself is proprietary, including the ISA and several micro-architectures that implement the ISA. I don't think it's shocking that folks who licensed a proprietary ISA, and then paid a stiff license fee to implement their own microarch off of a proprietary reference microarch, would keep their chips proprietary. risc-v is the only hope in this arena, and luckily David Patterson is generally signal enough that something might be a good idea.
- sneak 5y ago
  Is it possible to actually have a proprietary instruction set? Can't I design a new uarch that implements the ISA and is compatible like Cyrix and AMD did back in the day?

ThinkBeat 5y ago

Ideally on Serverless infrastructure the system would "move" jobs around so they run on the optimal platform for what they are doing continuously? (Based on price or based on raw compute etc) I know there are many reasons this will be difficult in real life, but it sounds like where it should be heading.

rewma 5y ago
> Ideally on Serverless infrastructure the system would "move" jobs around so they run on the optimal platform for what they are doing continuously? (Based on price or based on raw compute etc) AWS Lambda supports about half a dozen runtimes, some based on interpreted languages, and some based on compiled languages. Thus you can't simply move golang or rust code between completely different architectures and expect things to work. Also, one of AWS Lambda's usecases is having your runtime call precompiled binaries. I'd be pissed if my Lambda's ceased to work because AWS decided to run my trusty python-calling-C++ lib Lambdas in ARM just because they want to push people towards graviton.

estebarb 5y ago

I'm hoping that the next stop is Lambda on GPUs

JoshTriplett 5y ago

Awesome, I've wanted this for a while. Now if only I could run a custom kernel...

StratusBen 5y ago

I would love to know what the margin boost is to AWS as a result of Graviton. It's cheaper to customers but I have to imagine better margins to them - so kind of a win/win. It's just amazing how they're pushing it out to every AWS service at this point. We transitioned over our RDS instances to Graviton this last weekend and have a nice performance boost with ~10% savings. Disclosure as I'm Co-Founder and CEO of Vantage - but we'll likely be adding Graviton recommendations to our suite of cost saving recommendations on https://www.vantage.sh/ https://www.vantage.sh/ to give per resource views of potential savings (i.e. you can save X% on this Lambda function based upon the costs we saw by switching to Graviton, etc.)

Twirrim 5y ago
From folks I know at AWS, services are all being heavily pushed to use Graviton to run their services, to the point of having to justify why they couldn't. The cost savings internally are just as significant.
- fridif 5y ago
  I am now going to go down the rabbit hole of arm specific bugs in the libraries I use because so far all of the applications I dev (arm ubuntu) have never encountered any bugs. Yay free money!
- LukvonStrom 5y ago
  I think they will transition all services that do not expose infrastructure (think i.e. Cognito, Pinpoint, VPC, Api Gateway, CloudFormation, ...) ASAP.
discodave 5y ago
With Nitro they disclosed that every EC2 machine deployed in the last 5+ years has at least one ARM chip because each server has a nitro card. So reading the tea leaves... the first generation of EC2 ARM (a1) could have been almost as simple as two nitro cards hooked together. Graviton2 is likely a bigger chip than what they're sticking on Nitro cards, but if Graviton1 was just the overhead for each Intel machine...

tyingq 5y ago

I wonder if the ARM-powered Lambdas have a different vCPU scaling. Supposedly, you get between 2-6 vCPUs for x86-64 Lambdas, depending on how much memory you configure for the Lambda. But, the x86-64 vCPU is not a "real core". It's a hyperthread (SMT), or 1/2 a core. So I'm curious if the scale is still 2-6 vCPUs for Graviton, where a vCPU == A real core...since there's no SMT on Graviton.

ryanmarsh 5y ago
This is an important question and I hope they clarify it.
mjb 5y ago
Lambda does not use hyperthreads/SMT on either x86 or ARM.
- tyingq 5y ago
  Interesting. Just tried running "cat /proc/cpuinfo" on an X86 Lambda, and I get: siblings : 2 cpu cores : 2 Which would imply no hyperthreads, to the degree that their hypervisor is telling the truth. So Lambdas are then one of the few places where an x86 core is a core. Thanks for the info.
  - my123 5y ago
    It’s too hard/not really feasible today to guarantee SMT safety between VMs. As Lambda is a many-VMs platform…
    - tyingq 5y ago
      Yes, I had assumed SMT because they say they give you between 2-6 vCPUs. I assumed that meant 2,4, or 6 hyperthreads...where they could keep you from sharing a core with neighbors. But, seems you get real cores.
  - don-code 5y ago
    I highly recommend Andy Jassy's infrastructure keynote from re:Invent 2020. Among many other things, he discusses why Amazon never gives you a single hyperthread (it's mostly about cache coherency, IIRC).

imwillofficial 5y ago

Working on an internal AWS app today. Joked about using graviton. Now this happened. I’m pretty sure I’m Rasputin.

brian_herman 5y ago

I want to see an arm based computer that you can put together that has pci-e slots. It would be nice to build a computer in the arm ecosystem.

wang_li 5y ago
Depending on what your idea of a computer is, the reference carrier board for the Raspberry Pi 4 compute module has a pcie x1 slot. The jetson family of developer kits have pcie connectivity. The 4GB Jetson Nano has an m.2 E key slot which has a pcie x1 lane, with the right adapter available from Amazon or alibaba or where ever, you can plug in a pcie card of your choice. The Xavier NX has two m.2 slots, which again, with adapters can connect to pcie cards. Though I’m not sure of the use case as the jetson line has fairly capable gpus built in.
- my123 5y ago
  And the bigger AGX Xavier has a PCIe 4.0 x8 slot at the top, a PCIe 4.0 x4 M2 connector sandwiched in for NVMe and a x1 M2 one for a Wi-Fi adapter. Was among the earliest launched PCIe 4.0 platforms.
CameronNemo 5y ago
https://www.solid-run.com/arm-servers-networking-platforms/honeycomb-workstation/ https://www.solid-run.com/arm-servers-networking-platforms/h...

015a 5y ago

This is the biggest use-case for ARM on the short-term, and it isn't close. Lambdas are small, isolated, stateless, and highly abstract. All of these things mean, most users will probably just change one line, and immediately see a 20-40% cost reduction. Practically speaking, there is no reason for x86 lambdas to exist anymore (outside of, "we ship a go binary compiled to x86, need to change one line in the CI to compile to ARM, ok done"). More broadly speaking; as I type this on an M1 Macbook Air, a device with no fan, capable of matching the single-core performance of the brand new i7 11700KF and Ryzen 7 5900x, talking about how new AWS ARM processors will reduce lambda costs by up-to 40% while increasing performance by 20% over x86... we need consumer ARM chips, and we need broader non-Mac desktop application support. Seems like the writing is on the wall, and developers can either keep up with the curve or be left behind. Right now, very specifically: Looking at you Autodesk. We're coming up on a year of M1. Where's the non-Rosetta release of, say, Fusion 360?

nikita2206 5y ago
ARM is cheaper on AWS per the unit of work done, in the general case. But there should be no assumption that ARM is that snappy little brother that will launch your lambdas in no time while x86 is still starting up, this is the kind of idea that I get when reading the comments mentioning ARM. It won’t launch your virtualized environment that noticeably faster. It’ll just do more work for less money, and 40% is probably not the figure that you’ll see in real life.
- philliphaydon 5y ago
  I don't think it will have any affect on Lambda's, but with TeamCity spinning up cloud agents. A t3.medium instance with Ubuntu takes about 1m 20s to begin a build from starting. The t4g.medium with Ubuntu takes around 37 seconds.
  - pletnes 5y ago
    The latency reduction might be worth more than the money saved on AWS, is what I’m thinking.

qbasic_forever 5y ago

This is awesome. In my benchmarking on some EC2 instances the Graviton 2 is a great processor for web workloads (frontend web apps). It really is 20% more performance for 20% less cost. Where in the world is GCE and Azure with ARM processors to compete? It's been how many years now and nothing to show yet? Customers aren't going to wait much longer...

theevilsharpie 5y ago
> Where in the world is GCE and Azure with ARM processors to compete? I can't speak for Azure, but Google Cloud looks like they've gone all-in with AMD's EPYC platform for their premium option, and offers the E2 instance family for those that want cheap compute and don't care what the underlying platform is. > Customers aren't going to wait much longer... I can't imagine anyone outside of hobbyists and tiny startups changing their entire hosting platform due to it not having an ARM offering.
- klelatti 5y ago
  > I can't imagine anyone outside of hobbyists and tiny startups changing their entire hosting platform due to it not having an ARM offering. Don’t the economics work the other way though? The bigger the workload the bigger the saving and the easier it will be to justify moving. Probably a degree of flexibility in pricing to keep the biggest customers though!
ksec 5y ago
Microsoft is partnering ( as well as being an investor ) with Ampere. And Google is going with AMD EPYC offer vCPU as single CPU Core instead of a thread. I mean Amazon has been preparing for this since 2015 when they acquired Annapurna Labs. Considering TSMC somehow increased their planned capacity for 5nm by 50%. There may be a chance you will will see that going to Microsoft of Google coming in 2023.
- ksec 5y ago
  Just happened Ampere has released [1] their "listed" price. [1] https://news.ycombinator.com/item?id=28706131 https://news.ycombinator.com/item?id=28706131

srwx 5y ago

Drats, not available on edge lambdas yet.

DarthNebo 5y ago

Waiting on GPU lambdas to be launched for a lot of ML projects!

bvm 5y ago
so...my use case for GPU lambdas is for inference. Say 500 inferences/day ~1 sec/inference, has to be real time as the output is user facing. Can they really spin up quick enough to actually respond in real time? I don't really want to have to keep them warm as I'll end up paying through the nose for it.
- DarthNebo 5y ago
  For some of my use cases the real-time response isn't essential, just needs one of containers for training. I figured I can just go about ECS, but thought if these were wrapped as lambdas it would just be a matter of toggling the base-image/lambda type.

joelthelion 5y ago

Neat, but when can we get GPU lambdas?

vishnugupta 5y ago

I don't see an option to switch over existing lambdas. I hope they will soon enable this.

ColdHeat 5y ago
It seems doable. You can edit it under runtime settings: https://aws.amazon.com/blogs/aws/aws-lambda-functions-powered-by-aws-graviton2-processor-run-your-functions-on-arm-and-get-up-to-34-better-price-performance/ https://aws.amazon.com/blogs/aws/aws-lambda-functions-powere....
- vishnugupta 5y ago
  Fantastic!! Thanks. Note to self: RTFM.

yewenjie 5y ago

I would really love to see no cold-starts in AWS lambdas next, as already offered by Cloudflare workers.

apatheticonion 5y ago

I have been using a Graviton 2 based EC2 to host some production web server and database workloads. I love the savings! Publishing my docker images to my registry as ARM images from my x86 based CI pipeline is a little weird though. Would be nice if docker baked in all the QEMU magic and let me use `docker build --target-arch arm64`.

qbasic_forever 5y ago
Use docker buildx, it's the Buildkit engine which fully supports cross platform builds. It's essentially just what you mention, add a CLI flag and it spit out an ARM image even on your Intel machine. See: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/ https://www.docker.com/blog/multi-arch-build-and-images-the-...

Aardvark62 5y ago

I changed a few very simple Lambda routines running Node.js v14 from "x86_64" to "arm64". Not complicated modules that run relatively quickly doing some very basic calculations. I was seeing double the running time. One routine that was reporting a billed duration of 2049 ms with "x86_64" reported 4243 ms with "arm64". Switched it back to "x86_64" and the runtime halved as expected. So at least with Node.js, I see no benefit with the arm64 architecture. YMMV.