6 ms·
AWS Is Asleep at the Lambda Wheel
- tiew9Vii 4y ago> We’ve been able to use custom runtimes ourselves for a while, and via the magic of Docker container images we can basically make Lambda run whatever the heck we want it to. Unfortunately, building our own Python runtimes at home isn’t sustainable as a customer Custom runtimes with Docker images solves so many issues with Lambda. Artifact management for free and the ability to run whatever language/applicaiton you want on Lambda. It works well, it's simple, it's portable. I can run my containers I deploy to Lambda on Lambda, Kubernetes, my local machine, pretty much anywhere if I abstract the AWS services i use. Building a Python Docker image isn't sustainable as a customer? I don't understand that as most people package apps as containers already as a standard unit of deployment.Most large companies maintain their own base images. It looks like the only in-convenience for Python is doing `pip install awslambdaric` With custom runtime Docker images I think AWS was smart, removing AWS maintenance as the limiting factor to let users have pretty much full control of the runtime, fairly trivially with tools they already know.
- ericpauley 4y agoTotally agree. I wouldn't be surprised if Amazon is viewing the premade runtimes as a legacy option, with containers being the default (and eventually only) method in the future.
- deleted 4y ago[deleted]
- __turbobrew__ 4y agoThis is the way. The only drawback of docker on lambda is slower cold start times (which may have changed since I built on lambda)
- JPKab 4y agoThis is a great article, and sums up my team's frustrations with AWS Lambda and the AWS engineering leadership in general. I worked in the past under a slew of former AWS product managers, and I know from them how the AWS roadmap works: 1.) Real, crucial customer feedback is incorporated into roadmap, usually for the core services that are already used extensively like Lambda. 2.) Executives have new, novel, sexy flavor of the month projects like "we need to catch up with that ChatGPT thing my kid was showing the Au Pair the other day!" 3.) Product Management leaders are incentivized to prioritize said new/novel projects, and proceed to do a worker crushing sprint to build what any sane person would call a PoC. 4.) PoC is barely tested and pushed out as a half-baked garbage product. Garbage product languishes. (See AWS Data Science Studio for a great example of giant, useless POS) Meanwhile, developers are siphoned from real projects like, gee, I don't know, getting Python 3.10 out on Lambda? Or how about when you launch a sagemaker notebook instance you still top out at Python 3.8???? And don't get me started on recent versions of CUDA being available on GPU instances. My team is relegated to docker containers deployed to lambda due to the antiquated, ridiculous limits on layer sizes for any of our ML oriented lambdas. I have a buddy who works at AWS and talks about how utterly inefficient they are due to horrific leadership. At this point, you have a choice between a few highly dysfunctional cloud vendors. Google just SUCKS, Azure is a blatant financial rip off, and AWS is always developing new features nobody wants while ignoring the ones we do.
- ndneighbor 4y agoBefore my current role, I had to juggle roadmaps and integration partnerships with the Big Clouds. It's insane how much they ship their org chart but also their internal politics? I am convinced that some of the teams there are really trying to angle for releases that ship for PR rather than the feedback. (Although some AWS PMs really are trying despite everything else.)
- wallace_g 4y agoWhat are your issues with Google cloud?
- JPKab 4y ago
- gillesjacobs 4y ago+1 ran into the outdated Python Lambda runtime too. Nothing to add, OP said it all.
- dax77 4y agoPython 3.11 is ~25% faster… Lambda is billed based on compute duration… if you are AWS PM Lead, it’s a challenging eng investment to justify.
- nwah1 4y agoJevons Paradox, though
- scarface74 4y agoDisclaimer: I work at AWS in Professional Services. I can honestly say that I have never been pressured to create a more expensive solution for a customer. Everyone from sales, to account managers, to TAMs are are always trying to figure out how to help customer spend less money on their existing workloads. Before anyone says I’m drinking the kool aid, I’m 49 years old and this is my 8th job since 1996. I’m the most cynical person you can imagine. Now there is a difference between wanting customers to spend less on existing workloads and helping them optimize and trying to convince customers to spend more money by bringing in new work.
- Terretta 4y agoI've been an AWS customer since the thing before S3 I think was called SQS. And whether I've been small, or massive, ProServ and TAMs always always did what you're saying. When we were big enough, you'd even (no joke) get in an actual automobile and drive to us, to yell at us about wasting money. It was great you did that.
- 8note 4y agoWith dogfooding, it's a pretty easy argument -non-aws Amazon engineer It definitely feels like Amazon wants everyone to use lambdas for everything in the non-aws world (opinions my own)
- PartiallyTyped 4y agoIt's actually pretty easy: > Customer Obsession: Leaders start with the customer and work backwards. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers. >Frugality: Accomplish more with less. Constraints breed resourcefulness, self-sufficiency, and invention. There are no extra points for growing headcount, budget size, or fixed expense. > Deliver Results: Leaders focus on the key inputs for their business and deliver them with the right quality and in a timely fashion. Despite setbacks, they rise to the occasion and never settle. https://amazon.jobs/content/en/our-workplace/leadership-principles https://amazon.jobs/content/en/our-workplace/leadership-prin...
- Waterluvian 4y agoSeems like a pretty strong case against “serverless.” There’s always a server. You just have zero control over this one. So you’re stuck defactoring code to run an older version of Python because you can’t upgrade Python.
- MaxDPS 4y agoIt would be a case against server less if Lambda were the only option. You’re always free to setup your own container which would still give you control over all of that.
- BFLpL0QNek 4y agoWith managed runtimes it used to be you had no control. It was an opaque black box with complete vendor lock-in. Now it supports Containers it’s a place you can run any compute you can think of as long as it finishes in 15 minutes, uses less than 10gb of memory and you want to trigger it using http, direct lambda invocation or from one of the various aws event sources such as s3, sqs, sns. I used to avoid Lambda as it was an opaque black box. Now it’s a generic compute platform for short lived jobs using container support.
- klysm 4y agoI feel like AWS is one of those things where you gotta stick to the good parts. A lot of stuff is half baked but there’s a decent slice of it that works quite well
- onphonenow 4y agoProps to RDS team - they keep those databases updated. Anyone here who can figure out how far behind they are in months cs released db versions - for my workloads I’ve been impressed
- heebyjeeby 4y agoPossible explanation: lambda just isn’t profitable enough. Many workloads that are sporadic / rarely invoked gravitate to lambda. Id wager that the cost it takes to build and deploy a lambda and manage its logs and security and storage are >> the revenue from invoking it 24 times per day.
- joshstrange 4y agoRan into the same issues with Node. 18 only took 7 months for them to support but 16 took almost a full year. For a paid service this is simply ridiculous. Maybe they can't be ready day 1 (though I sort of think they should have people following node/etc progress and be testing leading up to a release) but 11 months (heck, even 7) is way too long to wait. This was made much more painful by the fact that M1 macs didn't really play super nice with node14 for whatever reason so my choices were 12 or nothing for a very long time.