9 ms·
Azure Functions vs. AWS Lambda – Scaling Face Off
- Axiverse 9y agoDo you know how Google Cloud Functions compares with AWS and Azure?
- spyspy 9y agoIf you're going to be handling http requests it's almost a given you'll be using AppEngine rather than Cloud Functions. Still "serverless" but more tools and languages to choose from.
- the_common_man 9y agoAppEngine is more like a PaaS and not like 'serverless'
- seangrogg 9y agoThis. AppEngine assumes an "always on" mentality and is far from "pay for use" unless your "use" is 24/7.
- octalmage 9y agoAppEngine standard can actually scale down to 0 instances when not in use to save money. Still very different from a serverless architecture though.
- nothrabannosir 9y agoNot the flexible environment iirc. Which is basically the reasonable env to expect, now. Standard env has some aging issues that keep getting worse. most notably jvm 7, if that’s your poison, but just all round crippled env lacking standard libs and with lock in substitutions. As far as I remember, that is..
- iangudger 9y agoJava 8 is now supported with the full standard library: https://cloud.google.com/appengine/docs/standard/java/runtime-java8 https://cloud.google.com/appengine/docs/standard/java/runtim...
- dragonwriter 9y ago> If you're going to be handling http requests it's almost a given you'll be using AppEngine rather than Cloud Functions Why? Cloud Functions support HTTP. I mean, if you can stay within AppEngine’s free tier quotas, there's a clear reason to use it, but otherwise it would seem to be a matter of weighing pros and cons, with HTTP alone not deciding that you should use AE over CF.
- vira28 9y agoAre you kidding? You know how much you have to pay for AppEngine's right?!!
- jpatokal 9y agoNothing, if it's a small app? https://cloud.google.com/free/docs/always-free-usage-limits#gae_name https://cloud.google.com/free/docs/always-free-usage-limits#... Disclaimer: I work at GCP.
- seangrogg 9y agoGCF still suffers from JS lock-in, as they currently only support Node, thus you can't make a .NET comparison in the first place. That being said, of the various offers I really like the UI and the like. Disclaimer: I am at Google, but not on GCF.
- thesandlord 9y agoYou technically can hack it to work: https://medium.com/@salmaan.rashid/running-net-on-google-cloud-functions-977e1a1c489 https://medium.com/@salmaan.rashid/running-net-on-google-clo... Native support for more languages is coming.
- tnolet 9y agoWow, that is pretty substantial difference between AWS and Azure. I’m pretty confident Microsoft is hard at work getting these numbers better, but it shows just how lonely AWS is at the top. I’m pretty deep into AWS-land but I’m rooting for the boys & girls in Redmond; more competition == better service.
- QuinnyPig 9y agoConcur; despite being so deep in AWS I've drowned here, competition makes all of the offerings in this space better. It'd be nice to have a migration strategy in my back pocket if I needed it; today that's a bit of a pipe dream.
- dmitriid 9y agoNever ever use averages in your stats. Use medians and percentiles. Averages are a meaningless stat especially if you benchmark.
- mr_toad 9y agoNever use the best linear unbiased estimator? That's a strong statement.
- dmitriid 9y agoIt’s not the best. There’s a Russian joke that goes “One person is cooling in the morgue, the other is running a high fever. The average temperature in the hospital is 36.6 C though” Averages are very susceptible to outliers and don’t give a proper insight into your data. What you really want is: how did half of my requests perform, 90% of requests perform, 95% of requests perform, 99% of requests perform
- screaminghawk 9y agoMedian is a type of average. You are confusing "average" with "mean". To be fair, it's not clear which form of average is used in the article and your assumption that it is the mean is likely correct.
- blueline 9y agoI have never seen median referred to as a "type of average" in any context. Is there any foundation to this?
- viggity 9y agothis is absolutely not a function of Azure vs AWS, the perf diff is completely between .Net Framework (which was used on Azure) and .Net Core (which was used on AWS). This compares Apples and Watermelons. https://www.ageofascent.com/2016/02/18/asp-net-core-exeeds-1-15-million-requests-12-6-gbps/ https://www.ageofascent.com/2016/02/18/asp-net-core-exeeds-1... http://codinggorilla.domemtech.com/?p=1499 http://codinggorilla.domemtech.com/?p=1499
- mnglkhn2 9y agoIs .Net Core available on Azure functions? If not, then the comparison is an accurate snapshot of the current status.
- starik36 9y agoIn preview. https://azure.microsoft.com/en-us/roadmap/azure-functions-support-for-net-core/ https://azure.microsoft.com/en-us/roadmap/azure-functions-su...
- jamesdrandall 9y agoAnd the performance is dreadful: https://www.azurefromthetrenches.com/azure-functions-v2-preview-performance-issues-net-core-standard/ https://www.azurefromthetrenches.com/azure-functions-v2-prev...
- karavelov 9y agoHe tried Azure functions with C# on .Net Core with even worse results: https://www.azurefromthetrenches.com/azure-functions-v2-preview-performance-issues-net-core-standard/ https://www.azurefromthetrenches.com/azure-functions-v2-prev... https://www.azurefromthetrenches.com/serverless-blog-christmas-2017-project/ https://www.azurefromthetrenches.com/serverless-blog-christm...
- taspeotis 9y ago“On Azure I’m using .NET 4.6 on the v1 runtime while on AWS I’m using the same code running under .NET Core 1.0.“ .NET Core has way less baggage than .NET 4.6 [1] so you’d be expecting AWS to come out ahead. It looks like Azure functions supports .NET Core now [2]. (Edit: just noticed the author is aware of this [3].) Of course it might also be Blob storage having higher latency than S3. Having used Azure Functions before I agree with the article’s criticisms about scaling. [1] https://www.ageofascent.com/2016/02/18/asp-net-core-exeeds-1-15-million-requests-12-6-gbps/ https://www.ageofascent.com/2016/02/18/asp-net-core-exeeds-1... Mind this article is talking about comparing ASP.NET with ASP.NET Core, and .NET with .NET Core, not just .NET Core’s performance. [2] https://github.com/Azure/Azure-Functions/issues/98 https://github.com/Azure/Azure-Functions/issues/98 [3] https://www.azurefromthetrenches.com/azure-functions-v2-preview-performance-issues-net-core-standard/ https://www.azurefromthetrenches.com/azure-functions-v2-prev...
- karavelov 9y agoHe also compares AWS Lambda with Azure blob storage backend and it still lot faster - just 25ms added latency to cross internet from AWS to Azure DC.
- deleted 9y ago[deleted]
- oceanswave 9y agoIt’s also difficult to run true x64 bit workloads on azure functions as it invokes the x86 version of the dotnet executable, regardless if the application is set to x64
- berns 9y agoThe fact that Azure provides no SLA for functions apps running under consumption plans is clear evidence that it's not ready for production use.
- QuinnyPig 9y agoTo be fair, there's no formal Lambda SLA either.
- deleted 9y ago[deleted]
- carlsborg 9y agoMy money is going to the innovators and first movers, regardless.
- p0rkbelly 9y agoThat mindset doesn't change these numbers then seeing that AWS released Lambda in 2014...years ahead.
- carlsborg 9y agoHave to agree with that part, and am in no way taking away from the commendable engineering on the Azure side. But as a developer, it makes more sense for me to invest time and effort getting good at the platform that innovates faster - and the pace at which AWS has been getting out new functionality in the platform has been breathtaking.
- appdrag 9y agoThe biggest issue with Azure functions is the pricing model where you have to provision and pay for servers to execute your functions... How is that serverless?
- Operyl 9y agoI mean, I guess some consider that you don’t have to maintain the servers and therefore it’s transparent to you and “serverless”.
- intrasight 9y agoYou don't have to provision servers with Azure Functions - but you optionally can.
- intrasight 9y agoThis face-off will be a lot more meaningful in a couple months when we can have both serverless platforms running on .Net Core 2.0. I do look forward to reading the results of that updated face off.
- intrasight 9y agoI'm personally leaning towards using AWS Lambda for .Net because unlike Azure it will also support ASP.NET.
- bonesss 9y agoIn general, from the perspective of a dev in a MS-shop, AWS gives you a bit more 'responsibility' for infrastructure than Azure, but that level of control lets you get further in practice sooner. Azure has a lot of great things going for it, but I've had a lot of issues getting newer tech out on it without considerable delay. I hope both Azure and GCE give AWS a run for their money in coming years :)
- jamesdrandall 9y agoHi, author here - completely agree. I did try the preview version of Core 2.0 on Azure Functions but it's absolutely woeful at the moment and the dev team confirmed they've done no performance work at all. As I recall the single request response time was around 15ms faster for Core than for full fat framework but the response to increased load was dreadful - never falling below a several second response time with the 400 concurrent user test and errors creeping in. I largely set off down this path as even in isolation the Azure numbers were concerning and not at all what I expected from a serverless architecture and so I wanted a reference point. As soon as I get an update from MS about Core performance work or it leaves preview I'll be rerunning the tests as I'm very aware that doing this across the two runtimes is less than ideal. However from what I've observed the issues look to be more about the broader architecture used for Azure Functions than the framework runtimes. I'm hoping to learn more from Microsoft.
- lvh 9y agoMy experience so far on Google Cloud has been just as bad. I want a simple app that ingests DMARC reports. - Google Cloud Functions can't ingest e-mail. Also, JS lockin if you care about which platforms are supported. - Google App Engine can only ingest e-mail in the Standard environment. OK, fine. But the standard environment regularly takes almost a full minute to load and boot my app, after which the request is killed for taking too long. Additionally, the sample code (!) for opening new Stackdriver contexts breaks with a totally inscrutable error message. You can say "well, maybe your app sucks", but it's been running on AWS (SES + Lambda) for months and every aspect about it has been painless.
- mayank 9y agoUse Mailgun to trigger a POST to your Cloud Function on receiving an email. Bonus: all the parsing is done for you.
- teej 9y agoI don’t know how magic is in Cloud Functions, but I personally set up Mailgun postbacks in AWS and it was a huge pain in the ass to configure correctly. I’d love I know if google does something differently that would have been easier.
- deleted 9y ago[deleted]
- lvh 9y agoBut the parsing code already works fine, that’s not really the hard bit. Then I get all of the lock-in for questionable benefit and something that Just Works on AWS.
- pwaai 9y agoso does this mean that the cold run startup time thing has really been improved enough to build web applications? To me that is the biggest killer in moving forward, not knowing what the end users experience is going to be like because of cold startups that forces a user to wait a long time. From the article's data, it's really tempting to use AWS Lambda now given this benchmark.
- daxfohl 9y agoI do like the "durable functions" concept in Azure preview. AWS has step functions, but coding in flowcharts isn't scalable.
- krisives 9y agoDoes anyone know if any of these services provide the ability to cycle between IPs? One of the use cases for Lambda (and now Azure Functions) is in web scraping where companies get mad if you access them too many times from a single address. Lambda made it clear while they do use a lot of IPs you're not guaranteed to get distribution among them. Seems like a long shot but I figured this is a good place to bring it up.
- intrasight 9y agoI have the same question. Not for scraping but for APIs that are rate-limited by IP address. Like some Google APIs. I certainly wouldn't count on IP distribution from getting around limits for long. Anyway, I'm more interested in understanding how, if at all, we will have to tweak our client code to be well-behaved.
- abol3z 9y agoDid anyone try Bluemix Functions?
- mcintyre1994 9y agoDoes AWS have first class fully production Python yet? In Azure a single import (pydocumentdb) took a function up to 20-30s on Python, I guess they're serious about it being experimental still sadly. Rewriting in Node it takes 50ms for the entire run, crazy difference.
- dronechip 9y agoInteresting article, but you don't mention what tier of AppService plan you tested against in Azure (or what memory limit your Lambdas had)? I would also be curious to see the same comparisons, but with Azure Functions in consumption mode which, despite the perceived drawbacks you mention, would really be the closer apples-to-apples test, wouldn't it?