13 ms·
Things I’ve learned using serverless
- odammit 9y agoLesson 7: when you reach webscale(TM) it gets expensive AF
- mattbillenstein 9y agoDo you have some example numbers here?
- mali9 9y agohttps://servers.lol/ https://servers.lol/ is a one resource at a very high level to see if EC2s or Lambda is a good fit for the use-case you are looking at. The site gives a cost estimate and an application score comments (latency, burstiness, function execution time)
- odammit 9y agoThat is an awesome resource, my friend.
- Finnucane 9y agoWhat this seems to be saying is that serverless is great if you are doing javascript-heavy SPA's, otherwise not so much. Serverless is good if you don't actually need to talk to the server.
- leesalminen 9y agoTotally agree. I used Google Cloud Functions + Google Cloud Storage to deploy a single function “app”. We replaced a SaaS that was a UI for recurring plan signups over Stripe. It started costing us $500/Mo. I replaced it with 16 hours dev+testing. It turned out really well and am very happy with it thus far. I expect maintenance to be minimal thanks to serverless. Though I don’t think it’d work as well for an app with 100+ database tables. Definitely a middle ground in there.
- tehsauce 9y agoJust curious, what kinds of apps use 100+ database tables? I don't know too much about this stuff.
- leesalminen 9y agoWe offer a SaaS product that helps manage pet services businesses. Think CRM+POS+ERP all mashed together, targeted at a niche. Our app is highly configurable by each business. It turns out these businesses are pretty complex :). We’re at 152 production tables today.
- dodders 9y agoOur core banking app has 1000+ tables.
- evrydayhustling 9y agoThis sub-thread is fascinating. On the pet service business, it sounds like there might be more tables due to some kind of object polymorphism on a per customer basis? Are there any other "per-whatever" expansion factors that are multiplying your table count?
- icebraining 9y agoA CRM+POS+ERP is pretty complex by itself, and being more configurable means more stuff is in the database vs the code. I work with a platform that also does CRM/POS/ERP (plus a bunch more), and the products alone have over 10 tables for describing them: the base products, their variants, a list of generic variant attributes, a table detailing which variants have which attributes, a table for specifying the values that those variants have of those attributes, the product categories, two tables for configuring the taxes applied to each product (on purchases and sales), the product images, the list of suppliers. We're already on 10 and we haven't even used those products for anything (stocking, selling, invoicing, purchasing, etc, etc).
- cle 9y agoI didn't get that from this article at all. It didn't really seem to say anything about backend stuff. I build huge serverless backend applications, and IMO it's been fantastic. There is a learning curve, because your application's execution environment is pretty different from traditional applications, but it's allowed us to build a remarkably complex and scalable application, very quickly. And it's been pretty maintainable too.
- wojcikstefan 9y agoCan you elaborate on what your application does, what’s a basic flow of a request, and finally, what are the high-level steps you take to ship a new feature or API endpoint?
- cle 9y agoI can't elaborate on what the application does, but I can talk about high-level architecture and development. Basically we have an API that performs asynchronous data analysis and processing. Our fronting service receives a request, writes some metadata, and places the request in a "queue", which is picked up by a backend poller that starts a workflow execution, which orchestrates the fulfillment of the request. This is all serverless (AWS tech...API Gateway, Lambda, Step Functions, S3, DynamoDB, CloudFormation, CloudWatch, etc.). Serverless makes deployments much easier, since we can version our Lambdas and State Machines using CloudFormation, and have many different versions running at the same time (not fun if you're managing your own hardware!). We have a CD pipeline that builds code changes, deploys them to a test account, and runs integration tests. We use CloudWatch Alarms to monitor production and alert us of any issues. We have some development scripts for pushing code changes with CloudFormation for testing during development. We use that to develop, then once we check the code in, it works its way through our pipeline and into production.
- jetako 9y agoCan't believe I've gone this long (2 years) without knowing about the `sls logs` command. It's life-changing.
- mataug 9y agoLesson 8: Serverless isn't the solution for everything / everyone.
- draw_down 9y agoWho ever claims that about anything? Come on
- nategri 9y ago> "And now we no longer worry about Python version 2 or 3 (is it ever upgrading?)" I just threw out my back cringing.
- jnwatson 9y agoYep. He uses a 2017 version of JavaScript but complains about a 9 year old version of Python.
- nzoschke 9y agoThese lessons match muni experience building https://github.com/nzoschke/gofaas https://github.com/nzoschke/gofaas Except I opt for Golang and the Serverless Application Model (SAM). Go let’s you ditch even more stuff by cross compiling binaries. And SAM is a framework built by AWS and vastly simplifies the config files.
- mooreds 9y agoThat was really great. It would have benefited from discussing the type of apps they were building (estimated traffic, etc). I'm curious whether the 70-90% savings include dev time as well?
- tomc1985 9y agoTwo paragraphs in and I want to slap this guy "That’s quaint" ... ugh. You're quaint.
- slig 9y agoThere's more few paragraphs down: > And now we no longer worry about Python version 2 or 3 (is it ever upgrading?)
- draw_down 9y agoDoesn't this guy know you're only supposed to beat up on JavaScript around here? He really needs to get with the program.
- flanbiscuit 9y agoAs if Node doesn't have multiple versions that a nvm* is exists and widely used (and I really like node) *Node version manager
- BaronVonSteuben 9y agoIt's much different tho. Node is is backwards compatible, python 3 -> 2 is definitely not. And python 3 isn't exactly new ...
- jnwatson 9y agoNode’s policy is that major versions may introduce backwards compatibility, and Node 4.0 did introduce some. Just like Python.
- paulryanrogers 9y agoDid you mean incompatibility?
- 9y ago
- sergiotapia 9y agoIt seems like you've traded in a bunch of open source solutions, to a walled garden in AWS and amazon tools.
- whoisjuan 9y agoTo be fair everything is a walled garden. Even open source solutions. You still need to have an infrastructure to run your code and unless you want to build your own servers you still need to pay AWS/Azure/GCP/DigitalOcean/etc for renting that infra. So I really don't see what's the problem of using something like AWS exclusively. If anything, it makes your life easier.
- deleted 9y ago[deleted]
- boogiewoogie 9y agoAgreeing here. If you can't run your code without AWS or whatever your vendor is, you got bigger problems. I'm talking about the core business code. Ops is important but replaceable.
- hinkley 9y agoYou really class DO with those other guys for lock-in? Or is that the part you are blind to?
- whoisjuan 9y agoI mean. Digital Ocean provides infrastructure too. Nothing is stopping you to run something like OpenFaas on DO.
- zAy0LfpBZLC8mAC 9y agoYeah, everything is a monopoly. Even a free market. You still need to buy stuff! Wut?
- nl 9y agoNote that this discussion is about AWS Lambda, not AWS generally. There are upsides and downsides to using AWS Lambda, but characterizing it as a walled garden is pretty reasonable. That's not the same as code you can run on any Linux server.
- daxfohl 9y agoIt's weird, with all the time and money and brainpower invested over the last 10 years, I still find Heroku to be the lowest maintenance.
- iampims 9y agoHeroku does everything every new platform hopes to achieve and has been doing it for years…
- fbonetti 9y agoAgreed. Serverless is not free, or even cheap for that matter. It’s an entirely new skillset that comes with a million new things to learn and worry about. No thanks.
- daxfohl 9y agoWell it's a thing. But the optimization problem needs to be stated clearly. And I think it is the solution to some optimization problems. Some analysis around that would be interesting.
- nickjj 9y agoAfter reading that I'm so happy that I develop traditional Flask and Rails applications with server side templates and tiny bits of JS thrown in when necessary. I'm all for moving forward and using new stuff if it makes my life better, but from the looks of it, Serverless is still many years away before discussions like that can even take place.
- fbonetti 9y agoI’m jealous. I left Rails behind a couple years ago because I wanted to learn new things and not get stuck on a single stack. I’m currently working with Typescript/Node on a serverless framework on AWS. It’s got all the buzzwords but dear lord it’s complicated. I would not recommend it.
- slig 9y agoI find TypeScript to be an absolute god send in medium/larger projects. It's more complex to get started, but to understand and maintain it's way better.
- deleted 9y ago[deleted]
- fbonetti 9y agoI like Typescript a lot, especially compared to plain JS. AWS lambda, on the other hand, requires very special use cases in order to justify the complexity and learning curve, IMO. It’s not great as a general programming environment.
- whoisjuan 9y agoYou can still use serverless while maintaining a traditional architecture. I have a LAMP app, but I use Lambda + API Gateway to handle a multitude of tasks. The main benefit I have seen so far is that I get to work on complex ideas without worrying too much if this is going to work or scale on my main stack.
- Touche 9y ago
- bklyn11201 9y agoThe same blog has a great post on cold-start times showing Python as the clear winner: https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76 Cold starts are a real issue and while warming via pings can mitigate the issue, you will still run into cold starts when demand scales up. Java with Spring is really difficult with AWS Lambda because of the slow cold starts. Five seconds for a cold start is unacceptable for many applications.
- iamalexm 9y agoThe biggest deal I have seen with cold starts is putting a lambda in a VPC. It has to put an ENI on it and that is expensive. The real question is, why would you use flask in a serverless app? You don't need it and shouldn't use it. The whole point is to get rid of that layer.
- mali9 9y agoIn my opinion, serverless is not there yet for large scale latency-sensitive use-cases (where they cannot be hidden by UI tricks). Startup time of lambda runtime (cold container and then the language runtime) is high for web use-cases where the tail latency of multiple seconds cannot be tolerated. Lambda serverless is really good if you have low RPS and want to pay-for-use due to the low RPS ( prototypes, small production apps, cron-jobs, regular scheduled events, compute intensive - image processing jobs)
- jtchang 9y agoI'm surprised there is no mention of all the downsides of JWT. It's good if you need to scale infinitely but a total pain if you ever have to invalidate specific tokens.
- iamalexm 9y agoWhat does flask have to do with going to serverless? If you are using flask to develop your serverless apps, you are doing it way wrong.
- kolanos 9y agoIf you're curious what cost savings AWS Lambda may provide, here's a handy calculator: https://servers.lol https://servers.lol
- callumjones 9y ago> old-time request-response style of a website with a session managed by the server Breaking news: your app still does this. You just moved the responsibility of request/response elsewhere.
- fapjacks 9y ago"AWS Certified Technologist" == "AWS Lock-In Specialist"
- alpb 9y agoI read this blog post assuming I'm actually going to learn a thing or to in terms of Best Practices in serverless paradigm, some ops/observability tricks and such. It turned out to be a complete AWS advertisement as well as hand-waving to bunch of other blog posts without any good explanations. What makes me curious is don't these people actually ever read Hacker News and know what good technical blog posts look like (unless they're just bunch of paid evangelists writing blogs with catchy titles)?
- reificator 9y agoacloud.guru is a name I recognize from some (halfway decent to be fair) Udemy classes I've taken a few years ago. Likely explains what you're seeing.
- jacques_chester 9y ago> But RDMS systems are just another monolith — failing to scale well and they don’t support the idea of organically evolving agile systems. RDBMS systems could handle billions of complex queries per day in the 1990s (ie last century) and ship with an entire language designed to allow you to safely, incrementally evolve your data model. MySQL and ORMs are not the limits of that universe.
- gdulli 9y ago20 years in this field has taught me that (1) we move on to new technologies more often because we don't understand the current ones than because the current ones are flawed, (2) we fail to weigh the costs and risks and setback of moving to new technologies, and (3) we don't realize that we're conserving overall complexity and flawedness, just moving it around.
- jacques_chester 9y ago> we don't realize that we're conserving overall complexity and flawedness, just moving it around. I broadly think so too: https://news.ycombinator.com/item?id=5262556 https://news.ycombinator.com/item?id=5262556 Though the shifting sands of the economics of compute, disk and network tend to favour this or that approach as time goes on. So while FaaSes are just CGI, they aren't just CGI; but we can at least try to be non-doomed with regards to repetition of history.
- keymone 9y ago(4) new tech is sexy, old tech is all cranky old guys, like 30+, yuk.
- dsparkman 9y agoProblem is that these young whippersnappers don't even realize that there is nothing "new" about this tech stack. You can recreate the new sexy with 30 year old tech. You are talking about a load balancer that is redirecting requests to individual cgi-scripts based on the url. They have just given up knowing how to setup and configure physical servers.
- OnlyRepliesToBS 9y agoServerless shit runs on Amazon's servers.
- evrydayhustling 9y agoThat was interesting. Two questions: 1) Is there a problem with python, or a problem with flask? Isn't this what chalice is for? https://github.com/aws/chalice https://github.com/aws/chalice 2) How are you dealing with cold starts? I learned stuff from this post, but I would have learned more with some background about the workload etc so I could reason about what generalizes and what doesn't.
- Flozzin 9y agoNot the author, but you can set up cloudwatch to hit your lambdas at defined intervals. I set up my lambdas that are accessed through api gateway with a special header to check. If the header is there with the correct value, it just returns. Most keep-alive checks are in the 10-20ms range, and since charges in increments of 100ms, it's the lowest possible tier for getting charged.
- evrydayhustling 9y agoWe have done this. The problem is that concurrent requests have to warm a new instance - so if your concurrent workload increases, newcomers face cold starts. Worth noting that we are more worried about user experience from slow returns than price. Edit: forgot to say thanks for suggestion! Also, here is a related article: https://hackernoon.com/im-afraid-you-re-thinking-about-aws-lambda-cold-starts-all-wrong-7d907f278a4f https://hackernoon.com/im-afraid-you-re-thinking-about-aws-l...
- Flozzin 9y agoThat's an interesting problem. We don't get many concurrent requests and if we do, well then it's not a huge deal. How many instances do you want running? You could set up a separate keep alive path that sends another request to the lambda, with a variable on how deep into the keep alive request 'recursion' and break out if you deep enough. Does that make sense? Super weird and just off the top of my head. edit: this isn't a good solution either because if you have a lambda kicking off 4 other lambdas because you want 5 running, and someone makes a request well then you still haven't warmed up that 6th and your 5 lambdas are running the keep warm code...
- sp527 9y ago> But RDMS systems are just another monolith — failing to scale well and they don’t support the idea of organically evolving agile systems. What the hell does this mean? Completely unsubstantiated nonsense. Hardly anyone needs to scale Postgres past 1B records and 50K QPS, which can be achieved on a relatively affordable pair of synchronously-replicated boxes. This guy clearly doesn't know anything beyond year 1 basics and the post reads like a fatal overdose of Kool-Aid.
- NicoJuicy 9y agoAny comments on the statement: Azure functions is extremely cheap to use? ( Work related and new @ Azure outside of appservices and VM part). Also looking for any gotchas, someone mentioned stability/compatibility issues in another post
- jbg_ 9y agoThere's no such thing as serverless, just someone else's server.
- FigmentEngine 9y agoThere's no such thing as the Internet, just some else's network.
- shruubi 9y agoMy god, This article is written like the author has just joined a cult and is about two more claims about serverless being the high-exalted away from drinking the cool-aide. I mean, it's wonderful that you went to a conference and was able to take some interesting lessons that you could apply to your product, but coming back and dropping your entire platform to rebuild in a new language just to use the fancy new things you learnt is utterly insane. But hey, Kubecon has just finished, so I look forward to the upcoming article "How I rebuilt everything around Containers", should be thrilling to hear about how you tore up your product and rebuilt it a third time based upon some cool conference talks.
- edem 9y agoThanks. You spared me some typing, that's what I wanted to write as well.
- dang 9y agoPlease see https://news.ycombinator.com/item?id=17007019 https://news.ycombinator.com/item?id=17007019 and https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and don't post like this here, regardless of how right you are and how ignorant someone else may be. Comments like these damage HN far more than a weak article.
- wheaties 9y agoWhoa, whoa, whoa. If you're building a serverless app, you don't start with Flask. You start with the lambda and plain old Python. Seriously, what he wrote basically said he tried to build to a construct which lambda isn't meant to directly support and then had all sorts of problems. Stop trying to write a full server and then map it to lambda. Start with lambda and map it to your service. There, done. That's all you need to know.
- edem 9y agoYep, the whole article looks like building a strawman against Python. This mentality of using "cool" frameworks and join the "cool" javascript kids (isn't that an oxymoron?) reeks of the hipsteresque mentality of the whole javascript community...just wait until the next `left_pad`.
- reilly3000 9y agoBlame Zappa. It’s a very cool framework, but it promises serverless for frameworks including Django and Flask. It may be too leaky an abstraction for somebody trying to migrate an existing monolith to serverless or semi-serverless.
- whalesalad 9y agoClassic post where you jump from one tech to another and completely shit all over the stuff you were previously using. There’s hardly a real tangible difference between express and flask. They both do routing and turn requests into responses. When folks make such naive and blanket statements as they do in articles like this it’s impossible to respect any of it. I feel bad for this team and the future they’re going to face with the poor decision making at the top.
- stingraycharles 9y agoI also felt that their main motivation was that they attended a seminar about serverless in late 2017, rather than having an actual problem that required serverless as a solution. I wouldnt worry about the devs in their team so much, it’s great resume building, but I feel for the rest of the company.
- dang 9y agoI'm sure you have a point, but making it in the form of a snarky dismissal breaks the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. Any good effect from being right is drowned out by the bad effect of being a jerk. Maybe someone who makes poorer technical choices than you doesn't deserve respect—that seems dubious, but we can argue about it. The community you're posting to, however, certainly does, and by posting like this you're not only disrespecting it but destroying it. HN is a large, diffuse online community, so the bonds here are inevitably weak. Agitating snark acts as a solvent on those bonds, making the community less cohesive. This is exactly the opposite direction to the one we need. All the default forces already point that way; please don't make them worse.
- whalesalad 9y agoI’ve been a member of this community a very long time. When content like this makes it to the homepage, it suggests the community thinks it’s good. Part of my responsibility as a member of this community is to try and point out when that is wrong. This post is full of misleading information. I never said I didn’t respect the person. I said I didn’t respect the information in the post. I really can’t imagine how my remarks suggest that I’m a jerk. Until I got flagged this was the top comment on the thread. The community seems to agree with my remarks. I stand by them.
- baus 9y agoThe article mentions they used Auth0 and Cognito. I spent quite a bit of time researching Cognito (https://github.com/baus/cognito-strap https://github.com/baus/cognito-strap), but I never figured out how to recognize which user is logged in when using federated identities. I found the docs to be misleading or wrong in many cases. I'm curious if anyone is actually using Cognito in production. It feels like an alpha product to me.
- edem 9y agoThere are zounds of languages which compile to javascript so it is definitely not the only option. With the advent of WASM this will likely get better so we can finally ditch this abomination.
- languagehacker 9y agoSome of these observations are okay, but some of them border on dangerous or not fully considered. Python is an absolutely fine tool in your tool belt for serverless. I use it along with JavaScript all the time -- depending on which has better libraries or makes more sense for a given requirement. Quite honestly, the best part about serverless is that you can generally pick and choose which tool is right for the job up to the language in a far more compositional manner than using more traditional distributed SOA platforms. Dynamo is pretty good, but its value starts to dwindle when you want to be able to do local development, possibly even without a network connection. And most of the traditional ways of interacting with the data layer aren't really available. So for instance, you're not doing to be able to use an ORM for a simple application with Dynamo, which means writing a lot of your stuff from scratch. So given that pragmatically, you probably still want a database, you're going to run into a position where you can't possibly be 100% "serverless". A persistent database connection is a good thing, and one where you can control the number of connections is an absolute requirement at scale. Even if you can tweak your lambdas just right to accommodate your DB's maximum number of connections, you're needless assuming the cost of opening one of those connections on each invocation of your lambda. My recommendation is to use serverless where it's really well suited, which is for distributed, event-driven processing. Your data backend becomes an RPC that can help work with the top of the funnel to map and distributed well-populated messages through your system. For this, I use protocol buffers, and base-64 encode their serialized bytes into an SNS topic. Depending on message size, your mileage may vary here. You can still use some of the more clever AWS offerings to reduce your dependence on some fixed, running server. For instance, Fargate may make it possible for you to run a persistent RPC server for managing read and write requests to your RPC, which is maintaining a well-optimized connection pool with your database. I agree with using JWT for authentication. I agreed with it when stateless authentication pre-dated the service offerings that made serverless a possible paradigm. Serverless generally requires stateless, but you can still reap the benefits from doing the same thing with servers. Hosting a static SPA in S3 I think is one of the less challenging arguments in this blog post and has been a good practice for getting on five years now. Vue isn't necessarily part of it, and marrying the framework with the choice of hosting I think muddies the waters on what's good advice and what's just an opinion. In all introducing serverless technologies to your platform is a great way to significantly minimize your infrastructure costs. t comes at a similar price as building any other SOA -- an increase in the cost of maintenance as debuggability becomes more difficult, and the network becomes more complex. So it's important to think critically about what parts you should take and what parts you should leave or just defer when it comes to your architecture and your business requirements.
- digitalpacman 9y agoWhy does this guy think that JWT tokens secure against CSRF? They're unrelated. This scares me. I want to know what projects he works on so I can avoid them. Not knowing something is insecure is one thing, but knowing that it's insecure scares me.