4 ms·
I really think there's more to this than most people will admit. Our very early, small startup (4 engineers, low six figures ARR) accumulates $4,000+/month in
by 013a 7y ago
I really think there's more to this than most people will admit.
Our very early, small startup (4 engineers, low six figures ARR) accumulates $4,000+/month in cloud bills. Its very difficult to manage cloud costs on a small engineering team.
One of our recent examples is with SQS. Our SQS bill is something like $50/month. We process ~100k messages every month. Why is it so expensive, then? It isn't because its processing tons of messages; its because we use open-source polling libraries that are pretty aggressive with their poll rate (then multiply that out by N instances of the poller and there's your bill).
I find it fun to think about "what could I get for $50/month" in regards to a queuing solution. As an extreme counterpoint, Redis running on a raspberry pi in the closet would be a rounding error in our cloud bills over the span of a year. The gray area between these two extremes is where the value lies, and I think that's the author's point: redis on a VPC at Hetzner or DigitalOcean. Billions upon billions of messages processed every month at a fraction of the cost.
So, why pay AWS? There's not NO reason. Reliability is the biggest one I think. But herein lies what I believe to be the intrinsic problem with the cloud: No one can put a price on reliability. If I pay an extra $10k a year to be 99.99999 available (~10s/year), versus 99.99 (~1hr/year), how much extra value does my business receive? No one even CARES what that number is. They just know that more nines is better, so we gotta pay it. But we won't pay too much for it. At some point, it crosses some TOTALLY arbitrary threshold in the CTO's head where its "too expensive" and we need to reduce costs. But, then, we're so bought in that we can't. We can tweak poll rates, we can buy reserved instances, maybe see a 20% reduction, but the pattern is set. We're just a serf to Amazon now.
The other quoted reason is always "the cloud is easier". "You'll save money on HR." "Your engineers can focus on the product." I think Amazon has shouted this line so much from the rooftops that even very smart engineering managers have begun to believe it. Heroku was really the last platform I've used that felt like it could fulfill this promise to any degree of meaning. It really did everything, but AWS has managed to create a network of services that is so fundamentally complex that even when they release services to "simplify" things they actually create more complexity (beanstalk, amplify). You're not getting around having an engineer spending a decent chunk of her time managing this complexity (building CI pipelines, managing IAM permissions, alerting strategies, centralized log ingestion, autoscaling, cost management, encoding everything into CloudFormation, ugh maybe terraform will be easier, figuring out the madness behind VPCs because some random CF stack you found that you thought could automate something for you just spat out an egress-only internet gateway and what the fuck even is that I'm just one person why did I need that).
I've been working in AWS since... the beginning. There are three good AWS services: S3, SQS, and EC2 (in that order). Everything else ranges somewhere between "Fuck it, I guess I'll use cloudwatch" and "what was the rekognition team smoking when they built this, and are they getting a daily dose of it through the vents because its been years and its still not better."
The best part about those "good" services is that they're all solved problems available on a wide variety of cloud providers. Block Storage, Queues, and VMs. I think that's what the author is getting at, in a weird way. The cloud is fine. Managed services on top of open source software is probably fine. Even closed-source managed services can be fine, if they're so stupidly simple that you could explain it to your 8 year old and he'd understand it. But the vast majority of AWS (or Azure, or Google Cloud) is none of these things, and that's the stuff that you should avoid. Or, if you want to save some money, head over to Hetzner or Digital Ocean.