6 ms·
IMO, AWS isn’t competing on cheaper-in-dollars-per-byte but rather in faster and cheaper for your engineering team. If your engineering team is free (as you mig
by sokoloff 5y ago
IMO, AWS isn’t competing on cheaper-in-dollars-per-byte but rather in faster and cheaper for your engineering team. If your engineering team is free (as you might decide your time is on a hobby/side project), it’s harder to make the case (I still run my side projects there though), but when they can make the ops team half wearing AWS badges, that offsets a lot of lone-item markups.
- noogle 5y agoBut it doesn't save much engineering time. You now need to manage those services, and the high mark-up means you need to invest effort in scaling things up/down. Yes, it took me about a week to learn to set-up a Postgresql high availability cluster. But now it saves me $4,000 per MONTH for each of our 10 databases. And if you are using EC2 instances, AWS saves very little effort compared to bare metal.
- Daishiman 5y agoSo did you send your database logs to a centralized logging system? Did you set up roles and keys to access those systems? Are your roles integrated with the rest of your permissions system? Do you have a perf dashboard where you can see the real-time usage of your DBs? Have you already rehearsed updating your database version? The thing is that it's totally possible that you learned everything needed to set up the cluster, in my experience most database systems that aren't set up by a professional DBA will sooner or later hit a configuration or maintenance snag. Once that happens, you're pretty much totally on your own and for critical systems that downtime is going to cost you more than the costs of your infra. So you either need to have a DBA on retainer if you're serious about data integrity, or you pay management costs which means your system was set by literally the most expert people on the planet in the area. If you're running a cluster where performance is the highest priority and downtime and maintenance isn't a huge issue because you have a nice decent maintenance window and enough dev cycles to spend on staying up to date, for sure, go for it. But in my experience, if you care more about a system staying up, good managed infra is so much more reliable that it's not even a question.
- noogle 5y ago> So did you send your database logs to a centralized logging system? Not needed. > Did you set up roles and keys to access those systems? Yes. > Are your roles integrated with the rest of your permissions system? Not needed. Access to the DB is limited to a very small circle, by design. Postgresql's mechanisms suffice. > Do you have a perf dashboard where you can see the real-time usage of your DBs? htop/task manager and Postgresql's internal monitoring have so far been sufficient. > Have you already rehearsed updating your database version? Indirectly yes, although AFAIK (happy to learn otherwise) major version update are also not supported by RDS. This has been working for years now, with actual paying customers (so "in production"). It's always about opportunity-cost and diminishing returns: RDS costs about as 3 developers. The benefit the product gets from 3 more developers far outweighs the benefit of another "9" in the SLA. Even if all of the savings would have gone to pay for a DBA, we'd still get a better deal: an in-house DBA will focus on addressing our specific needs. RDS also has a serious opportunity cost since it prohibits using extensions that are very useful for us, forcing us to spend time on workarounds. And frankly, it's not like RDS is even that reliable. We had multiple incidents where the database stopped responding or slowed down to a crawl for several hours, and we had now way to debug this (since we cannot access the server itself). They also don't seem to handle anything about Postgresql tuning. For the customer, the app slowing down (because RDS has a very limited IO bandwidth) is an outage (and is much more frequent than some system error). tl;dr: Many people can meet their needs with a Ford, and living on the streets to pay for the extra features of a Ferrari is not a good tradeoff for them. EDIT: line spacing
- ratww 5y ago> faster and cheaper for your engineering team. I really wonder how true that is. Sure, for things like S3 or RDS it’s indeed easier, but for most other things I find AWS either very limiting or extremely arcane. Even “simple” things like Lambda underdeliver, just this week we run into problems using it with VPC, for example. ElasticBeanstalk was another one, fine-ish for simplistic things but problematic with the smallest customization, also lots of undocumented and undebuggable quirks, like breaking if you use UTF-8 characters in your commit messages, for example. Of course, we now have the problem where some people, both seniors and juniors, only know or only ever worked with AWS, which makes the assertion that it is “faster and cheaper” correct, but is worrisome, as lots of people are not being taught what used to be the basics 10 or 20 years ago.
- zrail 5y agoCurious what your issue was with Lambda and VPCs. We use that combo all the time where I work and it's fine as long as you have the IAM roles correct.
- ratww 5y agoDon't get me wrong, I'm a Lambda fan. But unlike EC2/EBS/etc, talking to some AWS services from a Lambda that's inside a VPC requires additional infra and you have to pay for the egress. In the end it just wasn't worth the price for us. It was a bad surprise money-wise.
- selfhoster11 5y agoLambda had some gotchas around the warm-up time the last time I used it to implement something. We had to have some extra workarounds to prevent the functions from going "cold".
- Ancapistani 5y agoAh, that’s the issue, at least for me - IAM roles are pretty nuanced, and it’s difficult to understand all that’s happening. I’m working extensively with EventBridge now, and their “security” docs mix “what can access EventBridge” with “what EventBridge can access”. Also, different AWS services all seem to have different requirements - e.g, some are role-based, some are service-based, and some are resource-based. It gets complicated very quickly.
- rualca 5y ago> If your engineering team is free (..), it’s harder to make the case (...), but when they can make the ops team half wearing AWS badges, that offsets a lot of lone-item markups. I have to call bullshit on this claim. Let's look at the facts. With AWS there are only two scenarios: either you go with the classic "VMs provided by a cloud provider" which is represented by EC2, or you go with hosted services and higher level abstractions like AWS's serverless offerings. Regarding the EC2, AWS offers absolutely no operational advantage over any other cloud provider, at the expense of being far more expensive. Also, CloudFormation/CDK is arguably far worse and outright developer-hostile than any configuration-as-code alternative. This comparison makes even less sense if we look into AWS' containerization offering, which is either half-baked (ECS) or an afterthought that lags behind alternatives (EKS). Then we have the higher level abstractions of AWS' managed services and serverless options. Price-gouging runs rampant on this domain, and arguably demands much more training and man-hours to become effective at running production services when compared with just running your own services. This scenario entails higher costs and the only arguments that any ops team can muster revolve around sunk cost and vendor lock-in.
- sokoloff 5y agoThe price gouging services make sense if they avoid you having to hire additional employees. That's the benefit of any managed service provider: they can run it more efficiently than you can (once you add all the people, supervisors of those people, people to cover when those people are on vacation, etc). It's a way to shift people costs to IT operational expenses. If you don't do that, it's more expensive. If you do, it can easily be less expensive. I'm pretty sure we're at the point where it's less expensive because developers are waiting minutes rather than weeks [or more] for TechOps actions to happen (we were on-prem previously [and I ran TechOps]). That saves time and changes the way you think about TechOps changes. If they're lengthy, you make choices that avoid changes in TechOps. If they're fast, you make choices that make the most sense for the product and customer.
- rualca 5y ago> The price gouging services make sense if they avoid you having to hire additional employees. But the fact is that it doesn't. It's another service that needs training/experience to develop and operate. Arguing about these hypothetical savings is just a veiled appeal to the sunk cost fallacy and vendor lock-in I've mentioned. > I'm pretty sure we're at the point where it's less expensive because developers are waiting minutes rather than weeks [or more] for TechOps actions to happen (we were on-prem previously [and I ran TechOps]). I'm not sure this scenario is remotely realistic for the past decade or so, specially after the inception of containerization. Even in bare metal deployments anyone can get multiple databases configured and going in a matter of minutes.