2 ms·

While I've been in similar situations and mostly agree with your reasoning, one thing bothers me. Your calculations of engineering hours, which is how most clou

While I've been in similar situations and mostly agree with your reasoning, one thing bothers me. Your calculations of engineering hours, which is how most cloudy stuff gets motivated, hinges on two points. One is that operating the software and using it are so disparate that those two or three trained experts could not also be your developers. But in a small organization, that's likely to be the case. Or put another way, if your developers refuses to gain experitse in the technology, what business do you have relying on it? The other point is the implicated idea that, somewhat rightfully, operating the software without expert knowledge is "amateur hour" but apparently developing for the same software without knowing what you are doing is somehow ok? To me, that doesn't make sense. The same logic applies there, where you need a handful of people to counter sick leave, accidents and other unplanned human outages. This is especially true with Elastic. Sure, running a production cluster on some developer laptop which suddenly reboots for updates isn't great, but neither are broken schemas, uncontrolled bucket growth, broken stemming, or any other of a million things that can go haywire logically without operations being an issue. There's simply no getting away from requiring expertise. If it's your part of your day job, you need to really know the stuff. Yet not knowing what you're doing is so taken for granted that "throw it in the cloud so at least the operations people know what they're doing" is completely accepted logic. That doesn't mean cloud services are bad per se, but the expertise-not-needed argument is. This in turn changes the economics considerably. Sorry for going on a tangent. I know I'm the odd one here but I can't bring myself to accept it.

jillesvangurp 4y ago
Operating infrastructure responsibly is not a skill a lot of developers have and very different from just using infrastructure that is managed for you. I've seen what proper ops teams look like and if you can afford one they are a great asset. 24x7 uptime with five nines is a bit of a myth in our industry. But if you are honest about what that means, it means that you need people with a clue available at any point that you can rely on to be there and to be skilled ready to fix your system when it needs fixing. That's effectively what you pay for with managed services. You'll get a nice resilient setup, backups, monitoring, and support from their expert ops team to help you with whatever. Most small companies are just faking being very dependable and end up spending more on preventable outages and figuring out all sorts of weird shit than they save by self-hosting. I see startups wasting a lot of time on all sorts of silly things that are clearly far out of their comfort zone. Hosted infrastructure is cheap and dependable these days. Unless you can compete with that level of dependability, you should not be self hosting anything. The cost savings rarely add up to being meaningful. The expenses on the other hand are very substantial if you stop and think about it and acknowledge that the per hour cost of your developers quickly adds up to whatever you might actually spend on managed services. Most teams I've worked with spend more on a single developer per month than on their cloud hosting. That's how it should be. Once your bills hit tens of thousands per month, you might want to look at optimizing some of that. But before that, have your developers focus on developing rather than trying to be an amateur ops team. I actually do a lot of consulting related to Elasticsearch and I have dealt with some pretty hairy and misguided setups. By the time they talk to me, they've already wasted a lot of time, money and effort. Most teams I deal with have one or two people that know only a little bit about Elasticsearch. Enough to use it and benefit from it but then they still need me to tell them how to use it properly. So, clearly not enough to operate it responsibly. Even if I come in for just a few days, they'll spend more on me than years of hosting with Elastic Cloud would cost them.

jillesvangurp 4y ago

Operating infrastructure responsibly is not a skill a lot of developers have and very different from just using infrastructure that is managed for you. I've seen what proper ops teams look like and if you can afford one they are a great asset. 24x7 uptime with five nines is a bit of a myth in our industry. But if you are honest about what that means, it means that you need people with a clue available at any point that you can rely on to be there and to be skilled ready to fix your system when it needs fixing. That's effectively what you pay for with managed services. You'll get a nice resilient setup, backups, monitoring, and support from their expert ops team to help you with whatever. Most small companies are just faking being very dependable and end up spending more on preventable outages and figuring out all sorts of weird shit than they save by self-hosting. I see startups wasting a lot of time on all sorts of silly things that are clearly far out of their comfort zone. Hosted infrastructure is cheap and dependable these days. Unless you can compete with that level of dependability, you should not be self hosting anything. The cost savings rarely add up to being meaningful. The expenses on the other hand are very substantial if you stop and think about it and acknowledge that the per hour cost of your developers quickly adds up to whatever you might actually spend on managed services. Most teams I've worked with spend more on a single developer per month than on their cloud hosting. That's how it should be. Once your bills hit tens of thousands per month, you might want to look at optimizing some of that. But before that, have your developers focus on developing rather than trying to be an amateur ops team. I actually do a lot of consulting related to Elasticsearch and I have dealt with some pretty hairy and misguided setups. By the time they talk to me, they've already wasted a lot of time, money and effort. Most teams I deal with have one or two people that know only a little bit about Elasticsearch. Enough to use it and benefit from it but then they still need me to tell them how to use it properly. So, clearly not enough to operate it responsibly. Even if I come in for just a few days, they'll spend more on me than years of hosting with Elastic Cloud would cost them.