> This was surprising to read. Generally I feel like engineers want to do the right thing, but are pressured into short term kludges by management. I would say

> This was surprising to read. Generally I feel like engineers want to do the right thing, but are pressured into short term kludges by management. I would say many of the time engineers don't want to do something it's a better solution but they'll claim some technical reason why their way is better. A good example of this is, I was working on a geo search api endpoint that had to work with the TPEG specs. The system was powered by AWS Lambda, which can scale, Elasticsearch which can scale, redis which can scale, etc. The system couldn't scale. It fell on it's ass at 500 requests per second. The part that fell on it's ass was Elasticsearch. Realistically, at 500 reads per second you know it's not really elasticsearch's fault but a data model problem. These are literally excuses that were given: * "We shouldn't do those kinds of searches because they don't scale." - We were contractually obligated to do this search with a 2 million eur penalty fee if we didn't. * "The issue is we're returning too much data" - We weren't having timeout issue, we were having issues with running out of CPU. * "We should hire an elasticsearch consultant to solve it" - We should be able to make Elasticsearch go more than 1% of it's benchmarks. Which when Elasticsearch's benchmarks were brought up "Benchmark's are designed in a certain way to scale" - Which is true when you're getting 20k and they're getting 30k. But when you get 500 and they get 50k - yea that isn't standing up. * "The scale is too much. We'll need to serve millions of requests per second at their highest scale. And no one can do that." - Basically, trying to bamboozle non techies, mixing up database reads with end user requests. It's literally so easy with Elasticsearch you can't even find a blog post on it because it's not worth bragging about. YOu can find people talking about 1 million writes per second tho. The problem was ignored until the point we almost got sued and looked really bad to our biggest customer. This caused all sorts of issues for everyone. All because they didn't want to do the hardwork of figuring out how to build a data model that can scale easily. They ened up spending 30k a month on elasticsearch clusters from AWS to get 500 requests per second. They needed something like 196 CPUS and terrabytes of RAM to serve 500 requests per second of data that was 2-3MB RAW but 150 KB compressed. While on the surface it could appear like they wanted to do a good job, the reality was this was one of many areas that were too difficult for them to deal with so they didn't want to solve it. And people will say that's just a bad team, I've seen that repeatedly. People shy away from the hard to do things.