>costs are hidden from engineering Yeah one of my big pet peeves was when engineering teams build platforms to run things on that obscure the cost. There have

>costs are hidden from engineering Yeah one of my big pet peeves was when engineering teams build platforms to run things on that obscure the cost. There have been times where they said "hey we made this big platform for analytics, just ship your stuff as configuration changes and it's deployed!" Then when I did it with very simple small cases, some unoptimized stuff on their end (a lot of what I talked about before) resulted in runaway costs that they, of course, tagged to my team. Ultimately, you can only control what's in your scope and anything else you will need to hope that management can take that runaway cost feedback and make the correct team optimize it away. >I don't know if I would call them mediocre, but without a feedback loop its hard to get engineers to agree whether it's worth time reviewing the code to make it faster compared to just making the db one size larger. This started in the mid 2010s, by which point they should understand that you don't put terabytes of data into S3 in 100kb files. And if not, they should be willing to take some very simple steps to address it (literally just bundling them all in 100mb files with an index file containing the byte offsets of the individual ones would have solved a lot of their problems). There was a feedback loop. There just happened to be big egos more interested in their next fun project of reinventing another solution to another solved problem. I learned there that engineering driven companies sometimes wind up in situations in which the staff engineers love fun new database and infrastructure projects like that more than they enjoy improving their existing product.