4 ms·
> it may sound obvious, but - optimizing your app to fulfill a request in 1/10 the time is like adding 9 servers to a cluster. Optimizing to 1/100 the time (red
by renewiltord 6y ago
> it may sound obvious, but - optimizing your app to fulfill a request in 1/10 the time is like adding 9 servers to a cluster. Optimizing to 1/100 the time (reducing requests from say 1.5 sec to 15ms) is like adding 99 server
Who the hell is casually optimizing away 90% of the latency in the time it takes to spin up 9 more pods? That's insane. Like the latter is an operation on the scale of minutes.
Also, this is a throughput/latency conflation.
90% of the cost a startup incurs is in the things it could be doing instead, not in the infra. Opportunity cost dominates.
- greggyb 6y agoFrom a data analytics perspective, it is surprising to me if I don't find multiple opportunities for order of magnitude improvements in data processing infrastructure at my clients. At one client where I was doing more IC dev work, it got to the point where they'd just chuck reports over the wall at me and ask me to find efficiencies in these. This was supporting >15K internal users in a massive shared reporting and BI infrastructure, so it was well worth it to have someone spending 10s of hours on optimization on a regular basis. I could routinely find order of magnitude improvements in reporting queries that were already running in production. If bringing things from POC/end user mockup to production, I could frequently find two orders of magnitude of improvement in query time, which reflected directly in latency to report rendering for end users. That client was not special. I wouldn't call this sort of work casual, and it's a very different domain than optimizing serving web apps. I would spend days to weeks on such optimization tasks at the client I mentioned above. It might take a few weeks to do serious optimization work on large ETL pipelines, but I routinely find big gains in performance on such projects - usually in the range of 1/2-1/10 run time compared to baseline.
- jtsiskin 6y agoYes, the key is to at least be aware of it. I bet there is a non-insignificant amount of existing endpoints out there that could easily be improved a large percent if someone actually took the time to profile it. Just from the shifting codebase, build up of complexity, “legacy code”, etc. that no one is probably looking at. If there was, for example, easy to use developer tooling that automatically identified bottlenecks, the opportunity cost scale could tip towards optimization vs having to pay for and manage more servers. But the opportunity cost insight is great; optimization only wins if the cost to optimize is low and the amount that gets optimized is high.
- falcolas 6y agoRight behind the cost of your employees is the cost of your infrastructure. If your company is profitable, every dollar of infrastructure costs cut is a dollar more profit. For a company still working on a runway, every dollar cut from your infrastructure is more time you can spend working on a product. It's also important to note that those efficiencies don't go away on their own. It's more profit/runway until you retire that code. Opportunity costs exist, but so does spending your money wisely.
- stjohnswarts 6y agoYou can't look at it in dollars, you have to look at it in terms of what percentage of cost IT/cloud infra are vs the rest of the costs of the company. If it's not that much then from an economic point of view just scaling out may be the best choice.
- renewiltord 6y agoOkay, you're right, but I guess in my head I was thinking pre-seed/seed and you were thinking post-series-A.
- gravypod 6y agoI've optimized software at startups for a 10x to 100x performance improvement. It's not hard to find these areas when you have the fortune of hindsight. These past 2 weeks I took CI from ~40 min to ~2 min and not only improved everyone's lives at the company but also made it way cheaper to host beefier CI machines. If 1 build takes 20 minutes & 32 GB of ram vs 1 minute & 32gb of ram the costs are way different. At previous companies re-architecting core systems to allow better binpacking saved similarly large amounts of money (15k/month -> 1k/month). At way smaller companies (2 people), though, this is way harder unless someone has done something really wrong.
- renewiltord 6y agoThat's a fair point. I didn't clarify the size. I guess I was thinking of the latter case.
- htunnicliff 6y agoWhat sort of things did you optimize to improve the build times?
- gravypod 6y agoDropping our Docker+Gradle builds for Bazel and using Bazel's caching and running a Bazel daemon on CI machines so I don't pay startup/analysis costs. On builds with no change CI time is like under 1 minute. I could have done that with Gradle but we also want to support multiple languages (Java, Python, NodeJS & React, Golang).
- renewiltord 6y agoGreat answer. Honestly, while the theory is that you can Dockerize your build and you can do remote caching with Bazel I've never seen anyone do it. Like it seems some confluence of steps that doesn't occur. I think I wouldn't use Docker for builds at all just because of this performance regression.
- 6y ago
- QuadrupleA 6y agoThe point is more that the time involved to set up the scalable architectures in the first place - load balancer, app cluster / containerization, deployment automation, database replication etc. - might be better spent just doing some simple optimizations on a single VM first. A lot of people jump straight to big distributed architectures with 1/1000th the traffic to justify them.