4 ms·
They're kinda glossing over the difficulty in removing 99% of your runtime cost. Unless you've done an insanely bad job of that first draft, you won't be able t
by jeffbee 6y ago
They're kinda glossing over the difficulty in removing 99% of your runtime cost. Unless you've done an insanely bad job of that first draft, you won't be able to do it. Finding a 10% improvement in a system that has enjoyed even the slightest design work will be a challenge.
- t-writescode 6y agoA lot of the improvements I've seen in systems have come from adding indexes. Sometimes, they're just missed over time. "Oh, we never thought that this database would be searched for this way". Other times, it's batch calling the DB ahead of time, rather than calling it for each entry in an array. edit: these are easy mistakes to make over time.
- golergka 6y agoIf you've optimized something 2-5 times, you've done something smart. But if you've optimized something 100 times, you've stopped doing something really stupid.
- beagle3 6y ago“Stupid” is relative. Often, when designing a system, I did not have enough information about how it will actually be used (and in some cases - neither did anyone else; when you create a market all you have is guesses). Things I was sure weren’t important turned out to be important, and vice versa. E.g. in one of the systems, prior business analysis showed that legally valid “chain of custody” would be an issue, which dictated very specific (and often costly, performance wise) decisions. And then after the 3rd customer deployment, it turned out that they don’t care - they’d rather pay less for everything, and lose every 20th claim. And after the 20th customer it was proved beyond the shadow of a doubt. It was stupid to believe what customers said ahead of time, yes. But I don’t think that’s the kind of stupid you are Referring to.
- mlyle 6y agoMaybe. Building things to scale horizontally tends to have a lot of inherent overhead: a lot of fine locking, serialization, consensus, messaging, intermediate representations and storage, etc. Sure it increases the top-end of where you can go, but it can be a huge penalty.
- beny23 6y agoExperience has taught me no longer to be surprised when finding code that determines the number of items in a list by making a SELECT * call to the database and calling .size() on the resulting list and doing it inside a doubly nested loop. Sometimes finding improvements to the order of magnitude is less difficult than it ought to be.
- pvorb 6y agoBut in my experience, that's usually only affecting a small part of a system. If you could save 90% by such optimizations, you're at a very early stage of your application. Otherwise, this would've been spotted earlier. But in a larger system, these things can be around for some time but it's unlikely that fixing them brings an overall performance boost of 90%. More like 3% here, 5% there and so on. Edit: And I agree: I wouldn't be surprised to find a SELECT * and a call to size() in a moderately sized system.
- t-writescode 6y ago> If you could save 90% by such optimizations, you're at a very early stage of your application. You would be very, very surprised. Speaking from experience.
- qes 6y ago> More like 3% here, 5% there and so on. Ten years working on a code base that serves near-StackOverflow levels of traffic and 5% would be a _huge_ win. I don't come across those, or even 3%, very often. Our app has dozens of routes all seeing hundreds to thousands of requests a minute. To get a performance boost that big it has to be in some foundational code that's used nearly everywhere, and that code's already been poured over every way from Sunday. Occasionally we'll run into a bit of code that sneakily becomes a significant drag on the system as traffic through that code grows slowly over time. A fix might result in a large % load drop, but only because there's some pathological problem, which I hesitate to call an "optimization" rather than a "bug fix". We did also uncover many significant optimizations after migrating from dedicated hosted bare metal boxes to the cloud, when our network latency assumptions got thrown out the window - but the bulk of those optimizations were simply "cache it".
- peteradio 6y agoSometimes removing 99% of runtime cost becomes a political issue because of who it embarrasses!
- makapuf 6y agoThis. Its better polically to improve by 5x then later another 5x etc. if you dont want to embarrasse anyone.
- nostrademons 6y agoIt's not unusual to do an insanely bad job of that first draft. Arguably, it means that you did a good job for the primary task of a first draft, which in software is usually figuring out what you should do and whether you should do it without paying too much attention to how. There's perhaps a mistake in shipping that first draft, but usually that lies with management.
- beagle3 6y agoThere is no mistake in shipping the first draft - “no business plan survives first contact with the customer unchanged”. If you don’t ship it, you only learn some of the technical issues but not the customer facing ones. The mistake is often keeping it live past the time it is clear what the mistakes you made are and that they will kill you in the long term.
- hinkley 6y agoIt's usually a meet in the middle arrangement. You have 10,000 customers, and are getting 1000 a month, and each customer starts accumulating data. Your customers are starting to notice your app is slow, and your boss asks you to work out what it'll take to fix the problem, but the problem is you just don't scale. You get a pretty clear idea that by the time you hit 50k customers, you're going to need 10x as many servers, and by 100k, you'll need 25 times as many (because your costs per customer keep going up, and lots of things scale logarithmicly). Either you're smart enough to know your boss will laugh in your face, or you're not and they do. So I guess it's time to start optimizing. You cut some n^2 behavior down to nlogn with a smaller C, and you make your servers 4 times faster, but much more importantly you decrease your run rate. Now you only need 2x as many machines for 5x the customers, instead of 10x. That's 80%, but you're getting diminishing returns. Eventually you have to be 99% better because you have a million users and you couldn't possibly return a profit if you needed that much hardware.