3 ms·

I somehow became specialist at that randomly (like you, random troubled project thrown at me at the odd chance I d find someth as a cheap new joiner turned into

by xwolfi 5y ago

I somehow became specialist at that randomly (like you, random troubled project thrown at me at the odd chance I d find someth as a cheap new joiner turned into a lucky break and a predisposition for enjoying hunting these) and I start now to see patterns and be helpful in all troubled projects Im thrown at as a joker... to a point I lead a performance optimization mini team now... If I were to teach, which I kinda do as a leader of the team, I d just give a few shortcut: 99% of perf issues are stupid and discoverable by profiling (which nobody does somehow, the most experienced the least likely the dev will use a profiler and the most likely he ll have 50 wrong migration ideas to make everything better with a huge budget), once you compressed all leaves of code you can find on a hot path, look at why this hot path even exist, chances are it shouldnt even be called as often, if you did that and are still too slow, redesign, multi thread more, remove business need, expand the hardware. You start also getting an instinct on scale: a troubled team behind by half an hour on a settlement process with an exchange, so big deal, told me they wanted to move the cloud to spin up hundreds of big server with a new scalable design that would take a year to develop with 0 skill in the team yet. How many operation do they need to do in the alloted time ? 30k, with a database on the back. Well, I havent looked yet, but this sounds to me like we can try something before the mega migration lol because 30k units of work is not much and databases tend to be misused (I once fixed a million statement sent by hibernate on a SOAP call, so...) but as you see I try not to give an explanation, it s just an instinct, because prejudice is what kills perf optimization: we'll have to profile each step. One thing I try to teach is: never migrate to a new library while calling the slow code legacy: you'll end up either failing exactly the same way or fake-suceeding on a small sample and leave the actual failure to the next team who'll call your code legacy before moving to the next fancy fad framework of the year and start the cycle of disaster over again.

hinkley 5y ago
One of my weirdest fixes was removing a duplicate database request on an admin page that was taking 30 seconds to load. But instead of dropping to 15 seconds, the response time dropped to 3 seconds. I've also removed a lot of 10% calls that improved performance by 20%. Profilers don't show the cost of your code to the CPU and Virtual Memory caches. And there are new or malloc calls whose full cost is paid by another thread due to the tempo of the allocations. As you say, looking at the count column is a huge thing. But the biggest failure I've seen is one of imagination. So many people give up when the last tall tent pole has been addressed. Even if we've only hit a 3rd of our performance goal, they believe they've tried everything and we should move on. Nobody knows how to do perf analysis with a budget. For instance, the 'current users' count should not be entitled to 10% of the page load time.

by xwolfi 5y ago

hinkley 5y ago

One of my weirdest fixes was removing a duplicate database request on an admin page that was taking 30 seconds to load. But instead of dropping to 15 seconds, the response time dropped to 3 seconds. I've also removed a lot of 10% calls that improved performance by 20%. Profilers don't show the cost of your code to the CPU and Virtual Memory caches. And there are new or malloc calls whose full cost is paid by another thread due to the tempo of the allocations. As you say, looking at the count column is a huge thing. But the biggest failure I've seen is one of imagination. So many people give up when the last tall tent pole has been addressed. Even if we've only hit a 3rd of our performance goal, they believe they've tried everything and we should move on. Nobody knows how to do perf analysis with a budget. For instance, the 'current users' count should not be entitled to 10% of the page load time.