4 ms·
> Performance bugs are out there, lurking and unseen Is it a bug if it's unseen? Code that runs slower than it could in theory is literally everywhere. But is
by dahart 10y ago
> Performance bugs are out there, lurking and unseen
Is it a bug if it's unseen? Code that runs slower than it could in theory is literally everywhere. But is it worth paying the cost of the time to fix it, if you aren't experiencing any delays?
I love the way Michael Abrash talks about optimizing. He emphasizes and re-emphasizes that optimization is for people not computers, and that if you aren't experiencing a delay, you're done.
"In other words, high-performance code should ideally run so fast that any further improvement in the code would be pointless.
"Notice that the above definition most emphatically does not say anything about making the software as fast as possible. It also does not say anything about using assembly language, or an optimizing compiler, or, for that matter, a compiler at all. It also doesn’t say anything about how the code was designed and written. What it does say is that high-performance code shouldn’t get in the user’s way—and that’s all."
http://www.jagregory.com/abrash-black-book/#the-human-element-of-code-optimization http://www.jagregory.com/abrash-black-book/#the-human-elemen...
- vilya 10y agoI think that was true back when he wrote the words, but less so now. Even if you're not noticing a delay, code that completes quicker is generally code that uses less power and you can still notice the effect on your phones battery life - or on the size of your monthly bill from your data center.
- dahart 10y agoGreat point. Efficiency comes in different shapes. To respond, I'd say power & cost are human oriented metrics, and I take his point to be use a human oriented metric, for me it's as true now as it ever was.
- datenwolf 10y agoAh darn, I wrote along the same line an hour ago or so but got delayed in hitting the submit button until now. Yes, pretty much this.
- pdpi 10y ago"Unseen" is relative A performance issue that is hard to detect at 10 rps becomes an issue when you're trying to push 1000 rps. A performance issue in a mobile app (or a desktop app on a laptop) might be undetectable from a responsiveness or frame rate point of view, but burn through your battery. If you count memory footprint as a performance issue, thowr don't become apparent until you try to run more things concurrently, and find yourself thrashing when memory should be plentiful.
- dahart 10y agoYes, true, I agree with you. And performance becomes seen when you scale, or measure power consumption and battery life, or check your memory footprint. I think what you're saying is congruent with what Michael Abrash is saying, that you address it if it matters. And sometimes it matters. But almost all the code in the entire world can run faster if someone spends more time optimizing. Anything in any scripting language could be compiled and further optimized. Anything in a compiled language can be further improved, use less memory, access cache faster, etc. etc. Should it be optimized just because it can? Should it be optimized if it doesn't matter under any human oriented metric, including power consumption? (The author didn't mention power consumption or memory footprint.) Sometimes it doesn't matter, and looking for it when it doesn't matter is what we call premature optimization. Premature optimization is also a waste of resources, including time and money, and a huge factor in why almost all software arrives late and over budget.
- datenwolf 10y ago> I love the way Michael Abrash talks about optimizing. He emphasizes and re-emphasizes that optimization is for people not computers, and that if you aren't experiencing a delay, you're done. In a time of battery powered computers CPU cycles spent equate to energy consumed equates to reduction in battery life. Even if your program does not impact the user's experience of runtime behaviour (latencies, delays) it's still impacting in the user's experience on his devices recharge cycle. Also spinning fans generate noise, which is just as a nuisance as a laggy program.
- shanemhansen 10y agoUser focused metrics: 1. It's a user facing bug if I have to purchase a bigger laptop because someone decided to get the smallest element of a list by typing list.sort()[0]. 2. It's a user facing bug if the use of my app heats up their phone, drains the battery, and leaves someone stranded w/o a way to get safely get home on a friday night. 3. It's a business facing bug if my AWS bill is in the hundreds of thousands per month when I could be paying a mere 10s of thousands. (This is more common than you'd think) 4. It's a user facing bug if a sudden increase in traffic (due to my product being featured somewhere) takes down my services and results in lost sales and brand damage. I understand your point, it's not necessarily a problem if a backend feed is taking 45s instead of 30s, or if the the text input subroutine for an interactive app allocates. As far as I can tell as a software developer and daily computer user, people aren't erring on the side of making their software too fast.
- mattmanser 10y agoIf you've a widely used program and you've never run a profiler on it, that's the problem, not using list.sort()[0].
- dahart 10y ago100% agree with all of these, and you did what the author didn't and added criteria that affect people. If these happen, then make the code faster! Abrash is a well known games/graphics programmer, and part of his point is that if your game is already running at 60fps, there's no point in replacing list.sort()[0] with something else. No milliseconds saved will help anyone in that case. (Though yes, as many pointed out, power consumption may be an important metric). Personally, I'd say if your backend feed is taking 45s and could easily take 30s, then you should fix it. :) > people aren't erring on the side of making their software too fast :) 100% agree... too fast isn't a problem we have. But, over-engineering is an exceedingly widespread problem, and that's where I'm coming from. If there's a user-focused metric that shows a problem, yes fix it. If there isn't a user-focused metric, then think twice. The article didn't advocate a user-focused metric as far as I could tell.
- sqeaky 10y ago
- izacus 10y agoPerformance issues cost you money: either you're losing users due to bounce rate (there are studies out there that having pages load for more than 300-500ms or so will significantly increase your bounce rate), or you're paying huge prices for your cloud hosting or you're burning battery of your users and thus losing them as well. The issue is that usually smart design avoids significant performance issues and design which does not consider preformance at all ("we'll optimize it later") usually has to be scrapped and rewritten due to how many systemic issues accumulate over time. When you hit a scaling wall, it's usually orders of magnitude more expensive to rewrite the application than spending a bit of time getting basics right before you started.
- dahart 10y agoTotally agree with you. Those sounds like measurable human facing impacts that can and should be optimized. FWIW, I've also personally watched millions of dollars get wasted on over-engineering, that's part of the context behind my comment. I'm pretty sure I've seen more money lost on problems that don't exist than I've seen lost on avoiding problems that do exist. (Of course, that's impossible to prove, and I could be wrong, but that's my personal feeling.) And in no way do I advocate not considering performance at all. I've done a lot of high performance code, and I think it matters a lot. But I like that Abrash attaches it to something meaningful, not make it faster for the sake of making it faster. The author didn't demonstrate any meaningful metrics like you did, he only said you should make it faster because you can.
- izacus 10y agoOh yes of course... overnegineering is as dangerious as bad engineering. Finding the proper balance IMO is one of the most important skills of senior engineers. Sadly I've also seen way too many people misunderstand the "Don't optimize prematurely" tip and completely ignore any performance considerations when designing software in early stages. Fixing that hurts.
- btilly 10y agoMeh, there is a balance. A performance issue that isn't an issue when doing a single thing can become huge when you're trying to automate doing it many thousands of times. On the other hand there are few ways which are better to obfuscate a code base than to try to optimize performance. The rule of thumb is don't do anything stupid, but don't particularly worry about efficiency until there is a real need. And then, as Knuth advised, you start with a profiler because the performance problem probably isn't where you think.
- craigvn 10y ago> Is it a bug if it's unseen? It's not a bug, it's a feature. Gives you time to get coffee while the page loads.
- 0x07c0 10y ago>"In other words, high-performance code should ideally run so fast that any further improvement in the code would be pointless. And then define pointless, to make a quantified decision on optimization you should use a performance model. Common model is the roofline model. https://en.wikipedia.org/wiki/Roofline_model https://en.wikipedia.org/wiki/Roofline_model