2 ms·
In my opinion, this model is not the best for optimizing performance. First, it's expensive, performance regression testing is tricky as heck, to get accurate d
by InclinedPlane 17y ago
In my opinion, this model is not the best for optimizing performance. First, it's expensive, performance regression testing is tricky as heck, to get accurate data you need to go to extremes. Second, it concentrates most of the effort on not falling behind which can distract people from the idea that it's possible to actually exceed previous performance by significant margins, even with additional features. Third, sometimes it results in wasting effort tracking down and "fixing" tiny performance regressions without touching the code most responsible for slow performance.
In my opinion it's better to invest in a robust performance profiling infrastructure that gives you the ability to find out the parts of code that make the biggest contribution to the performance (or lack thereof) of your product. From there you can allocate dedicated performance improvement time budget to improve performance as much as possible. I suspect that with such a model you'd be more likely to end up with better performance at less cost and on a more dependable schedule.
- nostrademons 17y agoOne of the projects I work on has close to a zero-tolerance policy on performance regressions, but this is applied per launch and not per changelist. Basically, by the time you launch your feature, you have to show that you're not causing more than a percent or two performance regression (ideally zero, unless you add a lot of value). However, it's up to you whether you want to achieve this by optimizing the hell out of your own feature or by finding code elsewhere in the project that's slow and cleaning it up. You'll know your own code best, obviously, but sometimes the low hanging fruit is in ancient stuff that hasn't been looked at in a while, and a profiler will find it for you. Obviously this only works with collective code ownership, but you should probably have that anyway, since it makes refactoring so much easier.
- InclinedPlane 17y agoCollective code ownership is a dangerous game though, I've seen its pitfalls from many angles. Anywho, I wish there were more data for this sort of thing. Performance tuning is still somewhat of a black art, even performance testing is, to a lesser degree. My personal opinion is that a generally reliable, fast, unit-test-like performance regression testing system combined with solid profiling and making performance improvement (not just absence of regression) a priority are probably the best way to go about it. But there's so little data out there it's hard to back up any opinion with anything other than anecdotal evidence.