5 ms·
My experience writing large scale search engine, NLP and computer vision services, generally as backend web services that have tight performance constraints for
by mlthoughts2018 8y ago
My experience writing large scale search engine, NLP and computer vision services, generally as backend web services that have tight performance constraints for other client apps that consume from them, is basically fundamentally the opposite of what you say and what you quoted from the article.
It is virtually never the case that profiling reveals a uniformly slow mess of code, and in many cases this doesn’t even make sense as a possibility because you write modular performance tests and profiling tools the same as you write modular unit tests. You would never fire up the profiler and just naively profile a whole backend service all at once, apart from maybe getting extremely coarse latency statistics that only matter in terms of what stakeholder constraints you have to meet. For analyzing cases when you’re not just gathering whole-service stats for someone else, you’d always profile in a more granular, experiment-driven way.
> “Developers should have a solid grasp of computer hardware, compilers, interpreters, operating systems, and architecture of high performance software. Obliviousness to these is the source of performance-hostile program designs. If you know these things, you will write better performing code without consciously thinking about it.”
While it’s good to know more about more things generally, I think the specific claim that any of this will help you design better code in the first place is totally and emphatically wrong.
Instead it leads you down wrong tracks of wasted effort where someone’s “experience and intuition” about necessary optimizations or necessary architectural trade-offs ends up being irrelevant to the eventual end goal, the changing requirements from the sales team, etc. etc.
This happens so frequently and clashes so miserably with a basic YAGNI pragmatism that it really is a good heuristic to just say that early focus on optimization characteristics is always premature.
In the most optimization-critical applications I’ve worked on, getting primitive and poorly performing prototypes up and running for stakeholders has always been the most critical part to completing the project within the necessary performance constraints. Treating it like an iterative, empirical problem to solve with profiling is absolutely a sacred virtue for pragmatically solving the problem, and carries far, far less risk than precommitting to choices made on the speculative basis of performance characteristics borne out of someone else’s “experience” with performance-intensive concepts or techniques. Such experience really only matters later on, long after you’ve gathered evidence from profiling.
- hinkley 8y agoThe performance antagonistic design they’re talking about can’t benefit from what you’re talking about. Worse, it’s usually antagonistic to other things like well-contained modifications. For instance, look at the Big Ball of Mud design. When you are looking at a Big Ball of Mud, these per module performance tests are difficult or impossible to maintain. It’s quite common to see a flame chart with high entropy. Nothing “stands out” after the second or third round of tweaking. Yes there are ways to proceed at this point, but they don’t appear in the literature
- gridspy 8y agoThe quote you mention is specifically talking about the architecture / design of the optimal program. Once a design such as a linked-list of polymorphic classes being iterated on in a loop is established, it is hard to refactor. Those with performance experience would understand that this design will have caching, branch prediction and similar issues. However, I totally agree with you that the overall implementation should be driven by iterative coding and profiling. BUT when it is time to choose memory layouts, data models, etc - planning in advance with an eye to performance is very valuable.
- dman 8y agoI think the issue here is that the large scale systems you cite usually have performance being dominated by the relatively high cost of making a network hop. They dominate runtime to such an extent that unless you are writing code with which is >= O(n^2), your runtime is almost entirely dominated by network latency and the number of hops you make. Also many problems in these domains are embarrasingly parallel and you can effectively summon large number of machines to hold state for you in memory. Where it does get interesting is situations like games / garbage collectors where you want to minimise latency. On such problems you usually cant bolt on performance after the fact.
- mlthoughts2018 8y agoNo, this is not at all what I’m talking about. I’m specifically talking about cases with tight latency requirements after the request has been received. Like receiving a POST request containing many images, and invoking complex image preprocessing with a mix of e.g. opencv and homemade algorithms and then feeding into a deep convolutional neural network. Latencies of 2-3 seconds per request might be the target, and a naive unoptimized implementation of just the backend code (totally unrelated to any network parts) might result in something 10x too slow. In that case, it’s absolutely a best practice to start out with that 10x-too-slow-but-easy-to-implement-for-stakeholders version, and only harden or tighten your homemade computer vision code or use of opencv or some neural network tool after profiling. > “Where it does get interesting is situations like games / garbage collectors where you want to minimise latency. On such problems you usually cant bolt on performance after the fact.” No, that’s exactly the sort of case I’m talking about. It’s not only possibly to refactor for performance after the fact, but dramatically easier to do so, less risky in terms of optimizing for the wrong things, and less prone to re-work.