3 ms·

Can you give an example of what you mean by 'understanding the implications of second order effects'?

sixstringtheory 4y ago
That comment made me think of things like data relations that would require indexing or lookups on foreign keys/IDs. Or bottlenecks in components that perform hefty transformations due to the design of the data structures or unbalanced data sources, like where you need to merge infrequent but large chunks of data with smaller but much more numerous chunks from another async source.
jandrewrogers 4y ago
Computers are closed finite systems, use of resources is a zero-sum game. You cannot optimize aspects of the system design in a vacuum because they are all competing for the same set of resources. Resources applied to the optimization of one thing is the pessimization of something else. You have to apply them where they make the biggest net impact. A concrete example is cache replacement algorithms for storage. The ideal metric is cache hit rate i.e. the percentage of the time that the storage you were looking for is in the cache -- higher the better. Literature is full of academic algorithms that focus on improving cache hit rates under a variety of workloads. In many real systems, the cache replacement algorithm is in the hot path. Most "efficient" algorithms in literature either have poor CPU cache locality or thrash the CPU cache, causing significantly worse performance than is offset by the marginal gains in cache hit rates. Knowing this, good cache replacement algorithms are explicitly designed to minimize CPU cache locality/thrashing problems, at the cost of slightly lower cache hit rates because it has higher throughput in real systems. In this case, allowing the CPU cache to work efficiently is more important than a better disk cache hit rate. In complex systems, the optimal algorithm in isolation is almost never the optimal algorithm in a system. There is a global resource budget. By "second-order effects", I mostly meant understanding how subtle design choices for one part of the system will tacitly interact with the performance of other parts of the system by virtue of how they use global resources. More broadly, this is the discipline of accounting for the hardware resources available to the software at any point in time and identifying conflicts for those resources. Hopefully that gives you a sense of what I meant.
- anonymousDan 4y ago
  Yes thanks, I took it to mean understanding what are likely to be the bottlenecks in a particular system and how your design assumptions might shift those bottlenecks, which seems to dovetail with what you said above. Having said that your example above is interesting, I hadn't thought about conflicts between the cache algorithm itself and the cached data in memory.

That comment made me think of things like data relations that would require indexing or lookups on foreign keys/IDs. Or bottlenecks in components that perform hefty transformations due to the design of the data structures or unbalanced data sources, like where you need to merge infrequent but large chunks of data with smaller but much more numerous chunks from another async source.

jandrewrogers 4y ago

Computers are closed finite systems, use of resources is a zero-sum game. You cannot optimize aspects of the system design in a vacuum because they are all competing for the same set of resources. Resources applied to the optimization of one thing is the pessimization of something else. You have to apply them where they make the biggest net impact. A concrete example is cache replacement algorithms for storage. The ideal metric is cache hit rate i.e. the percentage of the time that the storage you were looking for is in the cache -- higher the better. Literature is full of academic algorithms that focus on improving cache hit rates under a variety of workloads. In many real systems, the cache replacement algorithm is in the hot path. Most "efficient" algorithms in literature either have poor CPU cache locality or thrash the CPU cache, causing significantly worse performance than is offset by the marginal gains in cache hit rates. Knowing this, good cache replacement algorithms are explicitly designed to minimize CPU cache locality/thrashing problems, at the cost of slightly lower cache hit rates because it has higher throughput in real systems. In this case, allowing the CPU cache to work efficiently is more important than a better disk cache hit rate. In complex systems, the optimal algorithm in isolation is almost never the optimal algorithm in a system. There is a global resource budget. By "second-order effects", I mostly meant understanding how subtle design choices for one part of the system will tacitly interact with the performance of other parts of the system by virtue of how they use global resources. More broadly, this is the discipline of accounting for the hardware resources available to the software at any point in time and identifying conflicts for those resources. Hopefully that gives you a sense of what I meant.

anonymousDan 4y ago
Yes thanks, I took it to mean understanding what are likely to be the bottlenecks in a particular system and how your design assumptions might shift those bottlenecks, which seems to dovetail with what you said above. Having said that your example above is interesting, I hadn't thought about conflicts between the cache algorithm itself and the cached data in memory.