4 ms·

Generally these are good, but in practice #1 doesn't hold. When you start out you need to have a _theory_ about what will be you bottleneck. A lot of times you

Generally these are good, but in practice #1 doesn't hold. When you start out you need to have a _theory_ about what will be you bottleneck. A lot of times you cant implement XYZ, and then measure what was slow and fix that. X, Y, and Z are connected and sometimes you need to build X and Z in a specific way, just so that Y can be made fast, and you know that Y is going to be the bottle neck. Later, when you do measure and know for a fact what is slow, you still have to make a bet on an approach that will make it faster. The more educated you bet is the better. Good programmers measure, but good programmers have to litterate less because they can predict what will be slow, buggy, use too much memory, and make educated bets in advance, that avouid issues. If you start saying that its a rule that you cant predict performance behaviour, then you are dismissing a lot of experience and skills that good programmer accumulate.

mumblemumble 3y ago
In practice, #1 is an iron rule for people who don't believe it, and a loose guideline for people who do. Because observing rule #1 happens to be the best way to get the experience and empirical background that are necessary to develop a good intuition for where to expect bottlenecks.
e12e 3y ago
Could you give a few concrete examples? I'm doubtful it makes a difference in most cases? If you're building a new system, from new requirements - often just getting started is fine? Build, test/measure, throw-away/refactor - repeat? Take rust as an example - start with a draft language, a compiler in ocaml. Iterate. (I concede that Hoare might have known the project might move from ocaml to self-hosted at some point - but I'm not sure that makes much of a difference?)
- quelsolaar 3y ago
  Sure, Right now I'm working on a Constructyive Solid Geometry algorithm, it requires me to cast a lot of rays, so i know this will be slow. I can up-front say that the Raycaster needs to have fast data structure and that its worth giving up some performance to set that data up correctly. I also know it needs to be multithreaded, so thinking up front about how to manage that is a given too. A lot of times, I try to count the zeros. If im doing anything over the network, its going to be miliseconds, but if i do something with memory its going to be nano seconds, so optimize to take remove network requests. If i work on a problem that is multi threaded, i need to conside what can be multi-threaded and what can not. Can i minimize the stuff that cant be multi-threaded, and is there other work to be done while waiting for things that cant be multi-threaded? A lot of times I consider latency vs bandwidth. I know that Latency will always be a harder problem, and a problem hardware has harder time solving, so I try to design for low latency first, and then worry about bandwidth. These are all architectiual decission, that are made early and have a big impact. They are VERY expencive to change, so you want to be as right as you can be the first time. If one part turns out to be slow you can profile and work on it, but changing the struecture of how the program operates and how data is stored and flows is much harder. The author is right, that data structures are the most important tool for optimization. Especially on modern hardware where cache misses are so expencive. The problem is that _everything_ depends on your data structures, so changing them is a lot of work. (I was just at the blender conference where there was a talk about changing the mesh structure from arrays of structs, to structs of arrays, and it took them two years to make this simple change)
  - ska 3y ago
    > I can up-front say that the Raycaster needs to have fast data structure and that its worth giving up some performance to set that data up correctly. I don't think this is a great example, really. You're going to want the brute force raycast working anyway for testing, and you're not going to know the details of what the right space partitioning approach will be until you a) understand the CSG application better, and b) have done some measurements. So it follows the 5 rules pretty well - the only thing I'd add is that you architect the system knowing that the data structures will change for representing the collections of objects and object/ray intersection paths etc. But your first pass should almost certainly be pretty simple and slow, and you should have your measurement code implemented and testing before you try anything sophisticated on the data side. Same goes for chunking up work across threads.
  - physicles 3y ago
    > structs of arrays It’s such a bummer that the optimal memory layout is also a pain to code with in most languages (thinking of numpy in particular, oof). Are there any languages out there that abstract this away? You’d have to give up pointers, but it’s worth it for some use cases.
    - earthboundkid 3y ago
      Zig!
      - physicles 3y ago
        And apparently Fortran too. TIL.
randomdata 3y ago
> they can predict what will be slow If that were the case, would developer-lead startups not have a 100% success rate? After all, a function that takes hours to complete, when an optimized function could take milliseconds, is still more than fast enough if you have no users. I'm not sure anyone has actually proven that they can make such predictions accurately.
- quelsolaar 3y ago
  No, Pro Poker player loose all the time, but they are far better than your average players becasu they make better bets, because they understand the game and the odds better. Good programers are wrong all the time, (I know i am) but they make less misstakes, and they can fix the misstakes faster, because they can predict when there may be an issue. Also being a good programmer is not the same as being good at running a startup.
  - randomdata 3y ago
    > Also being a good programmer is not the same as being good at running a startup. But being able to predict the future of the business is necessary to determine where to optimize. Again, if you have no users, you don't need to optimize at all –period. The success rate should be 100%, as those who predict what to optimize will know when not to go into business. If you get it wrong, you haven't predicted anything. You were guessing.
- aidenn0 3y ago
  1. GP didn't say that they could predict what will be "fast enough" they can predict what will be "slow". 2. A kind reading of GP would also understand that one is predicting what will be slow under some set of conditions. 3. Requiring 100% accuracy to call it a prediction is absurd. A weather forecast is definitely a prediction even though it's not 100% accurate. The NOAA isn't "guessing" weather in the general usage of the term "guess" 4. By your definition extensive benchmarking, measuring a program, and optimizing based on those results is wrong, because we don't even know if the end user will ever run the program!
sjducb 3y ago
You can spike the algorithm that you think will be slow. Usually the slow algorithm is simple to implement and test. If you’re wrong about predicting the speed of the algorithm then you’ve got needlessly complicated code for the rest of the life of the project. People are wrong about the speed of algorithms all the time. Often O(n) and O(n^2) take the same amount of real time because the computer spent 99% of the time getting n from the database server. Your algorithm written in C is often slower than the equivalent Python because the bytecode compiler did something clever. I’ve spent a lot of time speeding up legacy code. It’s usually much easier than you think, and it’s slow for reasons that wouldn’t have been obvious to the original authors. In fact it’s usually slow because the codebase got so complicated that the original author of the slow code couldn’t reason about it any more. But it’s easy for me to fix because I have a concrete example that is “too slow” so I can debug it by running the code and observing the slow points.
- physicles 3y ago
  I love it when I find an actual performance problem with code that doesn’t hit the disk or network. Optimizing is super fun, but it’s so rarely needed. My last time was last year. I noticed that our metrics service, which had been running for a couple years, was now often using 100% cpu. That can cause issues for other services on that node, so I looked into it. Turns out the Go code I wrote to parse Prometheus metrics was a little clunkier than you’d like for the sheer amount of metrics coming from some monitor (I think I’d just upgraded cadvisor or something). I tried getting it to return fewer metrics, but couldn’t figure it out. So I spent an afternoon and rewrote my parser. My hypothesis was that I could make it faster by allocating fewer strings, which turned out to be correct. End result was nearly 10x faster. 10/10, super fun and satisfying. I’ve got about a half dozen other things I’d love to optimize with our current system, but for now I can’t justify the cost.
  - matt_j 3y ago
    That’s a good opportunity to run a profiler and it will tell you pretty quick which things are taking the most time/cpu/memory.
furyofantares 3y ago
I think you are (understandably) not responding to the whole quote. It says not to put a "speed hack" in until you know where the bottleneck is. That doesn't sound like what you described at all. If you're making a video game with tons of physics objects and you know for sure from experience that collision detection is gonna be a major problem, and this is what you're pushing the limits on, I don't think it's a "speed hack" to design the game and its systems around this. Additionally, if you're working on something that you know is gonna be a big performance concern then you definitely want to measure it! Not to find out if it's a concern but to find out how successful you are at dealing with the concern.

mumblemumble 3y ago

In practice, #1 is an iron rule for people who don't believe it, and a loose guideline for people who do. Because observing rule #1 happens to be the best way to get the experience and empirical background that are necessary to develop a good intuition for where to expect bottlenecks.

e12e 3y ago

Could you give a few concrete examples? I'm doubtful it makes a difference in most cases? If you're building a new system, from new requirements - often just getting started is fine? Build, test/measure, throw-away/refactor - repeat? Take rust as an example - start with a draft language, a compiler in ocaml. Iterate. (I concede that Hoare might have known the project might move from ocaml to self-hosted at some point - but I'm not sure that makes much of a difference?)

quelsolaar 3y ago
Sure, Right now I'm working on a Constructyive Solid Geometry algorithm, it requires me to cast a lot of rays, so i know this will be slow. I can up-front say that the Raycaster needs to have fast data structure and that its worth giving up some performance to set that data up correctly. I also know it needs to be multithreaded, so thinking up front about how to manage that is a given too. A lot of times, I try to count the zeros. If im doing anything over the network, its going to be miliseconds, but if i do something with memory its going to be nano seconds, so optimize to take remove network requests. If i work on a problem that is multi threaded, i need to conside what can be multi-threaded and what can not. Can i minimize the stuff that cant be multi-threaded, and is there other work to be done while waiting for things that cant be multi-threaded? A lot of times I consider latency vs bandwidth. I know that Latency will always be a harder problem, and a problem hardware has harder time solving, so I try to design for low latency first, and then worry about bandwidth. These are all architectiual decission, that are made early and have a big impact. They are VERY expencive to change, so you want to be as right as you can be the first time. If one part turns out to be slow you can profile and work on it, but changing the struecture of how the program operates and how data is stored and flows is much harder. The author is right, that data structures are the most important tool for optimization. Especially on modern hardware where cache misses are so expencive. The problem is that _everything_ depends on your data structures, so changing them is a lot of work. (I was just at the blender conference where there was a talk about changing the mesh structure from arrays of structs, to structs of arrays, and it took them two years to make this simple change)
- ska 3y ago
  > I can up-front say that the Raycaster needs to have fast data structure and that its worth giving up some performance to set that data up correctly. I don't think this is a great example, really. You're going to want the brute force raycast working anyway for testing, and you're not going to know the details of what the right space partitioning approach will be until you a) understand the CSG application better, and b) have done some measurements. So it follows the 5 rules pretty well - the only thing I'd add is that you architect the system knowing that the data structures will change for representing the collections of objects and object/ray intersection paths etc. But your first pass should almost certainly be pretty simple and slow, and you should have your measurement code implemented and testing before you try anything sophisticated on the data side. Same goes for chunking up work across threads.
- physicles 3y ago
  > structs of arrays It’s such a bummer that the optimal memory layout is also a pain to code with in most languages (thinking of numpy in particular, oof). Are there any languages out there that abstract this away? You’d have to give up pointers, but it’s worth it for some use cases.
  - earthboundkid 3y ago
    Zig!
    - physicles 3y ago
      And apparently Fortran too. TIL.

randomdata 3y ago

> they can predict what will be slow If that were the case, would developer-lead startups not have a 100% success rate? After all, a function that takes hours to complete, when an optimized function could take milliseconds, is still more than fast enough if you have no users. I'm not sure anyone has actually proven that they can make such predictions accurately.

quelsolaar 3y ago
No, Pro Poker player loose all the time, but they are far better than your average players becasu they make better bets, because they understand the game and the odds better. Good programers are wrong all the time, (I know i am) but they make less misstakes, and they can fix the misstakes faster, because they can predict when there may be an issue. Also being a good programmer is not the same as being good at running a startup.
- randomdata 3y ago
  > Also being a good programmer is not the same as being good at running a startup. But being able to predict the future of the business is necessary to determine where to optimize. Again, if you have no users, you don't need to optimize at all –period. The success rate should be 100%, as those who predict what to optimize will know when not to go into business. If you get it wrong, you haven't predicted anything. You were guessing.
aidenn0 3y ago
1. GP didn't say that they could predict what will be "fast enough" they can predict what will be "slow". 2. A kind reading of GP would also understand that one is predicting what will be slow under some set of conditions. 3. Requiring 100% accuracy to call it a prediction is absurd. A weather forecast is definitely a prediction even though it's not 100% accurate. The NOAA isn't "guessing" weather in the general usage of the term "guess" 4. By your definition extensive benchmarking, measuring a program, and optimizing based on those results is wrong, because we don't even know if the end user will ever run the program!

sjducb 3y ago

You can spike the algorithm that you think will be slow. Usually the slow algorithm is simple to implement and test. If you’re wrong about predicting the speed of the algorithm then you’ve got needlessly complicated code for the rest of the life of the project. People are wrong about the speed of algorithms all the time. Often O(n) and O(n^2) take the same amount of real time because the computer spent 99% of the time getting n from the database server. Your algorithm written in C is often slower than the equivalent Python because the bytecode compiler did something clever. I’ve spent a lot of time speeding up legacy code. It’s usually much easier than you think, and it’s slow for reasons that wouldn’t have been obvious to the original authors. In fact it’s usually slow because the codebase got so complicated that the original author of the slow code couldn’t reason about it any more. But it’s easy for me to fix because I have a concrete example that is “too slow” so I can debug it by running the code and observing the slow points.

physicles 3y ago
I love it when I find an actual performance problem with code that doesn’t hit the disk or network. Optimizing is super fun, but it’s so rarely needed. My last time was last year. I noticed that our metrics service, which had been running for a couple years, was now often using 100% cpu. That can cause issues for other services on that node, so I looked into it. Turns out the Go code I wrote to parse Prometheus metrics was a little clunkier than you’d like for the sheer amount of metrics coming from some monitor (I think I’d just upgraded cadvisor or something). I tried getting it to return fewer metrics, but couldn’t figure it out. So I spent an afternoon and rewrote my parser. My hypothesis was that I could make it faster by allocating fewer strings, which turned out to be correct. End result was nearly 10x faster. 10/10, super fun and satisfying. I’ve got about a half dozen other things I’d love to optimize with our current system, but for now I can’t justify the cost.
- matt_j 3y ago
  That’s a good opportunity to run a profiler and it will tell you pretty quick which things are taking the most time/cpu/memory.

furyofantares 3y ago

I think you are (understandably) not responding to the whole quote. It says not to put a "speed hack" in until you know where the bottleneck is. That doesn't sound like what you described at all. If you're making a video game with tons of physics objects and you know for sure from experience that collision detection is gonna be a major problem, and this is what you're pushing the limits on, I don't think it's a "speed hack" to design the game and its systems around this. Additionally, if you're working on something that you know is gonna be a big performance concern then you definitely want to measure it! Not to find out if it's a concern but to find out how successful you are at dealing with the concern.