4 ms·

How do you know you are right if you didn't measure? I have done things "knowing" what the outcome would be only to be surprised, and I never would have known

by sqeaky 12y ago

How do you know you are right if you didn't measure? I have done things "knowing" what the outcome would be only to be surprised, and I never would have known if I hadn't measured.

implicit 12y ago
It's more about knowing your constraints ahead of time and working within them. If you're doing audio mixing, for instance, you probably have a thread that has to respond with samples within 1ms. Missing the window means catastrophic audio glitches. (it sounds terrible) It's a mistake to write this in Ruby and expect the profiler to tell you something you don't already know.
chadaustin 12y ago
You have to measure at some point to build up your mental model. I run experiments all the time. In the specific example of buffer-builder, I have already built up a (reasonably accurate) mental model of modern CPUs, and I knew what I wanted the generated code to look like. Once I made the generated code look like I wanted, then I was not surprised to find that it outperformed existing libraries by 5x. :) I suspect the alternative approach, "profile the existing libraries and optimize hot spots" would have taken a lot more time.
- sqeaky 12y ago
  With results that good it is surprising that you didn't document your benchmark procedures and include them in your article. I too feel comfortable working with modern CPUs but after performance sensitive projects I benchmark and/or profile to identify what I didn't know. How else can you learn (after listening to all the experts and reading all the documentation)? As for your feeling it would have taken longer with the "alternative approach", I must again ask for numbers. How do you know which approach would take longer without taking measurements on it? Is that with you taking that approach or an expert with that approach taking it. Are you an expert in that approach, yet humbly avoided stating so in the blog post? I don't really see them as alternatives. Using all the knowledge you have up front is simply a good design strategy, but once that knowledge is exhausted you can get more through testing empirically.

by sqeaky 12y ago

How do you know you are right if you didn't measure? I have done things "knowing" what the outcome would be only to be surprised, and I never would have known if I hadn't measured.

implicit 12y ago

It's more about knowing your constraints ahead of time and working within them. If you're doing audio mixing, for instance, you probably have a thread that has to respond with samples within 1ms. Missing the window means catastrophic audio glitches. (it sounds terrible) It's a mistake to write this in Ruby and expect the profiler to tell you something you don't already know.

chadaustin 12y ago

You have to measure at some point to build up your mental model. I run experiments all the time. In the specific example of buffer-builder, I have already built up a (reasonably accurate) mental model of modern CPUs, and I knew what I wanted the generated code to look like. Once I made the generated code look like I wanted, then I was not surprised to find that it outperformed existing libraries by 5x. :) I suspect the alternative approach, "profile the existing libraries and optimize hot spots" would have taken a lot more time.

sqeaky 12y ago
With results that good it is surprising that you didn't document your benchmark procedures and include them in your article. I too feel comfortable working with modern CPUs but after performance sensitive projects I benchmark and/or profile to identify what I didn't know. How else can you learn (after listening to all the experts and reading all the documentation)? As for your feeling it would have taken longer with the "alternative approach", I must again ask for numbers. How do you know which approach would take longer without taking measurements on it? Is that with you taking that approach or an expert with that approach taking it. Are you an expert in that approach, yet humbly avoided stating so in the blog post? I don't really see them as alternatives. Using all the knowledge you have up front is simply a good design strategy, but once that knowledge is exhausted you can get more through testing empirically.