5 ms·

> Hand-waving at the garbage collector whilst seemingly doing very little to understand what’s happening doesn’t convince me that there isn’t another, possibly

by JackFr 3y ago

> Hand-waving at the garbage collector whilst seemingly doing very little to understand what’s happening doesn’t convince me that there isn’t another, possibly pretty obvious issue going on. This is pure voodoo profiling. “The job is slow. I guess it’s. . . . here!” Although the author has my sympathy - I spent a month rewriting a module, optimizing every line of code, squeezing every last bit of performance out to discover that the slowness was entirely due to a downstream SQL statement to which I had made a minor “innocuous” change.

insanitybit 3y ago
You call it voodoo profiling, but isn't it kind of just being experienced? Like, if you show me code I can probably tell you where the expensive bits are just by looking. I could be wrong, but it's usually not very hard. Complex runtimes make it a lot harder because then I have to reason through things like JIT and any implicit yielding, but even still. So I'm pretty sympathetic to someone looking at a fairly small section of code, going "oh yeah, that's gonna create a lot of allocations", and then doing work to avoid those allocations. The alternative is that they write benchmarks and start profiling, which is actually quite hard to do well, and takes time. I'm not convinced that the author would have been better served by doing so here.
- smaudet 3y ago
  I'm inclined to agree - when there's a simple solution (easier more reliable language), use it, the right answer is not "oh you are doing it wrong just follow these contortions and you'll be fine". Maybe your language that requires contortions is the problem.
- _gabe_ 3y ago
  I’m curious, have you done much profiling? And what tools have you used if you have? This isn’t a judgment against you, it’s a genuine question because I’m wondering if perhaps I really am just this inexperienced. But the one thing I’ve learned is I’m always very surprised by where the hotspots in my code come from. I always assume big allocations, or bad cache coherency, or O(1) loops when I could use a hash map will be the killers, but then it turns out it’s something else entirely. Because of this, I’ve just learned to never trust my intuition when it comes to this. Compilers are crazy these days, and it’s very difficult to tell if something will be slow or if the compiler is going to clean it up behind the scenes and make it a non issue. Add to that JIT for certain languages and I don’t even know how you would begin to intuit where a hotspot would be. Edit: And just for additional context, I’ve used Intel’s VTune profiler, Tracy, Optick, and written a few manual benchmarks. Using these tools isn’t always easy, but they give a lot of insights that I wouldn’t pick up on my own.
  - Mawr 3y ago
    It's not usually that hard because the higher level the decision, the more impactful it is on performance and the less a compiler will be able to bail you out. That's why you should start optimizing from the highest level and work your way down until the performance is acceptable. Only once you're at the point of optimizing low-level decisions does profiling really start to make sense. A silly example: it doesn't matter how much you optimize your bubble sort implementation, it'll lose out to the most naively written quicksort, which will lose out to keeping the data always sorted and inserting with binary search, which will lose out to finding out how to make the data inherently sorted, which will lose out to figuring out how to avoid the need for the data to be sorted in the first place. The fastest code is no code at all.
  - insanitybit 3y ago
    Oh yes, quite a bit of profiling, certainly. I've used tools like valgrind and cachegrind, perf, manual instrumentation, JMX, or even just `time`, as well as a dozen others I'm sure. > But the one thing I’ve learned is I’m always very surprised by where the hotspots in my code come from. Are you looking at small or large codebases? The code in the article is quite tiny. The major complexity in terms of understanding that code's performance is really just V8 specific stuff. As someone who only reads casually about V8 I'd be much less comfortable guessing about performance. To be clear, there's plenty of times where I can't guess. Certain languages make it really hard, like if there's a lot of inheritance I can't "glance" at the code at all, I have to dig all around the hierarchies. Extremely annoying, I'll 100% have to benchmark in that sort of language. > Compilers are crazy these days, and it’s very difficult to tell if something will be slow or if the compiler is going to clean it up behind the scenes and make it a non issue. For sure, like I said, the big question here is V8. If this were in a system I'm more familiar with I'd have an easier time telling you which optimizations would or would not work. But even still it's not too hard to guess, sometimes. Knowledge of your runtime is the big thing, I think. For example, reusing an allocation might seem like a trivial optimization but allocations can actually have effects (like failing) so an optimization that removes one would be somewhat aggressive - the situations where that optimization will kick in is going to depend on your compiler/JIT. As an example, it's really easy to find documentation about how escape analysis in the JVM works. There are a few rules and you can sometimes look at code and evaluate those rules in your head. You could use those similar rules for C# or Go or JS but you'd be risking that the rules are different in those languages. Example of an article that goes into some depth on JVM escape analysis and the implications: https://blogs.oracle.com/javamagazine/post/escape-analysis-in-the-hotspot-jit-compiler https://blogs.oracle.com/javamagazine/post/escape-analysis-i... > Because of this, I’ve just learned to never trust my intuition when it comes to this. That's totally fine fwiw. I'm definitely pro "just benchmark it", but I will maintain that: a) Benchmarking and profiling are hard. Getting a representative benchmark of your system under load is particularly difficult and a GC can have wildly different performance characteristics under load. b) It's often not that hard to guess. The more complex the runtime, the more complex the system, the harder it is to guess. > and I don’t even know how you would begin to intuit where a hotspot would be. The short answer is you just learn about how a JIT works just like you learn about how a compiler works. Just to reiterate, I'm very much pro "measure before optimizing". Like, very pro. But also, the article shows that guessing definitely works. And I think guessing is super effective and a fine tool to use at times. Perhaps I've overstated my ability to "just look at code and see the slow bits" - it's obviously very much dependent on the code itself, my familiarity with the underlying components such as the runtime or compiler, and perhaps a bit of luck.

by JackFr 3y ago

insanitybit 3y ago

You call it voodoo profiling, but isn't it kind of just being experienced? Like, if you show me code I can probably tell you where the expensive bits are just by looking. I could be wrong, but it's usually not very hard. Complex runtimes make it a lot harder because then I have to reason through things like JIT and any implicit yielding, but even still. So I'm pretty sympathetic to someone looking at a fairly small section of code, going "oh yeah, that's gonna create a lot of allocations", and then doing work to avoid those allocations. The alternative is that they write benchmarks and start profiling, which is actually quite hard to do well, and takes time. I'm not convinced that the author would have been better served by doing so here.

smaudet 3y ago
I'm inclined to agree - when there's a simple solution (easier more reliable language), use it, the right answer is not "oh you are doing it wrong just follow these contortions and you'll be fine". Maybe your language that requires contortions is the problem.
_gabe_ 3y ago
I’m curious, have you done much profiling? And what tools have you used if you have? This isn’t a judgment against you, it’s a genuine question because I’m wondering if perhaps I really am just this inexperienced. But the one thing I’ve learned is I’m always very surprised by where the hotspots in my code come from. I always assume big allocations, or bad cache coherency, or O(1) loops when I could use a hash map will be the killers, but then it turns out it’s something else entirely. Because of this, I’ve just learned to never trust my intuition when it comes to this. Compilers are crazy these days, and it’s very difficult to tell if something will be slow or if the compiler is going to clean it up behind the scenes and make it a non issue. Add to that JIT for certain languages and I don’t even know how you would begin to intuit where a hotspot would be. Edit: And just for additional context, I’ve used Intel’s VTune profiler, Tracy, Optick, and written a few manual benchmarks. Using these tools isn’t always easy, but they give a lot of insights that I wouldn’t pick up on my own.
- Mawr 3y ago
  It's not usually that hard because the higher level the decision, the more impactful it is on performance and the less a compiler will be able to bail you out. That's why you should start optimizing from the highest level and work your way down until the performance is acceptable. Only once you're at the point of optimizing low-level decisions does profiling really start to make sense. A silly example: it doesn't matter how much you optimize your bubble sort implementation, it'll lose out to the most naively written quicksort, which will lose out to keeping the data always sorted and inserting with binary search, which will lose out to finding out how to make the data inherently sorted, which will lose out to figuring out how to avoid the need for the data to be sorted in the first place. The fastest code is no code at all.
- insanitybit 3y ago
  Oh yes, quite a bit of profiling, certainly. I've used tools like valgrind and cachegrind, perf, manual instrumentation, JMX, or even just `time`, as well as a dozen others I'm sure. > But the one thing I’ve learned is I’m always very surprised by where the hotspots in my code come from. Are you looking at small or large codebases? The code in the article is quite tiny. The major complexity in terms of understanding that code's performance is really just V8 specific stuff. As someone who only reads casually about V8 I'd be much less comfortable guessing about performance. To be clear, there's plenty of times where I can't guess. Certain languages make it really hard, like if there's a lot of inheritance I can't "glance" at the code at all, I have to dig all around the hierarchies. Extremely annoying, I'll 100% have to benchmark in that sort of language. > Compilers are crazy these days, and it’s very difficult to tell if something will be slow or if the compiler is going to clean it up behind the scenes and make it a non issue. For sure, like I said, the big question here is V8. If this were in a system I'm more familiar with I'd have an easier time telling you which optimizations would or would not work. But even still it's not too hard to guess, sometimes. Knowledge of your runtime is the big thing, I think. For example, reusing an allocation might seem like a trivial optimization but allocations can actually have effects (like failing) so an optimization that removes one would be somewhat aggressive - the situations where that optimization will kick in is going to depend on your compiler/JIT. As an example, it's really easy to find documentation about how escape analysis in the JVM works. There are a few rules and you can sometimes look at code and evaluate those rules in your head. You could use those similar rules for C# or Go or JS but you'd be risking that the rules are different in those languages. Example of an article that goes into some depth on JVM escape analysis and the implications: https://blogs.oracle.com/javamagazine/post/escape-analysis-in-the-hotspot-jit-compiler https://blogs.oracle.com/javamagazine/post/escape-analysis-i... > Because of this, I’ve just learned to never trust my intuition when it comes to this. That's totally fine fwiw. I'm definitely pro "just benchmark it", but I will maintain that: a) Benchmarking and profiling are hard. Getting a representative benchmark of your system under load is particularly difficult and a GC can have wildly different performance characteristics under load. b) It's often not that hard to guess. The more complex the runtime, the more complex the system, the harder it is to guess. > and I don’t even know how you would begin to intuit where a hotspot would be. The short answer is you just learn about how a JIT works just like you learn about how a compiler works. Just to reiterate, I'm very much pro "measure before optimizing". Like, very pro. But also, the article shows that guessing definitely works. And I think guessing is super effective and a fine tool to use at times. Perhaps I've overstated my ability to "just look at code and see the slow bits" - it's obviously very much dependent on the code itself, my familiarity with the underlying components such as the runtime or compiler, and perhaps a bit of luck.