3 ms·

Coz: Causal Profiling

89 points by keepamovin 3y ago

efferifick 3y ago
I've been a big fan of Emery's research. Coz is one tool that I am always wanting to use, but I haven't had the chance to do so. Check his other research. Some of it is highly accessible via youtube videos. I recommend watching / reading: * Stabilizer * Mesh * Scalene
brunoqc 3y ago
I remember seeing the video 8 years ago but I never heard about it again. I was wondering if the project was stuck or just a PoC. I wonder if it evolved and how useful it has been since.
- nestorD 3y ago
  I have seen Rust people use it, it has a library: https://crates.io/crates/coz https://crates.io/crates/coz
IshKebab 3y ago
I looked into this years ago but it was very complex to set up, and I believe it only really has benefit if you have a lot of threads waiting for a long time for other threads, which is pretty rare in my experience. Neat idea though.
- j-pb 3y ago
  Complex? You include the library, put some "progress" markers into your benchmarks, and run them. It took me minutes in both Rust and Zig to set up. I've seen huge improvements with it for a single threaded embedded database I'm building. The only downside is that it doesn't track some things like memcpy well so you also need a flamegraph.
  - IshKebab 3y ago
    Yes it was very complex. Maybe it has improved since I tried it. Compare to running with a traditional profiler which only requires setting an environment variable (for gperftools at least which is my preference). I don't even need to recompile my program. > I've seen huge improvements with it for a single threaded embedded database I'm building. How would this make any difference for a single threaded program?
    - j-pb 3y ago
      > How would this make any difference for a single threaded program? Coz can slow down individual sections, not only threads. This will show you how your program behaves with specific lines sped up regardless of threads. A modern computer is also a distributed system, so it has the same counter-intuitive performance behaviours, even when just looking at the memory hierarchy.
      - IshKebab 3y ago
        That still doesn't entirely make sense. If your program is single threaded then whenever your code is running, the program is waiting for your code and speeding it up will speed up the whole program by the same amount. If you are never doing concurrent work then there will never be any code where a speed up doesn't affect total runtime because it is waiting for some other work. As I understand it that's the gotcha that Coz fixes, but it doesn't apply in single threaded code. But I have never got it to work so maybe I'm missing something. Maybe it's just easier t interpret. Sampling profilers can be a bit confusing, and debug info doesn't always work as neatly as you'd like.
        j-pb 3y ago
        You are ignoring that even single threaded code has a lot of sources of concurrency and side-effects; memory/caches, branch prediction, prefetching... Watch the talk and look at the examples. Most of them are single-threaded; a bad hash-function causing bucket collisions and linear inserts; SQLite using an indirection table, killing speculative execution and code prefetching. Those two wouldn't really show up in a sampling profiler, because they still take up a tiny amount of time. Sampling profilers show you where time is spend, Causal profilers show you what performance side effects every line of code has.
jcalabro 3y ago
There's a great talk that explains how/why this works for those looking for more info. Also, Go UMass! https://www.youtube.com/watch?v=r-TLSBdHe1A https://www.youtube.com/watch?v=r-TLSBdHe1A
junon 3y ago
I've tried a number of times to integrate Coz into projects and the developer experience for it is pretty bad. I think of all the times I've tried it it linked correctly maybe once, and even then didn't really give me any data. I really want to love this research because having read into it I think it's really valuable, but not ever getting it working is so discouraging.

89 points by keepamovin 3y ago

I've been a big fan of Emery's research. Coz is one tool that I am always wanting to use, but I haven't had the chance to do so. Check his other research. Some of it is highly accessible via youtube videos. I recommend watching / reading: * Stabilizer * Mesh * Scalene

brunoqc 3y ago

I remember seeing the video 8 years ago but I never heard about it again. I was wondering if the project was stuck or just a PoC. I wonder if it evolved and how useful it has been since.

nestorD 3y ago
I have seen Rust people use it, it has a library: https://crates.io/crates/coz https://crates.io/crates/coz

IshKebab 3y ago

I looked into this years ago but it was very complex to set up, and I believe it only really has benefit if you have a lot of threads waiting for a long time for other threads, which is pretty rare in my experience. Neat idea though.

j-pb 3y ago
Complex? You include the library, put some "progress" markers into your benchmarks, and run them. It took me minutes in both Rust and Zig to set up. I've seen huge improvements with it for a single threaded embedded database I'm building. The only downside is that it doesn't track some things like memcpy well so you also need a flamegraph.
- IshKebab 3y ago
  Yes it was very complex. Maybe it has improved since I tried it. Compare to running with a traditional profiler which only requires setting an environment variable (for gperftools at least which is my preference). I don't even need to recompile my program. > I've seen huge improvements with it for a single threaded embedded database I'm building. How would this make any difference for a single threaded program?
  - j-pb 3y ago
    > How would this make any difference for a single threaded program? Coz can slow down individual sections, not only threads. This will show you how your program behaves with specific lines sped up regardless of threads. A modern computer is also a distributed system, so it has the same counter-intuitive performance behaviours, even when just looking at the memory hierarchy.
    - IshKebab 3y ago
      That still doesn't entirely make sense. If your program is single threaded then whenever your code is running, the program is waiting for your code and speeding it up will speed up the whole program by the same amount. If you are never doing concurrent work then there will never be any code where a speed up doesn't affect total runtime because it is waiting for some other work. As I understand it that's the gotcha that Coz fixes, but it doesn't apply in single threaded code. But I have never got it to work so maybe I'm missing something. Maybe it's just easier t interpret. Sampling profilers can be a bit confusing, and debug info doesn't always work as neatly as you'd like.
      - j-pb 3y ago
        You are ignoring that even single threaded code has a lot of sources of concurrency and side-effects; memory/caches, branch prediction, prefetching... Watch the talk and look at the examples. Most of them are single-threaded; a bad hash-function causing bucket collisions and linear inserts; SQLite using an indirection table, killing speculative execution and code prefetching. Those two wouldn't really show up in a sampling profiler, because they still take up a tiny amount of time. Sampling profilers show you where time is spend, Causal profilers show you what performance side effects every line of code has.

jcalabro 3y ago

There's a great talk that explains how/why this works for those looking for more info. Also, Go UMass! https://www.youtube.com/watch?v=r-TLSBdHe1A https://www.youtube.com/watch?v=r-TLSBdHe1A

junon 3y ago

I've tried a number of times to integrate Coz into projects and the developer experience for it is pretty bad. I think of all the times I've tried it it linked correctly maybe once, and even then didn't really give me any data. I really want to love this research because having read into it I think it's really valuable, but not ever getting it working is so discouraging.