3 ms·
> He notes the performance & scalability issues already noted here by other commenters. Go 1.19 has made some improvements in this regard [1]. But yes, profili
by felixge 4y ago
> He notes the performance & scalability issues already noted here by other commenters.
Go 1.19 has made some improvements in this regard [1]. But yes, profiling all goroutines does not scale to programs that use more than perhaps 10k goroutines which isn't entirely uncommon. To overcome this, the goroutine profile API would need to be extended to allow profiling a subset of goroutines. pprof labels could be used to specify which goroutines should be profiled.
> Probably the right thing to do is figure out more of a trace like the current trace profiles but perhaps less low level.
Yeah, in the long run the tracer, perhaps in combination with the cpu profiler [2], also offers a great way of capturing this data. But right now it's too much of a firehose, so it probably needs some way of selecting a subset of goroutines to trace as well. Additionally the unwinding of stack traces is a major bottleneck, so maybe frame pointer unwinding or similar will be needed to make it faster.
I've heard some stuff about future plans to the tracer that would help with the custom binary format problem, so hopefully this will improve in the future.
Anyway, I mostly see fgtrace as a "Do Things that Don't Scale" [3] kind of project. If people like the value it can provide, it will likely motivate myself and others to figure out how to build a version of it that is safe for production usage :).
[1] https://go-review.googlesource.com/c/go/+/387415 https://go-review.googlesource.com/c/go/+/387415
[2] https://go-review.googlesource.com/c/go/+/400795 https://go-review.googlesource.com/c/go/+/400795
[3] http://paulgraham.com/ds.html http://paulgraham.com/ds.html