4 ms·

Show HN: Goroutine-analyzer: A visual goroutine stack dump debugging tool

73 points by adhocpotato 4y ago

adhocpotato 4y ago
I wrote a tool to help dig into goroutine stack dumps from customer deployments. One of the things the open source project (https://github.com/openziti/ziti https://github.com/openziti/ziti) I work on does is mesh networking. This has been very helpful for looking at goroutines from multiple processes, debugging distributed systems issues. It can: * Load multiple stackdumps * Group similar goroutines automatically or manually * Group goroutines by regular expression It is written in Java, because I needed a tool and I've done Java UIs in the past. Since I do Go backend development I figured I'd spend my time getting the tool written. We also use a Go agent library (https://github.com/openziti/agent https://github.com/openziti/agent) which helps us gather stack dumps (and pprof dumps, etc) at runtime.
- cplusplusfellow 4y ago
  Not a criticism but did you consider any of the Golang libraries for doing user interfaces (DearImGUI is what I’m looking at now).
  - adhocpotato 4y ago
    I did poke around for a bit, but I really needed the tool, and I knew it would be days of work if I learned a new tool and hours if I stuck with what I knew. My coworker Andrew did start a similar tool, which is here: https://github.com/andrewpmartinez/grid https://github.com/andrewpmartinez/grid and uses https://github.com/AllenDang/giu https://github.com/AllenDang/giu for the UI. Next time I need a feature I probably take a look at that and see what the model looks like and and how straightforward it is to work with.
chrsig 4y ago
Definitely seems like a useful tool...being in java definitely is a bummer though, makes it harder to integrate into the ecosystem. I wish the go runtime would allow adding annotations to stack frames. just something simple to attach key/value pairs to a frame, or something. Enough to give the goroutine a name. runtime/trace gets close to it by annotating the context with a task/region, but if the trace hadn't been started before the task starts, then it's not going to show up anywhere.
- jeffbee 4y ago
  Does pprof.Do address that use case? https://pkg.go.dev/runtime/pprof#Do https://pkg.go.dev/runtime/pprof#Do
  - chrsig 4y ago
    very possibly! thanks, I totally missed those being added! I'll have to play around with them a bit
cplusplusfellow 4y ago
> The tool is written in Java for expediency on the UI front. I never thought I would live to see the day these words were unironically authored.
John23832 4y ago
https://github.com/bcicen/grmon https://github.com/bcicen/grmon grmon covers some of the functionality and is not Java based.
- adhocpotato 4y ago
  That's an interesting application. I tend to use an IDE if I'm debugging in realtime and otherwise I'm getting stackdumps from the support org. For CLI goroutine analysis there's another tool: https://github.com/linuxerwang/goroutine-inspect https://github.com/linuxerwang/goroutine-inspect which is pretty good. I think if I do a rewrite it might make more sense to go the web route, like the go tool pprof UIs.

73 points by adhocpotato 4y ago

I wrote a tool to help dig into goroutine stack dumps from customer deployments. One of the things the open source project (https://github.com/openziti/ziti https://github.com/openziti/ziti) I work on does is mesh networking. This has been very helpful for looking at goroutines from multiple processes, debugging distributed systems issues. It can: * Load multiple stackdumps * Group similar goroutines automatically or manually * Group goroutines by regular expression It is written in Java, because I needed a tool and I've done Java UIs in the past. Since I do Go backend development I figured I'd spend my time getting the tool written. We also use a Go agent library (https://github.com/openziti/agent https://github.com/openziti/agent) which helps us gather stack dumps (and pprof dumps, etc) at runtime.

cplusplusfellow 4y ago
Not a criticism but did you consider any of the Golang libraries for doing user interfaces (DearImGUI is what I’m looking at now).
- adhocpotato 4y ago
  I did poke around for a bit, but I really needed the tool, and I knew it would be days of work if I learned a new tool and hours if I stuck with what I knew. My coworker Andrew did start a similar tool, which is here: https://github.com/andrewpmartinez/grid https://github.com/andrewpmartinez/grid and uses https://github.com/AllenDang/giu https://github.com/AllenDang/giu for the UI. Next time I need a feature I probably take a look at that and see what the model looks like and and how straightforward it is to work with.

chrsig 4y ago

Definitely seems like a useful tool...being in java definitely is a bummer though, makes it harder to integrate into the ecosystem. I wish the go runtime would allow adding annotations to stack frames. just something simple to attach key/value pairs to a frame, or something. Enough to give the goroutine a name. runtime/trace gets close to it by annotating the context with a task/region, but if the trace hadn't been started before the task starts, then it's not going to show up anywhere.

jeffbee 4y ago
Does pprof.Do address that use case? https://pkg.go.dev/runtime/pprof#Do https://pkg.go.dev/runtime/pprof#Do
- chrsig 4y ago
  very possibly! thanks, I totally missed those being added! I'll have to play around with them a bit

cplusplusfellow 4y ago

> The tool is written in Java for expediency on the UI front. I never thought I would live to see the day these words were unironically authored.

John23832 4y ago

https://github.com/bcicen/grmon https://github.com/bcicen/grmon grmon covers some of the functionality and is not Java based.

adhocpotato 4y ago
That's an interesting application. I tend to use an IDE if I'm debugging in realtime and otherwise I'm getting stackdumps from the support org. For CLI goroutine analysis there's another tool: https://github.com/linuxerwang/goroutine-inspect https://github.com/linuxerwang/goroutine-inspect which is pretty good. I think if I do a rewrite it might make more sense to go the web route, like the go tool pprof UIs.