5 ms·

Codegraph – static code analyzator / code diagramer

97 points by sargstuff 4y ago

esafak 4y ago
That graph is illegible. I would remove most of the labels by default and show them on hover.
- prettyStandard 4y ago
  Agree but maybe only show labels for the largest modules and functions and also scale node size relative to function and module size.
- UncleMeat 4y ago
  Even doing that won't help. Like, a reasonably complex program is going to have a call graph with tens or even hundreds of thousands of nodes. Graphviz won't save you. Collapsing sccs won't save you. It won't actually be useful to navigate.
  - esafak 4y ago
    They could collapse nodes into clusters, and reveal them on zoom, etc. This needs more sophistication to be useful is what I'm saying.
theK 4y ago
Python only? Cool tool nonetheless, have built similar things to visualize code architecture and create doc artifacts in CI, usually the biggest challenge is to scope the pieces that will be included and what layout algo to use, maybe a feature to implement?
- wisefoolkp 4y ago
  Emerge-viz seems to have multiple languages and cool visuals. https://github.com/glato/emerge https://github.com/glato/emerge
marcodiego 4y ago
Unrelated, but curious: when "analyzator" should be used instead of "analyzer"?
- chrisweekly 4y ago
  Never. "Analyzator" is not a word. The HN title matches the repo's README, which was authored by someone whose English language skills are questionable.
Nezteb 4y ago
Shortly before this was posted, I asked for a similar thing! https://news.ycombinator.com/item?id=35603022 https://news.ycombinator.com/item?id=35603022 Although in my case it'd ideally be language-agnostic and hook into LSP so that I can click on nodes and jump to various files/modules/functions. Still a cool project though!
- kevin_thibedeau 4y ago
  Radare2 https://www.radare.org/r/ https://www.radare.org/r/
- sargstuff 4y ago
  emacs. LSP (under editor such as emacs[0]) provides the context/way for being able to define a specific language parse specifics about where to add/define emacs hooks/jump points for a given language nodes/files/modules/functions) [0] : https://www.masteringemacs.org/article/tree-sitter-complications-of-parsing-languages https://www.masteringemacs.org/article/tree-sitter-complicat...
adamc 4y ago
I haven't looked very hard, but `pip install codegraph` didn't result in a codegraph command anywhere. (This was macos, using a Python 3.8 venv.) Digging around a bit found a 'cg' command, which seems to be codegraph (based on the --help output). Didn't show anything interesting for a Django app, though... I'm guessing it couldn't really follow what manage.py was doing? Anyhow, if 'cg' is the command, might want to update the docs.
- sargstuff 4y ago
  README for codequery[0]outlines an overview of tool set to go from source code to visualization. (although, not as a generate live updates per editor changes) [0] : https://github.com/ruben2020/codequery https://github.com/ruben2020/codequery
anigbrowl 4y ago
Good concept. visualization is poor. Force Atlas is really not that great for structural analysis, and nodes need to be weighted by degree among other more code-specific factors. Consider bringing the code graph into NetworkX or some similar library and using graphML to leverage existing visualization solutions rather than reinventing the wheel.
xnuinside 4y ago
I was really surprised to saw a lot of stars on one of my oldest repos )) it was just for my purposes and I didn't continue work on it because thought that it is not needed at all ))
MoreQARespect 4y ago
[dead]
cmcconomy 4y ago
Although it's discontinued, SourceTrail is still great for this purpose.
jschrf 4y ago
Nice work! Other posts in this thread are touching on it, but graph layout and layer segmentations are where you could improve. Screen real-estate is the biggest challenge when visualizing large amounts of code. Similar to maps show different detail levels based on aspect ratio and zoom, you can decompose your "features" (mapping terminology) based on level, i.e. packages->modules->functions->code OP, have you ever used Gephi? I used to build static analysis tools for viz of large-scale codebases. Taking a bit of time to export analysis data into a Gephi format (GEXF in my case) really helped. In Gephi, you can play around with different layout algorithms (peep the plugins). This may help you select a layout algo or combination thereof to help render tons of data in a human-friendly form.
- jschrf 4y ago
  OP here's a couple examples of large scale static analysis diagrams. [1] is a TypeScript database with ~250K LoC. A fairly advanced product that would benefit from what you're building. The colors represent different architectural layers. Each node is a folder with files in it. Edges represent inbound/outbound dependencies. There's simply no way for it to be labelled effectively. [2] is the same codebase, but instead of inter-module dependencies, it's showing function/method calls. One takeaway after working in this space for a while was that visualizing spatial relationships of anything sufficiently complex is extremely hard on a monitor. These representations benefit tremendously from being interactive. [1] https://i.imgur.com/IdY5ghC.jpg https://i.imgur.com/IdY5ghC.jpg [2] https://i.imgur.com/rv4OYun.jpg https://i.imgur.com/rv4OYun.jpg
sasas 4y ago
Can anyone recommend something similar for Java / Smali ? Specific use case is reverse engineering Android application with partially obsufcated code. Current approach is following “xrefs” in JADX and hooking methods with dynamic instrumentation with Frida (displaying backtraces etc). Got to be a quicker way while chipping away renaming classes and methods and mapping out source/sinks.
- deleted 4y ago
  [deleted]
- zknill 4y ago
  I built https://packagemap.co https://packagemap.co which is similar and designed for java. I'd love to get your feedback on it.
  - sasas 4y ago
    Sure! Will have a play and will reach out if appropriate.
- jellyfishbrain 4y ago
  [dead]
ews 4y ago
Nice work !! Quick question: does this read python2 code or only 3?
pleonasticity 4y ago
Very nice. For those interested, you can get similar information using SourceGraph and LSIF in a standardized, language agnostic form: https://lsif.dev/ https://lsif.dev/. It still generally requires build information for each project/language, unfortunately.
lmeyerov 4y ago
I'd be curious if someone loaded these into http://github.com/graphistry/pygraphistry http://github.com/graphistry/pygraphistry and tried something like overlaying a GNN anomaly detection over the base visuals. Both parts are automated & GPU accelerated, so can probably handle some decently sized projects!
zknill 4y ago
Hey, cool idea. I built something similar for working with Java code, but it could be extended to other languages. https://packagemap.co https://packagemap.co I've seen engineers find it hard to work out if their code is organised well together. So I built this tool to show them visually.
HereBeBeasties 4y ago
I honestly don't wish to do the original poster down, but has anyone actually found code visualisation things like this (graph or otherwise) useful, as opposed to just pretty? It seems like you would only really have need for visualisation if your codebase is complex enough. And if it is complex enough, visualising it is generally something of a mess. What actionable insights can be generated? Has anyone changed anything as a result of such a visualisation? It always feels to me like you're better off figuring out which modules you have shouldn't depend on certain other modules and finding only those links. Which is often best done with a simple ripgrep, no?

97 points by sargstuff 4y ago

esafak 4y ago

That graph is illegible. I would remove most of the labels by default and show them on hover.

prettyStandard 4y ago
Agree but maybe only show labels for the largest modules and functions and also scale node size relative to function and module size.
UncleMeat 4y ago
Even doing that won't help. Like, a reasonably complex program is going to have a call graph with tens or even hundreds of thousands of nodes. Graphviz won't save you. Collapsing sccs won't save you. It won't actually be useful to navigate.
- esafak 4y ago
  They could collapse nodes into clusters, and reveal them on zoom, etc. This needs more sophistication to be useful is what I'm saying.

theK 4y ago

Python only? Cool tool nonetheless, have built similar things to visualize code architecture and create doc artifacts in CI, usually the biggest challenge is to scope the pieces that will be included and what layout algo to use, maybe a feature to implement?

wisefoolkp 4y ago
Emerge-viz seems to have multiple languages and cool visuals. https://github.com/glato/emerge https://github.com/glato/emerge

marcodiego 4y ago

Unrelated, but curious: when "analyzator" should be used instead of "analyzer"?

chrisweekly 4y ago
Never. "Analyzator" is not a word. The HN title matches the repo's README, which was authored by someone whose English language skills are questionable.

Nezteb 4y ago

Shortly before this was posted, I asked for a similar thing! https://news.ycombinator.com/item?id=35603022 https://news.ycombinator.com/item?id=35603022 Although in my case it'd ideally be language-agnostic and hook into LSP so that I can click on nodes and jump to various files/modules/functions. Still a cool project though!

kevin_thibedeau 4y ago
Radare2 https://www.radare.org/r/ https://www.radare.org/r/
sargstuff 4y ago
emacs. LSP (under editor such as emacs[0]) provides the context/way for being able to define a specific language parse specifics about where to add/define emacs hooks/jump points for a given language nodes/files/modules/functions) [0] : https://www.masteringemacs.org/article/tree-sitter-complications-of-parsing-languages https://www.masteringemacs.org/article/tree-sitter-complicat...

adamc 4y ago

I haven't looked very hard, but `pip install codegraph` didn't result in a codegraph command anywhere. (This was macos, using a Python 3.8 venv.) Digging around a bit found a 'cg' command, which seems to be codegraph (based on the --help output). Didn't show anything interesting for a Django app, though... I'm guessing it couldn't really follow what manage.py was doing? Anyhow, if 'cg' is the command, might want to update the docs.

sargstuff 4y ago
README for codequery[0]outlines an overview of tool set to go from source code to visualization. (although, not as a generate live updates per editor changes) [0] : https://github.com/ruben2020/codequery https://github.com/ruben2020/codequery

anigbrowl 4y ago

Good concept. visualization is poor. Force Atlas is really not that great for structural analysis, and nodes need to be weighted by degree among other more code-specific factors. Consider bringing the code graph into NetworkX or some similar library and using graphML to leverage existing visualization solutions rather than reinventing the wheel.

xnuinside 4y ago

I was really surprised to saw a lot of stars on one of my oldest repos )) it was just for my purposes and I didn't continue work on it because thought that it is not needed at all ))

MoreQARespect 4y ago

[dead]

cmcconomy 4y ago

Although it's discontinued, SourceTrail is still great for this purpose.

jschrf 4y ago

Nice work! Other posts in this thread are touching on it, but graph layout and layer segmentations are where you could improve. Screen real-estate is the biggest challenge when visualizing large amounts of code. Similar to maps show different detail levels based on aspect ratio and zoom, you can decompose your "features" (mapping terminology) based on level, i.e. packages->modules->functions->code OP, have you ever used Gephi? I used to build static analysis tools for viz of large-scale codebases. Taking a bit of time to export analysis data into a Gephi format (GEXF in my case) really helped. In Gephi, you can play around with different layout algorithms (peep the plugins). This may help you select a layout algo or combination thereof to help render tons of data in a human-friendly form.

jschrf 4y ago
OP here's a couple examples of large scale static analysis diagrams. [1] is a TypeScript database with ~250K LoC. A fairly advanced product that would benefit from what you're building. The colors represent different architectural layers. Each node is a folder with files in it. Edges represent inbound/outbound dependencies. There's simply no way for it to be labelled effectively. [2] is the same codebase, but instead of inter-module dependencies, it's showing function/method calls. One takeaway after working in this space for a while was that visualizing spatial relationships of anything sufficiently complex is extremely hard on a monitor. These representations benefit tremendously from being interactive. [1] https://i.imgur.com/IdY5ghC.jpg https://i.imgur.com/IdY5ghC.jpg [2] https://i.imgur.com/rv4OYun.jpg https://i.imgur.com/rv4OYun.jpg

sasas 4y ago

Can anyone recommend something similar for Java / Smali ? Specific use case is reverse engineering Android application with partially obsufcated code. Current approach is following “xrefs” in JADX and hooking methods with dynamic instrumentation with Frida (displaying backtraces etc). Got to be a quicker way while chipping away renaming classes and methods and mapping out source/sinks.

deleted 4y ago
[deleted]
zknill 4y ago
I built https://packagemap.co https://packagemap.co which is similar and designed for java. I'd love to get your feedback on it.
- sasas 4y ago
  Sure! Will have a play and will reach out if appropriate.
jellyfishbrain 4y ago
[dead]

ews 4y ago

Nice work !! Quick question: does this read python2 code or only 3?

pleonasticity 4y ago

Very nice. For those interested, you can get similar information using SourceGraph and LSIF in a standardized, language agnostic form: https://lsif.dev/ https://lsif.dev/. It still generally requires build information for each project/language, unfortunately.

lmeyerov 4y ago

I'd be curious if someone loaded these into http://github.com/graphistry/pygraphistry http://github.com/graphistry/pygraphistry and tried something like overlaying a GNN anomaly detection over the base visuals. Both parts are automated & GPU accelerated, so can probably handle some decently sized projects!

zknill 4y ago

Hey, cool idea. I built something similar for working with Java code, but it could be extended to other languages. https://packagemap.co https://packagemap.co I've seen engineers find it hard to work out if their code is organised well together. So I built this tool to show them visually.

HereBeBeasties 4y ago

I honestly don't wish to do the original poster down, but has anyone actually found code visualisation things like this (graph or otherwise) useful, as opposed to just pretty? It seems like you would only really have need for visualisation if your codebase is complex enough. And if it is complex enough, visualising it is generally something of a mess. What actionable insights can be generated? Has anyone changed anything as a result of such a visualisation? It always feels to me like you're better off figuring out which modules you have shouldn't depend on certain other modules and finding only those links. Which is often best done with a simple ripgrep, no?