5 ms·

Show HN: Sourcetrail – Visual Source Explorer Now Supports Python

89 points by egraether 7y ago

victor106 7y ago
Never knew a product like this existed. This is a godsend for me as I navigate lot of old codebases.
topazas 7y ago
Awesome! Will try without a doubt.
nebucnaut 7y ago
Here is the release post that describes the Python integration a bit more detailed: https://www.sourcetrail.com/blog/sourcetrail_supports_python/ https://www.sourcetrail.com/blog/sourcetrail_supports_python...
devbat8712 7y ago
Oh my god, I've been looking for something like this for like a full year. This is awesome.
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
tmikaeld 7y ago
There exists one for Javascript as well: https://github.com/Bogdan-Lyashenko/js-code-to-svg-flowchart https://github.com/Bogdan-Lyashenko/js-code-to-svg-flowchart Including a VSCode extension (but it seems unmaintained): https://marketplace.visualstudio.com/items?itemName=lucasbadico.code-flowchart https://marketplace.visualstudio.com/items?itemName=lucasbad...
- egraether 7y ago
  Thanks, this looks really nice! It doesn't seem to resolve symbol names however, so you can't see how different functions/definitions are related to each other and navigate between them. But the same author is working on another project that seems to go in that direction: https://github.com/Bogdan-Lyashenko/codecrumbs https://github.com/Bogdan-Lyashenko/codecrumbs
  - tmikaeld 7y ago
    That looks really awesome! Thanks for sharing or I'd miss it.
omeid2 7y ago
Absolutely amazing product! wish it supported more languages though. Other than being a great and useful product, their Global Pricing is based on big-mac index is fantastic. To put this in perspective, the "standard" price of 744.47$ is twice the median wage in Afghanistan.
- oakslab 7y ago
  Took me a while to find out that you refer to the AUD amount. Where did you get the global pricing? I selected different countries, but the amounts still show as equivalent to the USD one? Edit: They are using paypal rate (or close to it).
Asooka 7y ago
I tried the demo back when this was still called Coati. Very impressive, but not quite suitable for my current job. I didn't have time to fully evaluate it at the time, but these are my impressions from trying to use it on our codebase (I can't say exactly who we are due to NDAs). My first criticism would be that the UI wastes a lot of space with all the curves and thick borders. Might be due to me using Vim most of the time, but I'm used to a higher text density. It also felt sluggish, but this might have been improved in the meantime, and also I used it on Linux which is notoriously bad for graphical responsiveness. Second, the indexing was kind of slow. Our codebase is very far from the size of Chrome's, but it is a big commercial C++ project - something like 2M sloc (with comments). This was compounded by the fact that switching branches often led to nearly rebuilding a lot of files due to changes in often-used headers. Third, it can only index one build configuration. Our single source tree is used to build several products and indexing only one build configuration is helpful, but often I need to know if changes necessary for product A will impact product B. It would be really nice if the indexing was split off into its own daemon and I could e.g. have three daemons looking over three build configurations on Linux, an additional remote daemon indexing the MacOS source, and further two running on a Windows host (possibly a VM). This might sound extreme and convoluted, but the kind of large C++ project where Sourcetrail would shine is the kind that has its own very opinionated idiosyncrasies. Finally, it doesn't actually solve my most frequent use-case. Sourcetrail is great for browsing and understanding OOP structure of code. However, I am most often interested in exploring dataflow. I really want to know where a particular member of a struct is set, where the values in the expression come from, where those values are set, where the values in those expressions are from, etc. This can be accomplished with pretty much the same hierarchical interface that Sourcetrail currently has, but instead of classes and methods, the basic units should be expressions and values. Another useful feature would be "where is this value used" - say you have a member of a struct, or a method returning a constant value, and you want to know where the value of the member is used. Not where the member is used, because the value of the member is often copied around, but without being modified. I would really like something that can track through assignments to tell me where this value ends up. Right now Sourcetrail doesn't really cover this usecase better than Vim+ctags+rtags+ripgrep. Yes, this sounds a bit like "Dropbox is just sftp+rsync", but I couldn't make Coati work better for my usecase than my current setup. I can't demand that dataflow analysis be implemented, because I can't promise that I'll use Sourcetrail even if it is, but a data-oriented view of code might be a worthwhile development to consider.
dimmuborgir 7y ago
How does this compare to Code Maps in Visual Studio? Can anyone who has used both comment?
- egraether 7y ago
  Sourcetrail dev here. To my knowledge, without going into too much detail: * Sourcetrail has a tighter integration between source code and graph visualization, because it offers less features than a full blown IDE like Visual Studio and can keep the UI more simple. Users of Sourcetrail can use either code or graph to navigate and both update simultaneously. In Code Maps the visualization stays the static until explicitly updated by the user. * Sourcetrail's visualization is centered around one currently active symbol. Then other nodes are positioned up/below/left/right of it to encode additional meaning into the layout. Nodes are bundled together to de-clutter. In Code Maps the user starts with a certain symbol and then adds more and more symbols to the visualization, building a map. Filters can be applied to de-clutter. * Sourcetrail needs to index all source code upfront, but afterwards querying and visualization generation is fast. I'm not certain, but to my knowledge Code Maps indexes only on demand/only partly upfront, so queries are not as instant. If you are interested in Software Visualization, you can watch my recent talk at ACCU 2019, where I go through some design decisions of our visualization: https://www.youtube.com/watch?v=Gvmp3Gzhv8o https://www.youtube.com/watch?v=Gvmp3Gzhv8o
  - dimmuborgir 7y ago
    Thanks a lot.
albertzeyer 7y ago
I usually use PyCharm in a very similar manner, to browse others code locally. This is much more efficient compared to e.g. browsing/reading the code on GitHub. Whenever I spend more than a few minutes browsing some code on GitHub, I just clone the repo instead and open it in PyCharm. Finding some usages of some function, or simple searching for some string or so is much faster and more comfortable (also because the search on GitHub is so limited). I can only recommend this workflow to anyone who reads others source code. And reading others source code is anyway also something I would recommend everyone. About Sourcetrail, I see from the homepage that it also has some additional graphical representation. Is this more helpful over what PyCharm already offers? If so, maybe this also can be a feature request to PyCharm.
- bshipp 7y ago
  Another vote for the git clone / pycharm workflow. If I'm thinking I might end up tweaking the code I might fork instead of clone to make generating a pull request a simple click in the browser. But I'm definitely in love with Pycharm. Even to the point of using it to evaluate code on remote servers locally on my laptop. Just perfect.
argd678 7y ago
What frameworks did you use to create the UI?
mmmrk 7y ago
Looks very interesting, but seems to be excruciatingly slow at indexing Python projects. Ran it on https://github.com/fonttools/fonttools/tree/master/Lib/fontTools https://github.com/fonttools/fonttools/tree/master/Lib/fontT... (~230 files) and it took 45 minutes while keeping all 16 cores of my Ryzen 1700 at full load, sometimes using ~16 GB of RAM. Exiting the application ends in a SEGV, but seemingly without consequences? Also, after moving the database files to a different folder and renaming the "source group", it now asks me run re-index everything. AAAARRRGHHH.
- egraether 7y ago
  Sourcetrail dev here. Thanks for the feedback! Yes, Python indexing speed is still slow, mostly due to the underlying framework that needs a lot of time to resolve symbol names. We hope we can improve this. The crash on closing was already reported: https://github.com/CoatiSoftware/SourcetrailBugTracker/issues/698 https://github.com/CoatiSoftware/SourcetrailBugTracker/issue... I opened another issue to not inform about indexing when only renaming source groups: https://github.com/CoatiSoftware/SourcetrailBugTracker/issues/700 https://github.com/CoatiSoftware/SourcetrailBugTracker/issue...

89 points by egraether 7y ago

victor106 7y ago

Never knew a product like this existed. This is a godsend for me as I navigate lot of old codebases.

topazas 7y ago

Awesome! Will try without a doubt.

nebucnaut 7y ago

Here is the release post that describes the Python integration a bit more detailed: https://www.sourcetrail.com/blog/sourcetrail_supports_python/ https://www.sourcetrail.com/blog/sourcetrail_supports_python...

devbat8712 7y ago

Oh my god, I've been looking for something like this for like a full year. This is awesome.

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

tmikaeld 7y ago

There exists one for Javascript as well: https://github.com/Bogdan-Lyashenko/js-code-to-svg-flowchart https://github.com/Bogdan-Lyashenko/js-code-to-svg-flowchart Including a VSCode extension (but it seems unmaintained): https://marketplace.visualstudio.com/items?itemName=lucasbadico.code-flowchart https://marketplace.visualstudio.com/items?itemName=lucasbad...

egraether 7y ago
Thanks, this looks really nice! It doesn't seem to resolve symbol names however, so you can't see how different functions/definitions are related to each other and navigate between them. But the same author is working on another project that seems to go in that direction: https://github.com/Bogdan-Lyashenko/codecrumbs https://github.com/Bogdan-Lyashenko/codecrumbs
- tmikaeld 7y ago
  That looks really awesome! Thanks for sharing or I'd miss it.

omeid2 7y ago

Absolutely amazing product! wish it supported more languages though. Other than being a great and useful product, their Global Pricing is based on big-mac index is fantastic. To put this in perspective, the "standard" price of 744.47$ is twice the median wage in Afghanistan.

oakslab 7y ago
Took me a while to find out that you refer to the AUD amount. Where did you get the global pricing? I selected different countries, but the amounts still show as equivalent to the USD one? Edit: They are using paypal rate (or close to it).

Asooka 7y ago

I tried the demo back when this was still called Coati. Very impressive, but not quite suitable for my current job. I didn't have time to fully evaluate it at the time, but these are my impressions from trying to use it on our codebase (I can't say exactly who we are due to NDAs). My first criticism would be that the UI wastes a lot of space with all the curves and thick borders. Might be due to me using Vim most of the time, but I'm used to a higher text density. It also felt sluggish, but this might have been improved in the meantime, and also I used it on Linux which is notoriously bad for graphical responsiveness. Second, the indexing was kind of slow. Our codebase is very far from the size of Chrome's, but it is a big commercial C++ project - something like 2M sloc (with comments). This was compounded by the fact that switching branches often led to nearly rebuilding a lot of files due to changes in often-used headers. Third, it can only index one build configuration. Our single source tree is used to build several products and indexing only one build configuration is helpful, but often I need to know if changes necessary for product A will impact product B. It would be really nice if the indexing was split off into its own daemon and I could e.g. have three daemons looking over three build configurations on Linux, an additional remote daemon indexing the MacOS source, and further two running on a Windows host (possibly a VM). This might sound extreme and convoluted, but the kind of large C++ project where Sourcetrail would shine is the kind that has its own very opinionated idiosyncrasies. Finally, it doesn't actually solve my most frequent use-case. Sourcetrail is great for browsing and understanding OOP structure of code. However, I am most often interested in exploring dataflow. I really want to know where a particular member of a struct is set, where the values in the expression come from, where those values are set, where the values in those expressions are from, etc. This can be accomplished with pretty much the same hierarchical interface that Sourcetrail currently has, but instead of classes and methods, the basic units should be expressions and values. Another useful feature would be "where is this value used" - say you have a member of a struct, or a method returning a constant value, and you want to know where the value of the member is used. Not where the member is used, because the value of the member is often copied around, but without being modified. I would really like something that can track through assignments to tell me where this value ends up. Right now Sourcetrail doesn't really cover this usecase better than Vim+ctags+rtags+ripgrep. Yes, this sounds a bit like "Dropbox is just sftp+rsync", but I couldn't make Coati work better for my usecase than my current setup. I can't demand that dataflow analysis be implemented, because I can't promise that I'll use Sourcetrail even if it is, but a data-oriented view of code might be a worthwhile development to consider.

dimmuborgir 7y ago

How does this compare to Code Maps in Visual Studio? Can anyone who has used both comment?

egraether 7y ago
Sourcetrail dev here. To my knowledge, without going into too much detail: * Sourcetrail has a tighter integration between source code and graph visualization, because it offers less features than a full blown IDE like Visual Studio and can keep the UI more simple. Users of Sourcetrail can use either code or graph to navigate and both update simultaneously. In Code Maps the visualization stays the static until explicitly updated by the user. * Sourcetrail's visualization is centered around one currently active symbol. Then other nodes are positioned up/below/left/right of it to encode additional meaning into the layout. Nodes are bundled together to de-clutter. In Code Maps the user starts with a certain symbol and then adds more and more symbols to the visualization, building a map. Filters can be applied to de-clutter. * Sourcetrail needs to index all source code upfront, but afterwards querying and visualization generation is fast. I'm not certain, but to my knowledge Code Maps indexes only on demand/only partly upfront, so queries are not as instant. If you are interested in Software Visualization, you can watch my recent talk at ACCU 2019, where I go through some design decisions of our visualization: https://www.youtube.com/watch?v=Gvmp3Gzhv8o https://www.youtube.com/watch?v=Gvmp3Gzhv8o
- dimmuborgir 7y ago
  Thanks a lot.

albertzeyer 7y ago

I usually use PyCharm in a very similar manner, to browse others code locally. This is much more efficient compared to e.g. browsing/reading the code on GitHub. Whenever I spend more than a few minutes browsing some code on GitHub, I just clone the repo instead and open it in PyCharm. Finding some usages of some function, or simple searching for some string or so is much faster and more comfortable (also because the search on GitHub is so limited). I can only recommend this workflow to anyone who reads others source code. And reading others source code is anyway also something I would recommend everyone. About Sourcetrail, I see from the homepage that it also has some additional graphical representation. Is this more helpful over what PyCharm already offers? If so, maybe this also can be a feature request to PyCharm.

bshipp 7y ago
Another vote for the git clone / pycharm workflow. If I'm thinking I might end up tweaking the code I might fork instead of clone to make generating a pull request a simple click in the browser. But I'm definitely in love with Pycharm. Even to the point of using it to evaluate code on remote servers locally on my laptop. Just perfect.

argd678 7y ago

What frameworks did you use to create the UI?

mmmrk 7y ago

Looks very interesting, but seems to be excruciatingly slow at indexing Python projects. Ran it on https://github.com/fonttools/fonttools/tree/master/Lib/fontTools https://github.com/fonttools/fonttools/tree/master/Lib/fontT... (~230 files) and it took 45 minutes while keeping all 16 cores of my Ryzen 1700 at full load, sometimes using ~16 GB of RAM. Exiting the application ends in a SEGV, but seemingly without consequences? Also, after moving the database files to a different folder and renaming the "source group", it now asks me run re-index everything. AAAARRRGHHH.

egraether 7y ago
Sourcetrail dev here. Thanks for the feedback! Yes, Python indexing speed is still slow, mostly due to the underlying framework that needs a lot of time to resolve symbol names. We hope we can improve this. The crash on closing was already reported: https://github.com/CoatiSoftware/SourcetrailBugTracker/issues/698 https://github.com/CoatiSoftware/SourcetrailBugTracker/issue... I opened another issue to not inform about indexing when only renaming source groups: https://github.com/CoatiSoftware/SourcetrailBugTracker/issues/700 https://github.com/CoatiSoftware/SourcetrailBugTracker/issue...