8 ms·

Show HN: Tach – Visualize and untangle your Python codebase

263 points by the1024 2y ago

Hey everyone! We're Evan and Caelean, the authors of Tach. Tach lets you visualize the architecture of your Python codebase, and gives you the tools to incrementally improve it. It uses module boundaries to give teams the benefits of microservices without the deployment complexity. If your code has been getting tangled up as your team and codebase grows, Tach helps you move back in the right direction, incrementally and quickly. You can use Tach to incrementally adopt a "modular monolith" architecture [1], for better local reasoning and smoother feature development. Since our last Show HN (https://news.ycombinator.com/item?id=41359181 https://news.ycombinator.com/item?id=41359181) we've shipped support for layers, third party dependencies, visualizations, and more. Tach is: * Open source (MIT) * completely free * fast (written in Rust) * in use by teams at NVIDIA, PostHog, and more. One way Tach differs from existing systems that handle this problem (build systems, import linters, etc) is the ability to be incrementally adopted. Also, runtime speed. If you struggle with dependencies, onboarding new engineers, or a massive codebase, Tach is for you! We built it with developers in mind - with clean integrations into Git, CI/CD, and IDEs, and the performance for it to be effective in any form factor. [1] https://www.milanjovanovic.tech/blog/what-is-a-modular-monolith https://www.milanjovanovic.tech/blog/what-is-a-modular-monol...

jtwaleson 2y ago
Cool! Do you have plans to launch a paid offering? The website makes me think it's a company but I didn't see any pricing / sales details.
- 0x63_Problems 2y ago
  Co-author here - We do provide a web platform (https://www.gauge.sh/platform https://www.gauge.sh/platform) which we have been developing with design partners. The fundamental difference between using Tach alone vs. the platform is that the platform provides incremental enforcement at the pull request level. We're always happy to chat about adding more design partners! email: founders@gauge.sh
vednig 2y ago
This is pretty cool, thanks for sharing
hansonkd 2y ago
Really excited to see this project gain traction. > Note that this graph is generated remotely with the contents of your `tach.toml` Isn't shipping off parts of your codebase to a 3rd party without warning in the CLI a security risk? Or in regulatory environments you get audited that your code was only stored on properly vetted services which is why some sales cycles for AI coding assistant tools are so long. It would be kind of frustrating to have something like that happen and get set back on licensing, etc. Just from the video it doesn't seem like any sort of warning that you are shipping config files to your servers and the URL that you produced doesn't seem to have any authentication. Maybe i am misunderstanding that functionality, but it gives me pause to use it.
- 0x63_Problems 2y ago
  Co-author here, fair question! In short, we want to make the visualization UX as smooth as possible, and this is best done with a web app. The URLs use UUIDs, and the contents being sent don't include literal source code, only module names and Tach configuration. We will also delete graphs by UUID on request, and have done so in the past. That said, we do try to be up-front about this, which is why that disclaimer exists, and when running this command on the CLI, you must supply an explicit `--web` argument to `tach show`. Otherwise, the default behavior is to generate a GraphViz DOT file locally.
  - bmitc 2y ago
    Why not just let users run the web app locally? There's no reason it needs to be remote. Also, the mere fact that it sends any data, no matter what you say it contains is a non-starter at many places. And even module names can contain proprietary data.
    - 0x63_Problems 2y ago
      I can understand the frustration, but I think there are legitimate reasons to run this remotely. Tach is an installable Python package, shipping a full web app would have to come in a separate form factor and has significant maintenance implications. Given we are explicit about the remote app before anything is sent, require explicit opt-in, and we provide usable alternatives locally, we prioritize shipping a useful graph experience that is immediately usable. If you are at an enterprise that cannot tolerate this, then you can use a local viewer with either GraphViz DOT format or Mermaid which is generated by using `tach show` or `tach show --mermaid` respectively.
lijok 2y ago
Tools like this rub me the wrong way. We have well established conventions like prefixing private modules and symbols with an underscore, or declaring your public interfaces in the __init__.py file, but the Python developer decries it as "busywork", "weird" and "hard to read", so we instead use tools like this. We can manage dependencies with protocols, a type checker and generally following SOLID principles, but the Python developer decries it as "too indirect and convoluted", so we instead use tools like this. This is more commentary on the Python developer than this tool. Tach looks great.
- 0x63_Problems 2y ago
  Co-author here, I can understand where you're coming from! Part of the philosophy here is that the tools and techniques you're describing can (and should) be used diligently to solve this problem, and Tach is often a complement to this approach. The benefit of centralizing the concern into a single tool, and often a single config file, is that teams get better documentation, earlier feedback (in-editor vs. code review), and more visibility when planning new development. Teams also get to choose _how_ they would like to satisfy Tach's config, and other teams can still rely on the same guarantees due to Tach's static checks.
- echelon 2y ago
  > We have well established conventions like prefixing private modules and symbols with an underscore, or declaring your public interfaces in the __init__.py file, The language doesn't enforce them, so they may as well not exist. See: python dependency management. > This is more commentary on the Python developer than this tool. 100%. Python has become an unstructured Wild West, perhaps even worse than modern JavaScript. The "Zen of Python" is a bold faced lie. Python has incredible use cases. It blends together different disciplines effectively. But perhaps we should ask ourselves whether or not it's a language suitable for writing large monoliths in.
  - VerminOctopus1 2y ago
    The conventions are widely used and Python is used successfully in numerous “large monoliths”. Saying that the conventions may as well not exist if they’re not enforced is demonstrably nonsense.
adamc 2y ago
Having the example be a video that changes was confusing at first, and if you are going to show me something that is changing, I would like to be able to rewind to the beginning. But really I just think it's a bad idea to show something like that without making it obvious what it is.
- the1024 2y ago
  Thanks for this feedback! Here's a more structured walkthrough: https://www.loom.com/share/7c03d72ea2b54212a3509d4333f61b99?t=1 https://www.loom.com/share/7c03d72ea2b54212a3509d4333f61b99?... We'll add this to the README.
- mathfailure 2y ago
  What do you mean by "a video that changes"? I liked the video-example, it's way better than examples in many projects that use just text, when the tool does something quite complex that better be demonstrated in a video with a narrator explaining what's happening and why.
  - 0x63_Problems 2y ago
    I actually made that change in response to their comment! It used to be a live GIF with no narration.
    - mathfailure 2y ago
      Thank you for the change then. I liked the video with narration.
benrutter 2y ago
This looks nice! I vaguely know Grimp as a similar tool, any idea how they differ/compare?
- the1024 2y ago
  Thanks! You can think of Grimp as a lower-level tool for interacting with the import graph in Python, while Tach is a high-level tool responsible for 'modularity' as a whole (e.g. modules, interfaces, layers, deprecations etc.) Tach is also more opinionated - so it doesn't require you to write any custom code, and uses declarative config to enforce your desired architecture.
tracnar 2y ago
When I tried it, it seemed like you really need to list all modules in `tach.toml`. What I wanted was to work at a coarser package level. For example if you have the modules `foo.a`, `foo.b`, `bar.a`, and `bar.b`, I'd like a rule that `bar` can import from `foo` but not vice versa, without having to list or care about the submodules. Is that something you'd want to support?
- 0x63_Problems 2y ago
  This should definitely be supported out-of-the-box, we can take a closer look in Discord or through GH Issues! But generally I would expect: ``` [[modules]] path = "foo" depends_on = [] [[modules]] path = "bar" depends_on = ["foo"] ``` would do the trick, assuming both are within a configured source root, and their children are _not_ also marked as modules. If the children are marked as modules, their dependency rules are treated separately and wouldn't automatically inherit from a parent.
  - tracnar 2y ago
    Ok great, I indeed just tried, I might just have been confused last time! Maybe you could mention in the doc that a module can be a "package"? (Even though I suppose a package is also a module, I always find the Python terminology a bit confusing there.)
    - rswail 2y ago
      I always used to think of it as a module is a file `example.py`, a package is a directory `example` with a module `__init__.py`.
stavros 2y ago
We replaced our microservices architecture with a modular monolith and got tons of benefits, something I've been meaning to write up. However, while discussing that here, I was pointed to Tach, which looks fantastic if you have (or want to create) a modular monolith.
rubenvanwyk 2y ago
Does this work together with uv?
- 0x63_Problems 2y ago
  Yes! Here is an example project that is configured as if it were a uv workspace: https://github.com/gauge-sh/tach/tree/main/python/tests/example/monorepo_workspace https://github.com/gauge-sh/tach/tree/main/python/tests/exam... In that project, `tach check-external` would handle between-workspace dependencies, while the core `modules` and `interfaces` config would handle within-workspace dependencies. Soon these will be better unified, we kept the 1st-party/3rd-party distinction separate while we learned what the UX should be.
Attummm 2y ago
This sounds great. Python is really great for quickly developing applications. However, maintaining them is a real pain point—especially when it comes to packages and their dependencies. Furthermore, because there isn’t a compile-time checker, function or method signatures can change unnoticed. Compilers are great for catching such issues at compile time rather than at runtime. Python does have mypy, which can play that role, but the package must support it. Currently, you are dependent on the package maintainer regarding their adherence to semver. Maybe this project will be able to fill that hole.
drdrey 2y ago
I would recommend installing it with uv tool install tach rather than pip install tach that way tach is installed system-wide but in its own isolated venv
- globular-toast 2y ago
  I keep meaning to write a blog post about this. "pip install x" basically should be read as "this project is packaged and available on PyPI" rather than a literal installation instruction. A seasoned Python developer will rarely, if ever, directly pip install something. Instead they would manually add it to pyproject.toml or, if they use something like poetry, use that to add it, or they'd use something like pipx to install it as a "global" tool (or just their system package manager). This has been true for years now. I think it's time projects stop writing "pip install x" and we come up with a standard way to say "the package name is x" and maybe a recommended installation method (like use uv/pipx to install as a system tool, or add to your project dependencies etc).
- adammarples 2y ago
  Venv or no Venv, you can use pip. Asking people to have an additional requirement on uv is unreasonable, although I do prefer using it now myself.
godelski 2y ago
I've been surprised that there hasn't been much progress in code tracing. It's incredibly hard to jump into a new code base. Cscope and ctags are still used but uncommon. It's not common to see people use debuggers. I suspect this is a major reason why Python and interpreted languages are so popular. But as code bases have exploded we still haven't gotten much better than where we were over a decade ago. Yeah RAGs can help but I'm not sure are a huge improvement over tags. Maybe realistically regressed, relying more on print statement debugging. When do we improve from gdb, cscope, grep, awk, and find? (I'm aware of the improved version of those but listen carefully if that's what you're jumping to respond with) So I'm really happy to see a project like this. Well done. Can't wait to see more
butterlettuce 2y ago
Just wanted to say Caelean is such a cool name. Is it “kay-leen” or “kay-lee-an”
- the1024 2y ago
  "kay-len" - confusing I know
willgax 2y ago
do project like this exist for java spring boot it would be very cool if it could work on every codebase regardless of tech stack
- ameymh1571 2y ago
  yeah i would like to know that too as a to get big picture of how everything works it is needed
efitz 2y ago
What kind of data set are you trying to build using the dependency information from collected toml files, and what do you intend to do with it?
yogurt-male 2y ago
This is great! Really wish that I'd known about this tool a month ago. Would have saved me a lot of headache. Thanks for sharing.
xtiansimon 2y ago
Would this also apply to learning and exploring an unknown codebase?
chairhairair 2y ago
If this tool looks like it would improve your life I think you should consider using Bazel instead of whatever build system you are using. I don’t see much value add here for a project using Bazel.
KronisLV 2y ago
SourceTrail did something a bit similar, though it didn't focus just on the modules, but also what methods are called where. I really liked the tool, but it didn't really work out for them and they discontinued it: https://en.wikipedia.org/wiki/Sourcetrail https://en.wikipedia.org/wiki/Sourcetrail

263 points by the1024 2y ago

jtwaleson 2y ago

Cool! Do you have plans to launch a paid offering? The website makes me think it's a company but I didn't see any pricing / sales details.

0x63_Problems 2y ago
Co-author here - We do provide a web platform (https://www.gauge.sh/platform https://www.gauge.sh/platform) which we have been developing with design partners. The fundamental difference between using Tach alone vs. the platform is that the platform provides incremental enforcement at the pull request level. We're always happy to chat about adding more design partners! email: founders@gauge.sh

vednig 2y ago

This is pretty cool, thanks for sharing

hansonkd 2y ago

Really excited to see this project gain traction. > Note that this graph is generated remotely with the contents of your `tach.toml` Isn't shipping off parts of your codebase to a 3rd party without warning in the CLI a security risk? Or in regulatory environments you get audited that your code was only stored on properly vetted services which is why some sales cycles for AI coding assistant tools are so long. It would be kind of frustrating to have something like that happen and get set back on licensing, etc. Just from the video it doesn't seem like any sort of warning that you are shipping config files to your servers and the URL that you produced doesn't seem to have any authentication. Maybe i am misunderstanding that functionality, but it gives me pause to use it.

0x63_Problems 2y ago
Co-author here, fair question! In short, we want to make the visualization UX as smooth as possible, and this is best done with a web app. The URLs use UUIDs, and the contents being sent don't include literal source code, only module names and Tach configuration. We will also delete graphs by UUID on request, and have done so in the past. That said, we do try to be up-front about this, which is why that disclaimer exists, and when running this command on the CLI, you must supply an explicit `--web` argument to `tach show`. Otherwise, the default behavior is to generate a GraphViz DOT file locally.
- bmitc 2y ago
  Why not just let users run the web app locally? There's no reason it needs to be remote. Also, the mere fact that it sends any data, no matter what you say it contains is a non-starter at many places. And even module names can contain proprietary data.
  - 0x63_Problems 2y ago
    I can understand the frustration, but I think there are legitimate reasons to run this remotely. Tach is an installable Python package, shipping a full web app would have to come in a separate form factor and has significant maintenance implications. Given we are explicit about the remote app before anything is sent, require explicit opt-in, and we provide usable alternatives locally, we prioritize shipping a useful graph experience that is immediately usable. If you are at an enterprise that cannot tolerate this, then you can use a local viewer with either GraphViz DOT format or Mermaid which is generated by using `tach show` or `tach show --mermaid` respectively.

lijok 2y ago

Tools like this rub me the wrong way. We have well established conventions like prefixing private modules and symbols with an underscore, or declaring your public interfaces in the __init__.py file, but the Python developer decries it as "busywork", "weird" and "hard to read", so we instead use tools like this. We can manage dependencies with protocols, a type checker and generally following SOLID principles, but the Python developer decries it as "too indirect and convoluted", so we instead use tools like this. This is more commentary on the Python developer than this tool. Tach looks great.

0x63_Problems 2y ago
Co-author here, I can understand where you're coming from! Part of the philosophy here is that the tools and techniques you're describing can (and should) be used diligently to solve this problem, and Tach is often a complement to this approach. The benefit of centralizing the concern into a single tool, and often a single config file, is that teams get better documentation, earlier feedback (in-editor vs. code review), and more visibility when planning new development. Teams also get to choose _how_ they would like to satisfy Tach's config, and other teams can still rely on the same guarantees due to Tach's static checks.
echelon 2y ago
> We have well established conventions like prefixing private modules and symbols with an underscore, or declaring your public interfaces in the __init__.py file, The language doesn't enforce them, so they may as well not exist. See: python dependency management. > This is more commentary on the Python developer than this tool. 100%. Python has become an unstructured Wild West, perhaps even worse than modern JavaScript. The "Zen of Python" is a bold faced lie. Python has incredible use cases. It blends together different disciplines effectively. But perhaps we should ask ourselves whether or not it's a language suitable for writing large monoliths in.
- VerminOctopus1 2y ago
  The conventions are widely used and Python is used successfully in numerous “large monoliths”. Saying that the conventions may as well not exist if they’re not enforced is demonstrably nonsense.

adamc 2y ago

Having the example be a video that changes was confusing at first, and if you are going to show me something that is changing, I would like to be able to rewind to the beginning. But really I just think it's a bad idea to show something like that without making it obvious what it is.

the1024 2y ago
Thanks for this feedback! Here's a more structured walkthrough: https://www.loom.com/share/7c03d72ea2b54212a3509d4333f61b99?t=1 https://www.loom.com/share/7c03d72ea2b54212a3509d4333f61b99?... We'll add this to the README.
mathfailure 2y ago
What do you mean by "a video that changes"? I liked the video-example, it's way better than examples in many projects that use just text, when the tool does something quite complex that better be demonstrated in a video with a narrator explaining what's happening and why.
- 0x63_Problems 2y ago
  I actually made that change in response to their comment! It used to be a live GIF with no narration.
  - mathfailure 2y ago
    Thank you for the change then. I liked the video with narration.

benrutter 2y ago

This looks nice! I vaguely know Grimp as a similar tool, any idea how they differ/compare?

the1024 2y ago
Thanks! You can think of Grimp as a lower-level tool for interacting with the import graph in Python, while Tach is a high-level tool responsible for 'modularity' as a whole (e.g. modules, interfaces, layers, deprecations etc.) Tach is also more opinionated - so it doesn't require you to write any custom code, and uses declarative config to enforce your desired architecture.

tracnar 2y ago

When I tried it, it seemed like you really need to list all modules in `tach.toml`. What I wanted was to work at a coarser package level. For example if you have the modules `foo.a`, `foo.b`, `bar.a`, and `bar.b`, I'd like a rule that `bar` can import from `foo` but not vice versa, without having to list or care about the submodules. Is that something you'd want to support?

0x63_Problems 2y ago
This should definitely be supported out-of-the-box, we can take a closer look in Discord or through GH Issues! But generally I would expect: ``` [[modules]] path = "foo" depends_on = [] [[modules]] path = "bar" depends_on = ["foo"] ``` would do the trick, assuming both are within a configured source root, and their children are _not_ also marked as modules. If the children are marked as modules, their dependency rules are treated separately and wouldn't automatically inherit from a parent.
- tracnar 2y ago
  Ok great, I indeed just tried, I might just have been confused last time! Maybe you could mention in the doc that a module can be a "package"? (Even though I suppose a package is also a module, I always find the Python terminology a bit confusing there.)
  - rswail 2y ago
    I always used to think of it as a module is a file `example.py`, a package is a directory `example` with a module `__init__.py`.

stavros 2y ago

We replaced our microservices architecture with a modular monolith and got tons of benefits, something I've been meaning to write up. However, while discussing that here, I was pointed to Tach, which looks fantastic if you have (or want to create) a modular monolith.

rubenvanwyk 2y ago

Does this work together with uv?

0x63_Problems 2y ago
Yes! Here is an example project that is configured as if it were a uv workspace: https://github.com/gauge-sh/tach/tree/main/python/tests/example/monorepo_workspace https://github.com/gauge-sh/tach/tree/main/python/tests/exam... In that project, `tach check-external` would handle between-workspace dependencies, while the core `modules` and `interfaces` config would handle within-workspace dependencies. Soon these will be better unified, we kept the 1st-party/3rd-party distinction separate while we learned what the UX should be.

Attummm 2y ago

This sounds great. Python is really great for quickly developing applications. However, maintaining them is a real pain point—especially when it comes to packages and their dependencies. Furthermore, because there isn’t a compile-time checker, function or method signatures can change unnoticed. Compilers are great for catching such issues at compile time rather than at runtime. Python does have mypy, which can play that role, but the package must support it. Currently, you are dependent on the package maintainer regarding their adherence to semver. Maybe this project will be able to fill that hole.

drdrey 2y ago

I would recommend installing it with uv tool install tach rather than pip install tach that way tach is installed system-wide but in its own isolated venv

globular-toast 2y ago
I keep meaning to write a blog post about this. "pip install x" basically should be read as "this project is packaged and available on PyPI" rather than a literal installation instruction. A seasoned Python developer will rarely, if ever, directly pip install something. Instead they would manually add it to pyproject.toml or, if they use something like poetry, use that to add it, or they'd use something like pipx to install it as a "global" tool (or just their system package manager). This has been true for years now. I think it's time projects stop writing "pip install x" and we come up with a standard way to say "the package name is x" and maybe a recommended installation method (like use uv/pipx to install as a system tool, or add to your project dependencies etc).
adammarples 2y ago
Venv or no Venv, you can use pip. Asking people to have an additional requirement on uv is unreasonable, although I do prefer using it now myself.

godelski 2y ago

I've been surprised that there hasn't been much progress in code tracing. It's incredibly hard to jump into a new code base. Cscope and ctags are still used but uncommon. It's not common to see people use debuggers. I suspect this is a major reason why Python and interpreted languages are so popular. But as code bases have exploded we still haven't gotten much better than where we were over a decade ago. Yeah RAGs can help but I'm not sure are a huge improvement over tags. Maybe realistically regressed, relying more on print statement debugging. When do we improve from gdb, cscope, grep, awk, and find? (I'm aware of the improved version of those but listen carefully if that's what you're jumping to respond with) So I'm really happy to see a project like this. Well done. Can't wait to see more

butterlettuce 2y ago

Just wanted to say Caelean is such a cool name. Is it “kay-leen” or “kay-lee-an”

the1024 2y ago
"kay-len" - confusing I know

willgax 2y ago

do project like this exist for java spring boot it would be very cool if it could work on every codebase regardless of tech stack

ameymh1571 2y ago
yeah i would like to know that too as a to get big picture of how everything works it is needed

efitz 2y ago

What kind of data set are you trying to build using the dependency information from collected toml files, and what do you intend to do with it?

yogurt-male 2y ago

This is great! Really wish that I'd known about this tool a month ago. Would have saved me a lot of headache. Thanks for sharing.

xtiansimon 2y ago

Would this also apply to learning and exploring an unknown codebase?

chairhairair 2y ago

If this tool looks like it would improve your life I think you should consider using Bazel instead of whatever build system you are using. I don’t see much value add here for a project using Bazel.

KronisLV 2y ago

SourceTrail did something a bit similar, though it didn't focus just on the modules, but also what methods are called where. I really liked the tool, but it didn't really work out for them and they discontinued it: https://en.wikipedia.org/wiki/Sourcetrail https://en.wikipedia.org/wiki/Sourcetrail