5 ms·

Diagram as Code

151 points by sebg 4y ago

abraxas 4y ago
I'm not digging this. I don't think a full Turing complete language syntax adds much to a diagram tool. I much prefer the conciseness of the syntax of Mermaid compared to this.
SaulJLH 4y ago
Already posted https://news.ycombinator.com/item?id=31807021 https://news.ycombinator.com/item?id=31807021 EDIT Actually, might be different...
- tesrx 4y ago
  That is different. GH link vs YT video.
- Veuxdo 4y ago
  No, its been posted several times already: https://news.ycombinator.com/from?site=github.com/mingrammer https://news.ycombinator.com/from?site=github.com/mingrammer
sixstringtheory 4y ago
Pretty cool. The biggest thing I think is missing from many popular diagramming tools is the ability to easily diff the architecture across changes. Although that can already be done with graphviz source files. However, these days I prefer to just have diagrams generated from the source code itself, like: - https://github.com/cycloidio/inframap https://github.com/cycloidio/inframap - https://graphviz.org/Gallery/directed/bazel.html https://graphviz.org/Gallery/directed/bazel.html In order to make a compelling use case out of having to maintain a parallel definition like this, I think it needs to be able to contribute to analysis like formal verification or, despite them stating that it's not a goal, being able to create the described architecture. Or even generate something like a terraform plan. Otherwise it falls victim to the same problem as any other method of creating diagrams, that of them falling out of sync with the system. They are a long way there already by providing a way to use a programming language backed by a large ecosystem of other tools that can be used to work with e.g. terraform, like python/go.
- beardedwizard 4y ago
  Isn't it too late for the diagram if the code is already written? I think many diagram use cases are about informing which code should be written to begin with. For understanding an existing thing, diagram from code seems great.
  - sixstringtheory 4y ago
    Depends on the reason for having a diagram. If you need to work on a piece of existing code you’ve never seen before, it can be very helpful to have a diagram as a guide. If I’m prototyping, I’m probably just going to draw a diagram on a whiteboard, not write code to create one. That’s bringing way too much cognitive overhead into the process that will distract from the task at hand.
guessmyname 4y ago
A year ago, someone ported this Diagrams (Python) into Go — https://www.reddit.com/r/golang/comments/is8eo9/godiagrams_create_architecture_diagrams_with_go/ https://www.reddit.com/r/golang/comments/is8eo9/godiagrams_c... The Python→Go port was discussed on Sep 27, 2020 — https://news.ycombinator.com/item?id=24603926 https://news.ycombinator.com/item?id=24603926 Someone was also working on a web editor (maybe with Electron), but I forgot the name of the repository.
wmil 4y ago
Is it common for python libraries these days to overload the >> operator for oddball things like C++ does? Not doing crap like that is one of the things that I loved about Python...
- detaro 4y ago
  > Is it common for python libraries these days to overload the >> operator for oddball things like C++ does? no
- neuronexmachina 4y ago
  The Numpy and Tensorflow libraries do a fair bit of operator overloading, but it's IMHO more sensible than this example.
- tgarv 4y ago
  Airflow uses it for task ordering within a DAG, but that's used in the parsing step that happens before/outside of any actual task execution so it's not really in the source code of tasks at least.
  - Noumenon72 4y ago
    The Airflow 2.0 TaskFlow API[1] doesn't use the bitshift operators any more -- if you pass the result of one task into another, the second task is automatically downstream of the first. So it all just looks like function calls, despite using XComArgs behind the scenes. 1. https://airflow.apache.org/docs/apache-airflow/stable/tutorial_taskflow_api.html https://airflow.apache.org/docs/apache-airflow/stable/tutori...
- shabble 4y ago
  if you liked that, you'll love Plumbum[1] :) [1] https://github.com/tomerfiliba/plumbum https://github.com/tomerfiliba/plumbum
  - marginalia_nu 4y ago
    I'm waiting for Guido to rush in any moment going "Stop right there, criminal scum! Nobody breaks the law on my watch!"
j4mie 4y ago
from diagrams import Diagram from diagrams.aws.compute import EC2 with Diagram("Simple Diagram"): EC2("web") This has a very odd API. It's using (abusing?) context managers and contextvars to do weird spooky things that you could just as easily do with ordinary objects or functions.
- VectorLock 4y ago
  I think you can still use ordinary functions to string things together as well. I also don't like the abuse of context managers in this way.
mkl95 4y ago
Interesting, although I'm way more excited about the inverse (code as diagram).
AlbertCory 4y ago
I was ahead of my time. My M.S. thesis was "A flowchart-to-source-code conversion program for GPSS/360." I think those garbage "academia" sites are constantly emailing, asking me to "claim" it.
- beachy 4y ago
  I worked in CASE product team at Oracle last century. We had a significant development effort around autolayout algorithms. It's funny seeing all the same discussion decades later.
  - AlbertCory 4y ago
    We might have overlapped at Oracle. I left in Jan. 1998.
zander312 4y ago
Love the idea, definitely will use. But wondering why it's using ">>". I'm not great with Python. How does this ">>" thing work and why does it make sense to associate elements with ">>".
- aidos 4y ago
  Python has dunder methods that allow you to override behaviour (eg operator overloading, hashing, object identity etc). In this case they’re overriding bitshifting. In terms of why, it does feel like a natural visual choice. https://github.com/mingrammer/diagrams/blob/master/diagrams/__init__.py#L346 https://github.com/mingrammer/diagrams/blob/master/diagrams/...
  - AtlasBarfed 4y ago
    operator overloading. One of the great holy wars of PL. Of course C++ has it, and it is one of the great footguns in its arsenal.
    - aidos 4y ago
      With great power comes great responsibility.
shabble 4y ago
I love the idea of autogenerating diagrams from either a schema or from some sort of actual definitions (terraform/cloud inspection/etc), but I've generally found that the layout can be pretty poor for any semi-complex system. Even worse, a small change to the structure will typically cause a full re-layout, and place nodes completely differently to the previous output, making visual/mental comparisons between versions quite taxing. Given that autolayout is likely never going to be as good as someone doing some amount of manual layout or at least hinting, it'd be nice if there was some way to allow explicit placement of items and track it as metadata independently of the model changes. I think Structurizr[1] has some capabilities along these lines, but the editor wasn't great the last time I tried it, and I think largely requires their SaaS for tracking that layout info. Another option might be to feed it all into some flavour of graph DB with interactive querying/filtering and layout, and let users produce just the diffs / bits that they care about, if practical. [1] https://structurizr.com/ https://structurizr.com/
- TranquilMarmot 4y ago
  I agree with this. I'm usually not a fan of auto-generated diagrams at all. I would rather spend the extra time making something that's more readable and has a better layout. The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)
  - shabble 4y ago
    There's absolutely value in being able to partially generate (maybe just the nodes/edges, but without final layout), or possibly even just validate that a given diagram does actually match a given schema/definition, without necessarily solving the extremely hard problem of a good looking auto-layout. I've dabbled in the past with trying to do auto-layout of new items while preserving explicit changes by hacking at some of the Cytoscape.js layout providers to heavily penalise moves to manually positioned nodes, but I've never managed to make anything close to what I'd call acceptable for general use.
  - Veuxdo 4y ago
deltree7 4y ago
The next step would be to make it declarative like PlantUML or Ditaa. Surely, my diagram shouldn't have programming crud around it
bilalq 4y ago
I'm always on the lookout for more diagram as code authoring options. Mermaid is pretty decent, if frustrating at times. Looks like there's a port of it to TypeScript: https://www.npmjs.com/package/diagrams-ts https://www.npmjs.com/package/diagrams-ts
alexott 4y ago
What is the advantage over PlantUML, for example? Only syntax?
Yhippa 4y ago
I actually kind of like this. Would be neat if there was a way once the initial diagram is generated to visually edit it.
nikolay 4y ago
I tried doing an AWS systems architecture and it looked so ugly, so, we switcehd to Cloudcraft [0], which does both isometric 3D and 2D. All Terraform tools like inframap were terrible, too. It's 2022 and we still don't have pretty autodiagraming tools - we have ML generating unimaginable in terms of complexity photorealistic images, but it can't generate a pretty diagram yet! [0]: https://www.cloudcraft.co/ https://www.cloudcraft.co/

151 points by sebg 4y ago

I'm not digging this. I don't think a full Turing complete language syntax adds much to a diagram tool. I much prefer the conciseness of the syntax of Mermaid compared to this.

SaulJLH 4y ago

Already posted https://news.ycombinator.com/item?id=31807021 https://news.ycombinator.com/item?id=31807021 EDIT Actually, might be different...

tesrx 4y ago
That is different. GH link vs YT video.
Veuxdo 4y ago
No, its been posted several times already: https://news.ycombinator.com/from?site=github.com/mingrammer https://news.ycombinator.com/from?site=github.com/mingrammer

sixstringtheory 4y ago

Pretty cool. The biggest thing I think is missing from many popular diagramming tools is the ability to easily diff the architecture across changes. Although that can already be done with graphviz source files. However, these days I prefer to just have diagrams generated from the source code itself, like: - https://github.com/cycloidio/inframap https://github.com/cycloidio/inframap - https://graphviz.org/Gallery/directed/bazel.html https://graphviz.org/Gallery/directed/bazel.html In order to make a compelling use case out of having to maintain a parallel definition like this, I think it needs to be able to contribute to analysis like formal verification or, despite them stating that it's not a goal, being able to create the described architecture. Or even generate something like a terraform plan. Otherwise it falls victim to the same problem as any other method of creating diagrams, that of them falling out of sync with the system. They are a long way there already by providing a way to use a programming language backed by a large ecosystem of other tools that can be used to work with e.g. terraform, like python/go.

beardedwizard 4y ago
Isn't it too late for the diagram if the code is already written? I think many diagram use cases are about informing which code should be written to begin with. For understanding an existing thing, diagram from code seems great.
- sixstringtheory 4y ago
  Depends on the reason for having a diagram. If you need to work on a piece of existing code you’ve never seen before, it can be very helpful to have a diagram as a guide. If I’m prototyping, I’m probably just going to draw a diagram on a whiteboard, not write code to create one. That’s bringing way too much cognitive overhead into the process that will distract from the task at hand.

guessmyname 4y ago

A year ago, someone ported this Diagrams (Python) into Go — https://www.reddit.com/r/golang/comments/is8eo9/godiagrams_create_architecture_diagrams_with_go/ https://www.reddit.com/r/golang/comments/is8eo9/godiagrams_c... The Python→Go port was discussed on Sep 27, 2020 — https://news.ycombinator.com/item?id=24603926 https://news.ycombinator.com/item?id=24603926 Someone was also working on a web editor (maybe with Electron), but I forgot the name of the repository.

wmil 4y ago

Is it common for python libraries these days to overload the >> operator for oddball things like C++ does? Not doing crap like that is one of the things that I loved about Python...

detaro 4y ago
> Is it common for python libraries these days to overload the >> operator for oddball things like C++ does? no
neuronexmachina 4y ago
The Numpy and Tensorflow libraries do a fair bit of operator overloading, but it's IMHO more sensible than this example.
tgarv 4y ago
Airflow uses it for task ordering within a DAG, but that's used in the parsing step that happens before/outside of any actual task execution so it's not really in the source code of tasks at least.
- Noumenon72 4y ago
  The Airflow 2.0 TaskFlow API[1] doesn't use the bitshift operators any more -- if you pass the result of one task into another, the second task is automatically downstream of the first. So it all just looks like function calls, despite using XComArgs behind the scenes. 1. https://airflow.apache.org/docs/apache-airflow/stable/tutorial_taskflow_api.html https://airflow.apache.org/docs/apache-airflow/stable/tutori...
shabble 4y ago
if you liked that, you'll love Plumbum[1] :) [1] https://github.com/tomerfiliba/plumbum https://github.com/tomerfiliba/plumbum
- marginalia_nu 4y ago
  I'm waiting for Guido to rush in any moment going "Stop right there, criminal scum! Nobody breaks the law on my watch!"

j4mie 4y ago

from diagrams import Diagram from diagrams.aws.compute import EC2 with Diagram("Simple Diagram"): EC2("web") This has a very odd API. It's using (abusing?) context managers and contextvars to do weird spooky things that you could just as easily do with ordinary objects or functions.

VectorLock 4y ago
I think you can still use ordinary functions to string things together as well. I also don't like the abuse of context managers in this way.

mkl95 4y ago

Interesting, although I'm way more excited about the inverse (code as diagram).

AlbertCory 4y ago

I was ahead of my time. My M.S. thesis was "A flowchart-to-source-code conversion program for GPSS/360." I think those garbage "academia" sites are constantly emailing, asking me to "claim" it.

beachy 4y ago
I worked in CASE product team at Oracle last century. We had a significant development effort around autolayout algorithms. It's funny seeing all the same discussion decades later.
- AlbertCory 4y ago
  We might have overlapped at Oracle. I left in Jan. 1998.

zander312 4y ago

Love the idea, definitely will use. But wondering why it's using ">>". I'm not great with Python. How does this ">>" thing work and why does it make sense to associate elements with ">>".

aidos 4y ago
Python has dunder methods that allow you to override behaviour (eg operator overloading, hashing, object identity etc). In this case they’re overriding bitshifting. In terms of why, it does feel like a natural visual choice. https://github.com/mingrammer/diagrams/blob/master/diagrams/__init__.py#L346 https://github.com/mingrammer/diagrams/blob/master/diagrams/...
- AtlasBarfed 4y ago
  operator overloading. One of the great holy wars of PL. Of course C++ has it, and it is one of the great footguns in its arsenal.
  - aidos 4y ago
    With great power comes great responsibility.

shabble 4y ago

I love the idea of autogenerating diagrams from either a schema or from some sort of actual definitions (terraform/cloud inspection/etc), but I've generally found that the layout can be pretty poor for any semi-complex system. Even worse, a small change to the structure will typically cause a full re-layout, and place nodes completely differently to the previous output, making visual/mental comparisons between versions quite taxing. Given that autolayout is likely never going to be as good as someone doing some amount of manual layout or at least hinting, it'd be nice if there was some way to allow explicit placement of items and track it as metadata independently of the model changes. I think Structurizr[1] has some capabilities along these lines, but the editor wasn't great the last time I tried it, and I think largely requires their SaaS for tracking that layout info. Another option might be to feed it all into some flavour of graph DB with interactive querying/filtering and layout, and let users produce just the diffs / bits that they care about, if practical. [1] https://structurizr.com/ https://structurizr.com/

TranquilMarmot 4y ago
I agree with this. I'm usually not a fan of auto-generated diagrams at all. I would rather spend the extra time making something that's more readable and has a better layout. The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)
- shabble 4y ago
  There's absolutely value in being able to partially generate (maybe just the nodes/edges, but without final layout), or possibly even just validate that a given diagram does actually match a given schema/definition, without necessarily solving the extremely hard problem of a good looking auto-layout. I've dabbled in the past with trying to do auto-layout of new items while preserving explicit changes by hacking at some of the Cytoscape.js layout providers to heavily penalise moves to manually positioned nodes, but I've never managed to make anything close to what I'd call acceptable for general use.
- Veuxdo 4y ago

deltree7 4y ago

The next step would be to make it declarative like PlantUML or Ditaa. Surely, my diagram shouldn't have programming crud around it

bilalq 4y ago

I'm always on the lookout for more diagram as code authoring options. Mermaid is pretty decent, if frustrating at times. Looks like there's a port of it to TypeScript: https://www.npmjs.com/package/diagrams-ts https://www.npmjs.com/package/diagrams-ts

alexott 4y ago

What is the advantage over PlantUML, for example? Only syntax?

Yhippa 4y ago

I actually kind of like this. Would be neat if there was a way once the initial diagram is generated to visually edit it.

nikolay 4y ago

I tried doing an AWS systems architecture and it looked so ugly, so, we switcehd to Cloudcraft [0], which does both isometric 3D and 2D. All Terraform tools like inframap were terrible, too. It's 2022 and we still don't have pretty autodiagraming tools - we have ML generating unimaginable in terms of complexity photorealistic images, but it can't generate a pretty diagram yet! [0]: https://www.cloudcraft.co/ https://www.cloudcraft.co/