4 ms·

Hi HN, I'm Alex, at Terrastruct, where we've been making D2. This actually popped up on HN a couple months back, though it wasn't ready, e.g. not open source ye

Hi HN, I'm Alex, at Terrastruct, where we've been making D2. This actually popped up on HN a couple months back, though it wasn't ready, e.g. not open source yet. It is now! We also put up a site for you to compare D2 with MermaidJS, Graphviz, and PlantUML: https://text-to-diagram.com https://text-to-diagram.com. Full disclosure, we're a for-profit company. The open-core part is that we make an alternative layout engine which we sell (Jetbrains model, i.e. your copy is your's forever if you've paid for 12+ months). It's not packaged with D2, so you won't see it if you don't want it. D2 is perfectly usable without it, and integrates with multiple free open source layout engines (e.g. the one that Mermaid uses, "dagre", is D2's default). If you want to read more about our plans for D2: https://d2lang.com/tour/future https://d2lang.com/tour/future. Hope you can check it out! It's got an easy install (and uninstall) process.

deleted 4y ago
[deleted]
wg0 4y ago
Beautiful! And then in golang. Thank you for open sourcing the core part.
mooneater 4y ago
20+ year fan of graphviz here, love seeing what you are doing with D2! Any examples of larger diagrams (1000+s of nodes) and how does that perform in layout?
- alixanderwang 4y ago
  thank you! There isn't right now, we'd need to add a simpler layout algorithm for those, e.g. force-directed, radial, etc. That use case is not something D2 is trying to tackle though, we're just focused on software architecture diagrams right now, and I've yet to see one span that many nodes
  - nonrandomstring 4y ago
    Picking a set of domain specific targets and honing a tool to service that seems a good way to start building - rather than a general solver. I also love graphviz but had a project to create an app for legible quality management, and it was a headache to shoehorn some concepts into "nodes in an edge connected graph" way of thinking (ended up programatically scripting LaTeX). With so many visualisation concepts there's room for niche approaches (same in plotting; Matlab/Octave vs Gnuplot vs Pyplot vs R and so on)
    - alixanderwang 4y ago
      agreed! that reminds me, there's a company that specializes in this niche of visualizing large data: https://www.graphistry.com/ https://www.graphistry.com/ (haven't used it myself, looks well-made).
      - lmeyerov 4y ago
        100% agreement, spot on observations! Indeed, we end up referring folks to cool diagramming tools all the time at Graphistry as d2/mermaid/diagrams.net are well-optimized for quick & beautiful diagram presentation tasks on manageably sized & fairly static datasets. Using Graphistry to do a quick markdown of how a 4 node cluster might work is a bit like driving a tank to pick up some milk :) Teams go to us more for gnarly investigation & splunking tasks that need a visual power tool, like looking at alert logs or big systems, so we optimize for making that scale interactive & easy. The overlap is real however, so a lot of room for teams to learn. Ex: We're the easiest graph tool for jupyter/databricks/streamlit etc teams who use dataframes, and I can imagine those tools learning from us here. In the reverse direction, our work more in terms of quickly configurable global data<>viz data bindings ("using the UI or API, bind each event's score to a hot-and-cold coloring palette and use a warning icon on all type=alert events"), but we have a ways to go to support the more manual artisinal effects of diagramming tools like Figma, where each element might have a super fancy & unique border style.
  - deathanatos 4y ago
    IDK if it's quite the use case you're looking for, but at my company, we turn our CI YAML for our monorepo into Dot (Graphviz), and graph it, to visualize the dependencies between steps in our CI, & to highlight the critical path. It's an ~100 node DAG. Graphviz struggles a bit with it, particularly with edge layout. (Though really, the layout I want for it is a Gnatt chart style layout.)
- riedel 4y ago
  I am always quite astonished how bad the default layouters for graphs perform. When I was still doing compiler optimization in the beginning of the 2000s we did not struggle with quite big graphs thanks to cool graph visualizers such as vcg [1] . Two weeks ago I was tempted to try it again after nearly 20 years because I was frustrated even visualising a relatively small graph in python (cytoscape seemed to be the only working software in the end but it was quite a pain to get it to just render what I wanted) [1] https://www.rw.cdl.uni-saarland.de/people/sander/private/html/gsvcg1.html https://www.rw.cdl.uni-saarland.de/people/sander/private/htm...
  - ygra 4y ago
    Laying out larger graphs is tricky, often because the size simply stands in the way of generating anything that's useful to the viewer. To add to that, most layout algorithms prioritize optimizing certain criteria, while other parts of the visualization emerge from that; and if the human viewer chooses a layout algorithm because of one of the latter properties, they are often surprised that the result doesn't look like they envisioned (because, hey, the algorithm optimized something entirely different). We see this disconnect fairly often in our own customer support, but I haven't really found a good way of putting an explanation in writing. Then there's generally the problem of larger graphs, which tend to devolve into a tangled mess and hairballs, simply because they often tend to be well-connected. If there's no way of pruning them beforehand, or perhaps grouping, aggregating or clustering (in a way that makes sense to the viewer, not necessarily only structurally), then it can be hard to get good result.
JonChesterfield 4y ago
Is the language described anywhere? Couldn't find an example other than the one on text-to-diagram and can't find anything beyond that. edit: guessing URL found https://d2lang.com/tour/intro/ https://d2lang.com/tour/intro/ which covers it, perhaps worth linking to there from the GitHub page edit2: ok it is linked, just above the section I was looking at: Reference A more formal reference specification is in the works.
localhost 4y ago
This is a great idea; my go-to approach is the "I'll use PowerPoint to do that and take a screenshot" mindset since I don't have a need for publication quality output. I really like the comparison site that you have created - it's great for learning these languages. I would love to see a plugin for Obsidian for the times where I need to add a custom drawing to my notes. It looks like there already are plugins for PlantUML[1] and graphviz[2] today. [1] https://github.com/QAMichaelPeng/obsidian-graphviz https://github.com/QAMichaelPeng/obsidian-graphviz [2] https://github.com/joethei/obsidian-plantuml https://github.com/joethei/obsidian-plantuml
- alixanderwang 4y ago
  thanks! i don't use Obsidian personally and don't know how the plugins work, but if anyone sees this and wants to do a plugin for it, please reach out or open a Github issue and we'll support whatever API changes are needed (if any) to D2's plugin system to make integration easy. (created a Github Discussion to track. https://github.com/terrastruct/d2/discussions/138 https://github.com/terrastruct/d2/discussions/138 . if there's enough demand, we'll do it ourselves)
bitcurious 4y ago
Is the name inspired by d3.js?
- alixanderwang 4y ago
  no
  - Tenal 4y ago
jmyeet 4y ago
This looks really interesting. It made me curious about something. As an example your tree is laid or left-to-right. The GraphViz one in comparison is top down. It’s not clear to me as a user if that’s forced or a choice (in either case). Are you able to clarify that sort of difference by either making the diagrams similarly laid out or adding a note if something isn’t possible?
- alixanderwang 4y ago
  Good observation! Those are just defaults, we don't change the defaults for the comparisons to match (e.g. font sizes, shape dimensions, etc).
ritzaco 4y ago
wow thank you for the transparency - you literally answered all the questions up front that I care about re open core, for profit, killswitches, etc etc that usually take hours to find from a marketing landing page.
billconan 4y ago
I really want to learn the algorithms behind automatic layout engines. Are there any books on this topic?
- alixanderwang 4y ago
  not that i know of, but there's a lot of research papers, under the topic of "graph visualization". we read a ton of these and combined and modified ideas for TALA. The folks at ELK are doing fantastic work in public as an academic team out of Germany. https://www.eclipse.org/elk/documentation/algorithmdevelopers.html https://www.eclipse.org/elk/documentation/algorithmdeveloper... https://www.rtsys.informatik.uni-kiel.de/en/team/soren-domros https://www.rtsys.informatik.uni-kiel.de/en/team/soren-domro... Graphviz's papers are also must-reads (for hierarchical layouts): https://graphviz.org/theory/ https://graphviz.org/theory/ In general they are surprisingly readable even without an academic background. Sometimes they go into proofs of correctness like method X really results in minimal edge crossings, and those are gnarly math, but skippable. The bad part is that a lot of these research papers have code that's just broken or wrong, or is NP-hard.
  - ejuliobatista2 4y ago
    There is this open book https://cs.brown.edu/people/rtamassi/gdhandbook/ https://cs.brown.edu/people/rtamassi/gdhandbook/ We've drawn some inspiration from there and followed to some references too
- escot 4y ago
  dagrejs has a section on "Recommended Reading" which lists which algorithms from which papers they implement. https://github.com/dagrejs/dagre/wiki#recommended-reading https://github.com/dagrejs/dagre/wiki#recommended-reading
trey-jones 4y ago
I'm excited to see this. It's the first repository I've starred in a long time. I'm asked for these types of diagrams from time to time and I've bounced between many different tools, none of which I've found to be satisfactory. I will try this next. Thank you for your work on it.
laomai 4y ago
Looks awesome! Very easy to read! Can't wait for an Obsidian plugin! :D
linuxdude314 4y ago
Do you maintain a dagr fork internally? The project looks to be deprecated: https://github.com/dagrejs/dagre https://github.com/dagrejs/dagre
- alixanderwang 4y ago
  we maintain a wrapper around dagre, which adds functionality like supporting container to container connection. dagre is unmaintained, but it's also "complete", in that it implements the theoretical papers it set out to.
  - tunesmith 4y ago
    I'm wondering why you don't use elkjs? I recently moved my own side project from dagre to elkjs for exactly this reason.
    - alixanderwang 4y ago
      elkjs is an alternative! https://github.com/terrastruct/d2#plugins https://github.com/terrastruct/d2#plugins. we haven't gotten the defaults to look as good though
      - tunesmith 4y ago
        ah! I understand, I've spent a little too much time playing with way too many combinations of elk's configs.
        alixanderwang 4y ago
        if you'd like to help, we're trying to figure some of these configs here in case something stands out to you! https://github.com/terrastruct/d2/pull/95 https://github.com/terrastruct/d2/pull/95
- deleted 4y ago
  [deleted]
Abishek_Muthian 4y ago
Congratulations on the launch! The comparison website is neat. I don't like to draw the architectural diagrams which graphical tools for my Open-source projects, I'll try D2 for my next project.
t0k0l0sh 4y ago
Can you include https://diagrams.mingrammer.com/ https://diagrams.mingrammer.com/ in the comparisons? :-D

deleted 4y ago

[deleted]

wg0 4y ago

Beautiful! And then in golang. Thank you for open sourcing the core part.

mooneater 4y ago

20+ year fan of graphviz here, love seeing what you are doing with D2! Any examples of larger diagrams (1000+s of nodes) and how does that perform in layout?

alixanderwang 4y ago
thank you! There isn't right now, we'd need to add a simpler layout algorithm for those, e.g. force-directed, radial, etc. That use case is not something D2 is trying to tackle though, we're just focused on software architecture diagrams right now, and I've yet to see one span that many nodes
- nonrandomstring 4y ago
  Picking a set of domain specific targets and honing a tool to service that seems a good way to start building - rather than a general solver. I also love graphviz but had a project to create an app for legible quality management, and it was a headache to shoehorn some concepts into "nodes in an edge connected graph" way of thinking (ended up programatically scripting LaTeX). With so many visualisation concepts there's room for niche approaches (same in plotting; Matlab/Octave vs Gnuplot vs Pyplot vs R and so on)
  - alixanderwang 4y ago
    agreed! that reminds me, there's a company that specializes in this niche of visualizing large data: https://www.graphistry.com/ https://www.graphistry.com/ (haven't used it myself, looks well-made).
    - lmeyerov 4y ago
      100% agreement, spot on observations! Indeed, we end up referring folks to cool diagramming tools all the time at Graphistry as d2/mermaid/diagrams.net are well-optimized for quick & beautiful diagram presentation tasks on manageably sized & fairly static datasets. Using Graphistry to do a quick markdown of how a 4 node cluster might work is a bit like driving a tank to pick up some milk :) Teams go to us more for gnarly investigation & splunking tasks that need a visual power tool, like looking at alert logs or big systems, so we optimize for making that scale interactive & easy. The overlap is real however, so a lot of room for teams to learn. Ex: We're the easiest graph tool for jupyter/databricks/streamlit etc teams who use dataframes, and I can imagine those tools learning from us here. In the reverse direction, our work more in terms of quickly configurable global data<>viz data bindings ("using the UI or API, bind each event's score to a hot-and-cold coloring palette and use a warning icon on all type=alert events"), but we have a ways to go to support the more manual artisinal effects of diagramming tools like Figma, where each element might have a super fancy & unique border style.
- deathanatos 4y ago
  IDK if it's quite the use case you're looking for, but at my company, we turn our CI YAML for our monorepo into Dot (Graphviz), and graph it, to visualize the dependencies between steps in our CI, & to highlight the critical path. It's an ~100 node DAG. Graphviz struggles a bit with it, particularly with edge layout. (Though really, the layout I want for it is a Gnatt chart style layout.)
riedel 4y ago
I am always quite astonished how bad the default layouters for graphs perform. When I was still doing compiler optimization in the beginning of the 2000s we did not struggle with quite big graphs thanks to cool graph visualizers such as vcg [1] . Two weeks ago I was tempted to try it again after nearly 20 years because I was frustrated even visualising a relatively small graph in python (cytoscape seemed to be the only working software in the end but it was quite a pain to get it to just render what I wanted) [1] https://www.rw.cdl.uni-saarland.de/people/sander/private/html/gsvcg1.html https://www.rw.cdl.uni-saarland.de/people/sander/private/htm...
- ygra 4y ago
  Laying out larger graphs is tricky, often because the size simply stands in the way of generating anything that's useful to the viewer. To add to that, most layout algorithms prioritize optimizing certain criteria, while other parts of the visualization emerge from that; and if the human viewer chooses a layout algorithm because of one of the latter properties, they are often surprised that the result doesn't look like they envisioned (because, hey, the algorithm optimized something entirely different). We see this disconnect fairly often in our own customer support, but I haven't really found a good way of putting an explanation in writing. Then there's generally the problem of larger graphs, which tend to devolve into a tangled mess and hairballs, simply because they often tend to be well-connected. If there's no way of pruning them beforehand, or perhaps grouping, aggregating or clustering (in a way that makes sense to the viewer, not necessarily only structurally), then it can be hard to get good result.

JonChesterfield 4y ago

Is the language described anywhere? Couldn't find an example other than the one on text-to-diagram and can't find anything beyond that. edit: guessing URL found https://d2lang.com/tour/intro/ https://d2lang.com/tour/intro/ which covers it, perhaps worth linking to there from the GitHub page edit2: ok it is linked, just above the section I was looking at: Reference A more formal reference specification is in the works.

localhost 4y ago

This is a great idea; my go-to approach is the "I'll use PowerPoint to do that and take a screenshot" mindset since I don't have a need for publication quality output. I really like the comparison site that you have created - it's great for learning these languages. I would love to see a plugin for Obsidian for the times where I need to add a custom drawing to my notes. It looks like there already are plugins for PlantUML[1] and graphviz[2] today. [1] https://github.com/QAMichaelPeng/obsidian-graphviz https://github.com/QAMichaelPeng/obsidian-graphviz [2] https://github.com/joethei/obsidian-plantuml https://github.com/joethei/obsidian-plantuml

alixanderwang 4y ago
thanks! i don't use Obsidian personally and don't know how the plugins work, but if anyone sees this and wants to do a plugin for it, please reach out or open a Github issue and we'll support whatever API changes are needed (if any) to D2's plugin system to make integration easy. (created a Github Discussion to track. https://github.com/terrastruct/d2/discussions/138 https://github.com/terrastruct/d2/discussions/138 . if there's enough demand, we'll do it ourselves)

bitcurious 4y ago

Is the name inspired by d3.js?

alixanderwang 4y ago
no
- Tenal 4y ago

jmyeet 4y ago

This looks really interesting. It made me curious about something. As an example your tree is laid or left-to-right. The GraphViz one in comparison is top down. It’s not clear to me as a user if that’s forced or a choice (in either case). Are you able to clarify that sort of difference by either making the diagrams similarly laid out or adding a note if something isn’t possible?

alixanderwang 4y ago
Good observation! Those are just defaults, we don't change the defaults for the comparisons to match (e.g. font sizes, shape dimensions, etc).

ritzaco 4y ago

wow thank you for the transparency - you literally answered all the questions up front that I care about re open core, for profit, killswitches, etc etc that usually take hours to find from a marketing landing page.

billconan 4y ago

I really want to learn the algorithms behind automatic layout engines. Are there any books on this topic?

alixanderwang 4y ago
not that i know of, but there's a lot of research papers, under the topic of "graph visualization". we read a ton of these and combined and modified ideas for TALA. The folks at ELK are doing fantastic work in public as an academic team out of Germany. https://www.eclipse.org/elk/documentation/algorithmdevelopers.html https://www.eclipse.org/elk/documentation/algorithmdeveloper... https://www.rtsys.informatik.uni-kiel.de/en/team/soren-domros https://www.rtsys.informatik.uni-kiel.de/en/team/soren-domro... Graphviz's papers are also must-reads (for hierarchical layouts): https://graphviz.org/theory/ https://graphviz.org/theory/ In general they are surprisingly readable even without an academic background. Sometimes they go into proofs of correctness like method X really results in minimal edge crossings, and those are gnarly math, but skippable. The bad part is that a lot of these research papers have code that's just broken or wrong, or is NP-hard.
- ejuliobatista2 4y ago
  There is this open book https://cs.brown.edu/people/rtamassi/gdhandbook/ https://cs.brown.edu/people/rtamassi/gdhandbook/ We've drawn some inspiration from there and followed to some references too
escot 4y ago
dagrejs has a section on "Recommended Reading" which lists which algorithms from which papers they implement. https://github.com/dagrejs/dagre/wiki#recommended-reading https://github.com/dagrejs/dagre/wiki#recommended-reading

trey-jones 4y ago

I'm excited to see this. It's the first repository I've starred in a long time. I'm asked for these types of diagrams from time to time and I've bounced between many different tools, none of which I've found to be satisfactory. I will try this next. Thank you for your work on it.

laomai 4y ago

Looks awesome! Very easy to read! Can't wait for an Obsidian plugin! :D

linuxdude314 4y ago

Do you maintain a dagr fork internally? The project looks to be deprecated: https://github.com/dagrejs/dagre https://github.com/dagrejs/dagre

alixanderwang 4y ago
we maintain a wrapper around dagre, which adds functionality like supporting container to container connection. dagre is unmaintained, but it's also "complete", in that it implements the theoretical papers it set out to.
- tunesmith 4y ago
  I'm wondering why you don't use elkjs? I recently moved my own side project from dagre to elkjs for exactly this reason.
  - alixanderwang 4y ago
    elkjs is an alternative! https://github.com/terrastruct/d2#plugins https://github.com/terrastruct/d2#plugins. we haven't gotten the defaults to look as good though
    - tunesmith 4y ago
      ah! I understand, I've spent a little too much time playing with way too many combinations of elk's configs.
      - alixanderwang 4y ago
        if you'd like to help, we're trying to figure some of these configs here in case something stands out to you! https://github.com/terrastruct/d2/pull/95 https://github.com/terrastruct/d2/pull/95
deleted 4y ago
[deleted]

Abishek_Muthian 4y ago

Congratulations on the launch! The comparison website is neat. I don't like to draw the architectural diagrams which graphical tools for my Open-source projects, I'll try D2 for my next project.

t0k0l0sh 4y ago

Can you include https://diagrams.mingrammer.com/ https://diagrams.mingrammer.com/ in the comparisons? :-D