10 ms·

Devs need system design tools, not diagramming tools

215 points by argoeris 2y ago

deleted 2y ago
[deleted]
gtirloni 2y ago
I read the whole article and it has great points. Almost nodded the whole time. However, I couldn't identify whether an alternative exists today.
- anotherhue 2y ago
  Maybe checkout https://c4model.com/ https://c4model.com/
- imglorp 2y ago
  It sounded like it was working up to a C4 pitch but never got there. It bears a look exactly for this post's title. https://c4model.com https://c4model.com
- Veuxdo 2y ago
  Almost all of his complaints stem from using generic drag-and-drop diagramming tools. A modern take on system diagramming, like Ilograph[0], solves most of these issues (IMBO). [0] https://www.ilograph.com https://www.ilograph.com
- qznc 2y ago
  I believe the answer is to have a single model of everything. You can build the model from whatever: Draw it with a GUI, query your server infrastructure, analyze binaries, collect it from databases, parse it from documentation, process your issues, etc. The important points is to merge all the information into a single model. You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions. Does it exist today? Many tools and processes have been developed to achieve it. For example, "Model-based systems engineering" is a very formal approach. I have yet to see it realized in practice though. I'm pretty sure it isn't a single tool or process though. By definition it integrates with nearly everything and that means a lot of customization. You won't get it get it off the shelf.
  - Veuxdo 2y ago
    > You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions. It sounds like you're describing: https://www.ilograph.com/blog/posts/concrete-diagramming-models/ https://www.ilograph.com/blog/posts/concrete-diagramming-mod...
    - qznc 2y ago
      It looks quite limited in what kind of diagrams it produces. What about a class diagram, a state machine, timing behavior, etc?
  - AnimalMuppet 2y ago
    The problem with "a single model of everything" is that it is an inferior tool. It's a drawing or diagramming tool, but the stand-alone diagramming tools are better. It's a text composition tool, but word processors are better. It's a code writing tool, but IDEs are better. In every single thing it tries to do, a specialized tool is better. So this single tool needs to be close enough to the best in every category in order to not be a boat anchor holding you back. So far, nothing has come close.
    - qznc 2y ago
      Yes, that is good point. It might be reason why it fails in practice. Whatever view one produces from a single model, it will look crappy and cheap compared to the Powerpoint slide of someone else. It might be more truthful and more up to date, but it isn't as persuasive. Persuasion is what a presentation is ultimately about: It should influence the behavior of the audience.
    - TeMPOraL 2y ago
      Nothing stops us from using specialized tools in this case. They just need to not work on "single source of truth" of their respected domain, but on a projection of a single artifact into relevant dimension. Devil's in the details, of course, but at least with IDEs I can tell with certainty, based on my experience, that sticking to editing directly the single source of truth plaintext codebase is wasting much more time and cognitive effort than IDEs are saving us.
- argoeris 2y ago
  There are quite a few tools cropping up trying to solve this problem. Multiplayer.app is one example - they use OTel to gather distributed traces from your system and ensure you automatically get notified when there's drift.
dennisy 2y ago
Great post but lacking an answer. I guess we are yet to find one!
- sam1r 2y ago
  Agreed! If every acknowledgement of the(or any imo) issue at hand, was accompanied with a minimally viable solution (even if it doesn't work) -- as the defacto standard for providing critical feedback.. As long as some effort is put in towards a countering possible solution in response-- I truly feel that we would be closer to an end solution. I guess should this "problem" truly exist.
- Joel_Mckay 2y ago
  I think the folks working on constraint programming with MiniZinc and Phoenix/LiveView have achieved a fairly practical solution. https://github.com/bokner/solverview https://github.com/bokner/solverview https://github.com/bokner/solverl https://github.com/bokner/solverl https://www.minizinc.org/ https://www.minizinc.org/ YMMV, as I've yet to personally try a toy project with this approach. =3
- ChoHag 2y ago
  [dead]
- argoeris 2y ago
  Check out https://www.multiplayer.app/ https://www.multiplayer.app/ The author is one of the co-founders, but I understand not wanting to just promote his tool and just have a conversation about the problem.
RcouF1uZ4gsC 2y ago
>Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams, One issue is that diagrams in general are pretty universal. As long as your tool can makes shapes and connect them, you can use it for any kind of architecture. For example it will work flow charts for 50 year old Fortran to UML from the 90s to microservices diagrams from now and I bet to whatever is common 50 years from now.
- sam1r 2y ago
  >> >Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams, "Outdated", imo, is subjective to the content and is bias'd by the time since creation. As long as wireframed thought is the bare skeleton, the diagrams are merely a read-only user interface to communicate to a particular end-user audience.
- argoeris 2y ago
  I don't think the issue is with diagrams per se, but with how we create them. They are super helpful in conveying meaning but why do we need to create and update them manually?
jawns 2y ago
This reminds me of the following: * Devs: We need a better language than Javascript as the lingua franca of the web! * Users: Sure, use whatever language you want -- just make sure it compiles to Javascript, which is already the lingua franca of the web. Keep in mind that the consumers of technical diagrams are often non-technical folks. And they don't care about how they get their diagrams. They just want to be able to understand, at a high level, what's going on in the black box. You can either convince every single one of them that devs need to focus on better system design tools ... or you can continue to give them the diagrams they want, just using a smarter process to generate them. Or you can treat them as entirely separate problems, because fundamentally system design tools are building tools, and system diagrams are communication tools. In most cases you can improve them independently.
- BadHumans 2y ago
  > * Users: Sure, use whatever language you want -- just make sure it compiles to Javascript, which is already the lingua franca of the web. I don't think users are saying this and I don't think they care as long as it works.
  - Jtsummers 2y ago
    Exactly. Users don't care about the underlying tech as long as it works and works on the platforms they're using and with other systems they're using. That's it. When users start caring about the underlying tech, you need to actively dissuade them unless there is a sound technical argument. Otherwise you end up with people insisting on using DOS for enterprise applications in 2024 because that's what they've used since 1984 (wish I hadn't experienced this).
    - mschuster91 2y ago
      > Otherwise you end up with people insisting on using DOS for enterprise applications in 2024 because that's what they've used since 1984 (wish I hadn't experienced this). DOS as a tech stack is ludicrous (unless you're working in embedded/industrial scenarios, but hey, at least you can command a significant paycheck in exchange for that level of horrors). A terminal user interface however? Absolutely not, particularly if your userbase often has literally decades worth of muscle memory. Banks, governments or travel agencies (see an example for Amadeus here [1]) live and die with the "legacy" TUI. [1] https://servicehub.amadeus.com/c/portal/view-solution/832976/how-to-display-a-list-of-amadeus-interface-records-a.i.r.s- https://servicehub.amadeus.com/c/portal/view-solution/832976...
canterburry 2y ago
We have the tools and ways to express architecture. The problem is no one is bothering to actually learn it and do it well. Everyone seems to look for answers involving not having to spend any time learning anything new. UML, BPMN, 4+1 view, it can do it all. Just learn it and make sure everyone else in your team does too.
- Supermancho 2y ago
  Product people will not. So we have the inglorious situation of 2 sets of diagrams and notes on the translation between them. If the problem was easy to solve, it wouldn't be something you can express in a sentence, while simultaneously asserting how easy it is. Some people do not adequately constrain the problem while others overly constrain it, conceptually.
  - canterburry 2y ago
    BPMN was created for product people. UML use case diagrams were created for product people. I don't see the argument why product people don't need to learn this. They work in a software organization, they build software, they should be expected to know the diagrams pertinent to them.
    - malfist 2y ago
      Use Case Diagrams are only one small part of UML, usually the part people learn, but it's way bigger than that.
      - steve1977 2y ago
        And the infamous stick figure diagrams are even only a small part of Use Case analysis and design themselves within UML.
    - pessimizer 2y ago
      > I don't see the argument why product people don't need to learn this. They need to, but in my experience, they simply don't want to. They want to get the credit for the product, but they want to wave their arm and just have its features appear without actually specifying them. I understand because it is hard, exacting work. But as it is specificity just gets pushed from product people down to the lowest-level person responsible for implementation.
SrslyJosh 2y ago
This would be more interesting if there was actually a proposed solution.
- argoeris 2y ago
  Check out https://www.multiplayer.app/ https://www.multiplayer.app/ That's the project the author is working on.
qwerty456127 2y ago
Also GUI design tools like the WinForms designer but for more modern toolkits please. VB/Delphi/WinForms were such a joy to design and XAMl (let alone web frontent) is such a pain to write.
ranger_danger 2y ago
I feel like https://plantuml.com/ https://plantuml.com/ gets close to what I want by being able to make diagrams with code, but for system design what I really want is to be able to have diagrams generated directly from the code itself, maybe with some extra comments/annotations that help it along. Does anything like that exist already?
- nomoreipg 2y ago
  Commented this above, but I actually built a site to auto generate up-to-date interactive visual diagrams for codebases. It's pretty plug&play, using static analysis & LLMs we generate documentation + an interactive system diagram you can recursively explore. https://vision.useadrenaline.com https://vision.useadrenaline.com All the auto-generated diagrams are exportable as mermaid!
  - ranger_danger 2y ago
    proprietary? no thanks
Veuxdo 2y ago
You need to use diagramming tools that are meant for complex systems. Almost all of the complaints in this article are caused be using generic drag-and-drop diagramming tools.
- hemloc_io 2y ago
  What are those tools? Every big tech job I've had just uses the more generic ones like draw.io or maybe an internal only version.
  - ABoredBirb 2y ago
    C4 diagrams using Mermaid markdown. Easy to reason about, easily version-controlled.
  - argoeris 2y ago
    My vote is for https://www.multiplayer.app/ https://www.multiplayer.app/
agumonkey 2y ago
I wonder if modeling/system design people didn't have new insights since the last 20 years (when Model Driven * started to sink). Languages changed a bit, more types, more concurrency, more functional idioms. Yet I feel we're not going anywhere.
bunsenhoneydew 2y ago
As others have said the C4 model is a great way to address a number of these issues. I can’t find the right video at the moment but Simon Brown (creator of C4) gives a great talk about creating his DSL, Structurizr, for C4, which he developed during COVID lockdown (if memory serves). There are many videos on YouTube of Simon talking about “C4 Models as Code” so I’m sure any one of those will suffice. The focus is on creating the model of your system architecture, from which the diagrams you extract are a specific projection of that model. Rather than a diagram be the main artifact. It’s a simple but very powerful concept that I’m always surprised isn’t more widely used. Structurizr models can also be exported to display as ilograph diagrams, mermaid diagrams and more. Also very much worth a mention is icepanel, a lovely tool for architectural model that implements the C4 model heavily. I saw Simon talk at a conference in Sydney about 10-15 years ago and heard about C4 for the first time in that talk, it’s been one of the most influential talks I’ve been to in my career as it made a lot of fuzzy things in my head all start to come together in a way that just made sense. https://c4model.com/ https://c4model.com/ https://structurizr.com/ https://structurizr.com/ https://www.ilograph.com/ https://www.ilograph.com/ https://mermaid.js.org/ https://mermaid.js.org/ https://icepanel.io/ https://icepanel.io/
- simon_brown 2y ago
  Thank you! :-)
- nomoreipg 2y ago
  I actually built a site to auto generate up-to-date interactive visual diagrams for codebases. It's pretty plug&play, using static analysis & LLMs we generate documentation + an interactive system diagram you can recursively explore. https://vision.useadrenaline.com https://vision.useadrenaline.com All the auto-generated diagrams are exportable as mermaid
  - _caw 2y ago
    I really wanted to check this out (using redis as an example), but it appears I need an account. Is there no public demo?
bitwize 2y ago
In 1971, M. Bryce and Associates marketed PRIDE, the first commercial software methodology for general business use. It was originally an entirely manual process using paper forms, but it was already far more comprehensive than anything called a "methodology" we use today. For example, in PRIDE, every artifact of the systems design and implementation process -- each high-level requirement, each software module, each database table and column, and more -- is assigned a tracking number and extensively documented as to its purpose in a unified knowledge repository. This was decades before Git or JIRA, and at first it was all done by hand, but not for long. In the 80s, they marketed PRIDE/ASDM, which combines PRIDE with Automated Systems Design Methodology, a suite of system design tools written in COBOL for mainframes. Far from being mere diagramming tools, they assisted in all aspects of an information systems design from initial requirements down through coding and database management. A key component of ASDM was the Information Resource Manager (IRM), a searchable central database of all of the information artifacts described above along with their documentation. Another component was Automated Instructional Materials (AIM), the online documentation facility, which not only provided instructions on how to use the system, it also provided step-by-step directions (called "playscripts" in PRIDE-speak) for each member of the development team to follow, in order to see a business system through from abstract design down to software implementation and deployment. Provided the directions were followed, it was next to impossible to screw up a system or subsystem implemented through PRIDE/ASDM. This level of comprehensiveness and clarity is the gold standard for business IS development. And it seems to be nearly lost to time.
- theamk 2y ago
  "every artifact of the systems design and implementation process [..] is assigned a tracking number and extensively documented", "it also provided step-by-step directions [...] for each member of the development team to follow" Maybe this made sense back in times of COBOL, but with the modern high-level languages this sounds like bureaucratic hell, the worst kind of micromanagement.
  - bitwize 2y ago
    Well, what's happened in the years since is that programmers have been in the driver's seat when it comes to IS, and that's resulted in a lot of sloppy, disorganized thinking in the field. Programmers in general fancy themselves as artists and resist organization and discipline -- as true today as it was in the COBOL days -- so it's no surprise that a method based on improved process control seems like micromanagement. But that level of discipline is needed in order to do the work right. Ultimately it should come down to self-discipline. Unfortunately, IS has morphed into a programmer-centric field, which leads us to Agile, a family of methodologies by programmers for programmers. Agile mythology has it exactly backward: Agile is not the thing that saved us from waterfall. Agile was the "traditional" software development process PRIDE was created as a response to: start programming right away, deploy rough versions to production, and keep iterating with code changes until you have something that kind of, sort of works. Compared to this, PRIDE delivers improved productivity and quality, reduced costs, and reduced administrative overhead. I've never been on an on-time, under-budget Agile project, and one of my former employers basically aborted their entire Agile transformation. Once the industry has shaken off its Agile wine, especially as profitability becomes more important in the post-ZIRP economy, software development will look more like PRIDE than like anything we see today.
danjl 2y ago
Google Slides and Drawings (under + > More...) are great tools. They support the diagramming objects, connectors, shared collaboration, and integration with the other Google Doc tools. They also have great history and diff tools as well as team comments, with actions.
- argoeris 2y ago
  .. but they are manual. It's great to have history, diff, comments, etc. But why do I have to spend time manually creating a diagram and updating it every time I add a new dependency when it can be automatically done for me?
digger495 2y ago
This is a useless and insipid editorial. I don't want the problem redeclared. We know this. We want solutions.
- argoeris 2y ago
  Fair point. I'm sure we all agree on why using general-purpose diagramming tools is too much manual work for system design. This is a possible solution: https://www.multiplayer.app/ https://www.multiplayer.app/ It allows dynamic diagrams and keeps them automatically updated leveraging OTel.
throwaway290232 2y ago
[dead]
yuliyp 2y ago
It's not so hard to do static and tracing-based analysis to capture all the calls that various systems make between themselves. Similarly, it's not so hard to graph all of the metrics of a system. That's not really all that useful. Diagrams, like other forms of documentation, are a format for communicating something to the reader. It means that they should spend more space on the important flows rather than the exceptional ones. They'll explain the meaningful parts of a sequence rather than just giving a series of function calls. The various tools we have for doing this (boxes showing what systems talk to what, sequence diagrams, database schema diagrams) provide a rich enough language for that communication. Death star diagrams are bad because they spend a bunch of time and effort to convey one piece of information: "there are a lot of systems here" rather than anything actually useful for someone attempting to navigate a specific part of the system.
steve1977 2y ago
I think at least part of the problem is that todays "agile" methodologies lure people into almost completely neglecting analysis and design, as if these steps are forbidden in agile. So for many dev teams I have seen, agile basically means "we have no plan".
- spyspy 2y ago
  I’d characterize it as the degradation of the product manager role. Too many project managers get promoted to product but don’t really know what that job entails so they just project manage harder. And to a project manager “I’m gonna take a week and think about it” is not an acceptable answer.
- argoeris 2y ago
  I agree that many teams, in an effort to move away from waterfall development and Big Design Up Front, have gone the opposite way and completely skip system design. Which is a mistake, because you need some upfront design. As Dave Thomas said: “big upfront design is dumb. No upfront design is dumber”.
webprofusion 2y ago
If we had a complete and evolving set of requirements for any reasonably complex system, we wouldn't really read them once they no longer match the system we have delivered. Systems and requirements go hand in hand but the reality is requirements are eventually codified, and after a point only the system matters because the system is the reality and the requirements were just the formal inspiration. I would love to see automated requirement analysis and verification from code.
_xiaz 2y ago
No, I need a diagramming tool. Excalidraw lives on
- argoeris 2y ago
  Excalidraw is great for brainstorming and sketching. But I don't exclude pairing it with a tool that also automatically shows me all the metadata of system components, automatically detects architecture drift, etc.
tomjohnson3 2y ago
Thank you everyone for reading my article! I’m the author, Thomas Johnson. This article stems from my frustration with the typical approach of asking, “What diagramming tool should we use?” instead of addressing the root problem: the need for up-to-date, easily accessible system architecture information. That’s why I co-founded Multiplayer. We focus on automating the creation and maintenance of system architecture diagrams and creating a single source of truth for system information. This includes your individual components, APIs, dependencies, and repositories. We’re language and environment agnostic and you can start with a napkin sketch or a photo of your whiteboard. And this is just the start, we have many plans for how to evolve system design tooling including supporting popular integrations and models like C4. It’s early days for us. I’d love to hear what you think: https://www.multiplayer.app/ https://www.multiplayer.app/ (It’s free!)
kmerroll 2y ago
To take this a different way, Devs need AI-driven system design tools, not diagramming tools. The work to automate the process to define, code, deploy, and document tech/app solutions using GenAI is arguably well underway and the days of humans producing layers of boxes and arrows documentation is thankfully numbered. More directly, the need for the special snowflakes and the ensuing complex system design processes will be reduced as we drive towards more automation N(A) frameworks.
airbreather 2y ago
There is a potentially interesting standard many may not know - IEC 61499 - Standard for Distributed Automation, first ed 2005!!! and related to PLC type controllers. It was meant to be the next step forward from IEC 61131 that described the five PLC control languages, but far ahead of it's time. 61499 introduced the concept of an executable specification, and went on develop tie ins with XML and OPC. The executable specification is a particularly interesting idea, effectively emerging as no code these days But two decades on is still only starting to get traction, seemingly because it was so far ahead of it's time, and the target audience was potentially mostly only just branching out into software because they were now programming ladder logic, instead of designing panel boards with hundred of relays and timers. But, it is worth a look for some ideas in the space, particularly the executable specification concept (you know this can only lead to a DSL of some kind), but also including their diagramming that looks a lot like UML but maybe works better, along with 4Diac software package, consideration of how some of the more esoteric aspects of OPC fit into all this, and more slightly obliquely, TLA+.

215 points by argoeris 2y ago

deleted 2y ago

[deleted]

I read the whole article and it has great points. Almost nodded the whole time. However, I couldn't identify whether an alternative exists today.

anotherhue 2y ago
Maybe checkout https://c4model.com/ https://c4model.com/
imglorp 2y ago
It sounded like it was working up to a C4 pitch but never got there. It bears a look exactly for this post's title. https://c4model.com https://c4model.com
Veuxdo 2y ago
Almost all of his complaints stem from using generic drag-and-drop diagramming tools. A modern take on system diagramming, like Ilograph[0], solves most of these issues (IMBO). [0] https://www.ilograph.com https://www.ilograph.com
qznc 2y ago
I believe the answer is to have a single model of everything. You can build the model from whatever: Draw it with a GUI, query your server infrastructure, analyze binaries, collect it from databases, parse it from documentation, process your issues, etc. The important points is to merge all the information into a single model. You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions. Does it exist today? Many tools and processes have been developed to achieve it. For example, "Model-based systems engineering" is a very formal approach. I have yet to see it realized in practice though. I'm pretty sure it isn't a single tool or process though. By definition it integrates with nearly everything and that means a lot of customization. You won't get it get it off the shelf.
- Veuxdo 2y ago
  > You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions. It sounds like you're describing: https://www.ilograph.com/blog/posts/concrete-diagramming-models/ https://www.ilograph.com/blog/posts/concrete-diagramming-mod...
  - qznc 2y ago
    It looks quite limited in what kind of diagrams it produces. What about a class diagram, a state machine, timing behavior, etc?
- AnimalMuppet 2y ago
  The problem with "a single model of everything" is that it is an inferior tool. It's a drawing or diagramming tool, but the stand-alone diagramming tools are better. It's a text composition tool, but word processors are better. It's a code writing tool, but IDEs are better. In every single thing it tries to do, a specialized tool is better. So this single tool needs to be close enough to the best in every category in order to not be a boat anchor holding you back. So far, nothing has come close.
  - qznc 2y ago
    Yes, that is good point. It might be reason why it fails in practice. Whatever view one produces from a single model, it will look crappy and cheap compared to the Powerpoint slide of someone else. It might be more truthful and more up to date, but it isn't as persuasive. Persuasion is what a presentation is ultimately about: It should influence the behavior of the audience.
  - TeMPOraL 2y ago
    Nothing stops us from using specialized tools in this case. They just need to not work on "single source of truth" of their respected domain, but on a projection of a single artifact into relevant dimension. Devil's in the details, of course, but at least with IDEs I can tell with certainty, based on my experience, that sticking to editing directly the single source of truth plaintext codebase is wasting much more time and cognitive effort than IDEs are saving us.
argoeris 2y ago
There are quite a few tools cropping up trying to solve this problem. Multiplayer.app is one example - they use OTel to gather distributed traces from your system and ensure you automatically get notified when there's drift.

dennisy 2y ago

Great post but lacking an answer. I guess we are yet to find one!

sam1r 2y ago
Agreed! If every acknowledgement of the(or any imo) issue at hand, was accompanied with a minimally viable solution (even if it doesn't work) -- as the defacto standard for providing critical feedback.. As long as some effort is put in towards a countering possible solution in response-- I truly feel that we would be closer to an end solution. I guess should this "problem" truly exist.
Joel_Mckay 2y ago
I think the folks working on constraint programming with MiniZinc and Phoenix/LiveView have achieved a fairly practical solution. https://github.com/bokner/solverview https://github.com/bokner/solverview https://github.com/bokner/solverl https://github.com/bokner/solverl https://www.minizinc.org/ https://www.minizinc.org/ YMMV, as I've yet to personally try a toy project with this approach. =3
ChoHag 2y ago
[dead]
argoeris 2y ago
Check out https://www.multiplayer.app/ https://www.multiplayer.app/ The author is one of the co-founders, but I understand not wanting to just promote his tool and just have a conversation about the problem.

RcouF1uZ4gsC 2y ago

>Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams, One issue is that diagrams in general are pretty universal. As long as your tool can makes shapes and connect them, you can use it for any kind of architecture. For example it will work flow charts for 50 year old Fortran to UML from the 90s to microservices diagrams from now and I bet to whatever is common 50 years from now.

sam1r 2y ago
>> >Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams, "Outdated", imo, is subjective to the content and is bias'd by the time since creation. As long as wireframed thought is the bare skeleton, the diagrams are merely a read-only user interface to communicate to a particular end-user audience.
argoeris 2y ago
I don't think the issue is with diagrams per se, but with how we create them. They are super helpful in conveying meaning but why do we need to create and update them manually?

jawns 2y ago

This reminds me of the following: * Devs: We need a better language than Javascript as the lingua franca of the web! * Users: Sure, use whatever language you want -- just make sure it compiles to Javascript, which is already the lingua franca of the web. Keep in mind that the consumers of technical diagrams are often non-technical folks. And they don't care about how they get their diagrams. They just want to be able to understand, at a high level, what's going on in the black box. You can either convince every single one of them that devs need to focus on better system design tools ... or you can continue to give them the diagrams they want, just using a smarter process to generate them. Or you can treat them as entirely separate problems, because fundamentally system design tools are building tools, and system diagrams are communication tools. In most cases you can improve them independently.

BadHumans 2y ago
> * Users: Sure, use whatever language you want -- just make sure it compiles to Javascript, which is already the lingua franca of the web. I don't think users are saying this and I don't think they care as long as it works.
- Jtsummers 2y ago
  Exactly. Users don't care about the underlying tech as long as it works and works on the platforms they're using and with other systems they're using. That's it. When users start caring about the underlying tech, you need to actively dissuade them unless there is a sound technical argument. Otherwise you end up with people insisting on using DOS for enterprise applications in 2024 because that's what they've used since 1984 (wish I hadn't experienced this).
  - mschuster91 2y ago
    > Otherwise you end up with people insisting on using DOS for enterprise applications in 2024 because that's what they've used since 1984 (wish I hadn't experienced this). DOS as a tech stack is ludicrous (unless you're working in embedded/industrial scenarios, but hey, at least you can command a significant paycheck in exchange for that level of horrors). A terminal user interface however? Absolutely not, particularly if your userbase often has literally decades worth of muscle memory. Banks, governments or travel agencies (see an example for Amadeus here [1]) live and die with the "legacy" TUI. [1] https://servicehub.amadeus.com/c/portal/view-solution/832976/how-to-display-a-list-of-amadeus-interface-records-a.i.r.s- https://servicehub.amadeus.com/c/portal/view-solution/832976...

canterburry 2y ago

We have the tools and ways to express architecture. The problem is no one is bothering to actually learn it and do it well. Everyone seems to look for answers involving not having to spend any time learning anything new. UML, BPMN, 4+1 view, it can do it all. Just learn it and make sure everyone else in your team does too.

Supermancho 2y ago
Product people will not. So we have the inglorious situation of 2 sets of diagrams and notes on the translation between them. If the problem was easy to solve, it wouldn't be something you can express in a sentence, while simultaneously asserting how easy it is. Some people do not adequately constrain the problem while others overly constrain it, conceptually.
- canterburry 2y ago
  BPMN was created for product people. UML use case diagrams were created for product people. I don't see the argument why product people don't need to learn this. They work in a software organization, they build software, they should be expected to know the diagrams pertinent to them.
  - malfist 2y ago
    Use Case Diagrams are only one small part of UML, usually the part people learn, but it's way bigger than that.
    - steve1977 2y ago
      And the infamous stick figure diagrams are even only a small part of Use Case analysis and design themselves within UML.
  - pessimizer 2y ago
    > I don't see the argument why product people don't need to learn this. They need to, but in my experience, they simply don't want to. They want to get the credit for the product, but they want to wave their arm and just have its features appear without actually specifying them. I understand because it is hard, exacting work. But as it is specificity just gets pushed from product people down to the lowest-level person responsible for implementation.

SrslyJosh 2y ago

This would be more interesting if there was actually a proposed solution.

argoeris 2y ago
Check out https://www.multiplayer.app/ https://www.multiplayer.app/ That's the project the author is working on.

qwerty456127 2y ago

Also GUI design tools like the WinForms designer but for more modern toolkits please. VB/Delphi/WinForms were such a joy to design and XAMl (let alone web frontent) is such a pain to write.

ranger_danger 2y ago

I feel like https://plantuml.com/ https://plantuml.com/ gets close to what I want by being able to make diagrams with code, but for system design what I really want is to be able to have diagrams generated directly from the code itself, maybe with some extra comments/annotations that help it along. Does anything like that exist already?

nomoreipg 2y ago
Commented this above, but I actually built a site to auto generate up-to-date interactive visual diagrams for codebases. It's pretty plug&play, using static analysis & LLMs we generate documentation + an interactive system diagram you can recursively explore. https://vision.useadrenaline.com https://vision.useadrenaline.com All the auto-generated diagrams are exportable as mermaid!
- ranger_danger 2y ago
  proprietary? no thanks

Veuxdo 2y ago

You need to use diagramming tools that are meant for complex systems. Almost all of the complaints in this article are caused be using generic drag-and-drop diagramming tools.

hemloc_io 2y ago
What are those tools? Every big tech job I've had just uses the more generic ones like draw.io or maybe an internal only version.
- ABoredBirb 2y ago
  C4 diagrams using Mermaid markdown. Easy to reason about, easily version-controlled.
- argoeris 2y ago
  My vote is for https://www.multiplayer.app/ https://www.multiplayer.app/

agumonkey 2y ago

I wonder if modeling/system design people didn't have new insights since the last 20 years (when Model Driven * started to sink). Languages changed a bit, more types, more concurrency, more functional idioms. Yet I feel we're not going anywhere.

bunsenhoneydew 2y ago

As others have said the C4 model is a great way to address a number of these issues. I can’t find the right video at the moment but Simon Brown (creator of C4) gives a great talk about creating his DSL, Structurizr, for C4, which he developed during COVID lockdown (if memory serves). There are many videos on YouTube of Simon talking about “C4 Models as Code” so I’m sure any one of those will suffice. The focus is on creating the model of your system architecture, from which the diagrams you extract are a specific projection of that model. Rather than a diagram be the main artifact. It’s a simple but very powerful concept that I’m always surprised isn’t more widely used. Structurizr models can also be exported to display as ilograph diagrams, mermaid diagrams and more. Also very much worth a mention is icepanel, a lovely tool for architectural model that implements the C4 model heavily. I saw Simon talk at a conference in Sydney about 10-15 years ago and heard about C4 for the first time in that talk, it’s been one of the most influential talks I’ve been to in my career as it made a lot of fuzzy things in my head all start to come together in a way that just made sense. https://c4model.com/ https://c4model.com/ https://structurizr.com/ https://structurizr.com/ https://www.ilograph.com/ https://www.ilograph.com/ https://mermaid.js.org/ https://mermaid.js.org/ https://icepanel.io/ https://icepanel.io/

simon_brown 2y ago
Thank you! :-)
nomoreipg 2y ago
I actually built a site to auto generate up-to-date interactive visual diagrams for codebases. It's pretty plug&play, using static analysis & LLMs we generate documentation + an interactive system diagram you can recursively explore. https://vision.useadrenaline.com https://vision.useadrenaline.com All the auto-generated diagrams are exportable as mermaid
- _caw 2y ago
  I really wanted to check this out (using redis as an example), but it appears I need an account. Is there no public demo?

bitwize 2y ago

In 1971, M. Bryce and Associates marketed PRIDE, the first commercial software methodology for general business use. It was originally an entirely manual process using paper forms, but it was already far more comprehensive than anything called a "methodology" we use today. For example, in PRIDE, every artifact of the systems design and implementation process -- each high-level requirement, each software module, each database table and column, and more -- is assigned a tracking number and extensively documented as to its purpose in a unified knowledge repository. This was decades before Git or JIRA, and at first it was all done by hand, but not for long. In the 80s, they marketed PRIDE/ASDM, which combines PRIDE with Automated Systems Design Methodology, a suite of system design tools written in COBOL for mainframes. Far from being mere diagramming tools, they assisted in all aspects of an information systems design from initial requirements down through coding and database management. A key component of ASDM was the Information Resource Manager (IRM), a searchable central database of all of the information artifacts described above along with their documentation. Another component was Automated Instructional Materials (AIM), the online documentation facility, which not only provided instructions on how to use the system, it also provided step-by-step directions (called "playscripts" in PRIDE-speak) for each member of the development team to follow, in order to see a business system through from abstract design down to software implementation and deployment. Provided the directions were followed, it was next to impossible to screw up a system or subsystem implemented through PRIDE/ASDM. This level of comprehensiveness and clarity is the gold standard for business IS development. And it seems to be nearly lost to time.

theamk 2y ago
"every artifact of the systems design and implementation process [..] is assigned a tracking number and extensively documented", "it also provided step-by-step directions [...] for each member of the development team to follow" Maybe this made sense back in times of COBOL, but with the modern high-level languages this sounds like bureaucratic hell, the worst kind of micromanagement.
- bitwize 2y ago
  Well, what's happened in the years since is that programmers have been in the driver's seat when it comes to IS, and that's resulted in a lot of sloppy, disorganized thinking in the field. Programmers in general fancy themselves as artists and resist organization and discipline -- as true today as it was in the COBOL days -- so it's no surprise that a method based on improved process control seems like micromanagement. But that level of discipline is needed in order to do the work right. Ultimately it should come down to self-discipline. Unfortunately, IS has morphed into a programmer-centric field, which leads us to Agile, a family of methodologies by programmers for programmers. Agile mythology has it exactly backward: Agile is not the thing that saved us from waterfall. Agile was the "traditional" software development process PRIDE was created as a response to: start programming right away, deploy rough versions to production, and keep iterating with code changes until you have something that kind of, sort of works. Compared to this, PRIDE delivers improved productivity and quality, reduced costs, and reduced administrative overhead. I've never been on an on-time, under-budget Agile project, and one of my former employers basically aborted their entire Agile transformation. Once the industry has shaken off its Agile wine, especially as profitability becomes more important in the post-ZIRP economy, software development will look more like PRIDE than like anything we see today.

danjl 2y ago

Google Slides and Drawings (under + > More...) are great tools. They support the diagramming objects, connectors, shared collaboration, and integration with the other Google Doc tools. They also have great history and diff tools as well as team comments, with actions.

argoeris 2y ago
.. but they are manual. It's great to have history, diff, comments, etc. But why do I have to spend time manually creating a diagram and updating it every time I add a new dependency when it can be automatically done for me?

digger495 2y ago

This is a useless and insipid editorial. I don't want the problem redeclared. We know this. We want solutions.

argoeris 2y ago
Fair point. I'm sure we all agree on why using general-purpose diagramming tools is too much manual work for system design. This is a possible solution: https://www.multiplayer.app/ https://www.multiplayer.app/ It allows dynamic diagrams and keeps them automatically updated leveraging OTel.

throwaway290232 2y ago

[dead]

yuliyp 2y ago

It's not so hard to do static and tracing-based analysis to capture all the calls that various systems make between themselves. Similarly, it's not so hard to graph all of the metrics of a system. That's not really all that useful. Diagrams, like other forms of documentation, are a format for communicating something to the reader. It means that they should spend more space on the important flows rather than the exceptional ones. They'll explain the meaningful parts of a sequence rather than just giving a series of function calls. The various tools we have for doing this (boxes showing what systems talk to what, sequence diagrams, database schema diagrams) provide a rich enough language for that communication. Death star diagrams are bad because they spend a bunch of time and effort to convey one piece of information: "there are a lot of systems here" rather than anything actually useful for someone attempting to navigate a specific part of the system.

steve1977 2y ago

I think at least part of the problem is that todays "agile" methodologies lure people into almost completely neglecting analysis and design, as if these steps are forbidden in agile. So for many dev teams I have seen, agile basically means "we have no plan".

spyspy 2y ago
I’d characterize it as the degradation of the product manager role. Too many project managers get promoted to product but don’t really know what that job entails so they just project manage harder. And to a project manager “I’m gonna take a week and think about it” is not an acceptable answer.
argoeris 2y ago
I agree that many teams, in an effort to move away from waterfall development and Big Design Up Front, have gone the opposite way and completely skip system design. Which is a mistake, because you need some upfront design. As Dave Thomas said: “big upfront design is dumb. No upfront design is dumber”.

webprofusion 2y ago

If we had a complete and evolving set of requirements for any reasonably complex system, we wouldn't really read them once they no longer match the system we have delivered. Systems and requirements go hand in hand but the reality is requirements are eventually codified, and after a point only the system matters because the system is the reality and the requirements were just the formal inspiration. I would love to see automated requirement analysis and verification from code.

_xiaz 2y ago

No, I need a diagramming tool. Excalidraw lives on

argoeris 2y ago
Excalidraw is great for brainstorming and sketching. But I don't exclude pairing it with a tool that also automatically shows me all the metadata of system components, automatically detects architecture drift, etc.

tomjohnson3 2y ago

Thank you everyone for reading my article! I’m the author, Thomas Johnson. This article stems from my frustration with the typical approach of asking, “What diagramming tool should we use?” instead of addressing the root problem: the need for up-to-date, easily accessible system architecture information. That’s why I co-founded Multiplayer. We focus on automating the creation and maintenance of system architecture diagrams and creating a single source of truth for system information. This includes your individual components, APIs, dependencies, and repositories. We’re language and environment agnostic and you can start with a napkin sketch or a photo of your whiteboard. And this is just the start, we have many plans for how to evolve system design tooling including supporting popular integrations and models like C4. It’s early days for us. I’d love to hear what you think: https://www.multiplayer.app/ https://www.multiplayer.app/ (It’s free!)

kmerroll 2y ago

To take this a different way, Devs need AI-driven system design tools, not diagramming tools. The work to automate the process to define, code, deploy, and document tech/app solutions using GenAI is arguably well underway and the days of humans producing layers of boxes and arrows documentation is thankfully numbered. More directly, the need for the special snowflakes and the ensuing complex system design processes will be reduced as we drive towards more automation N(A) frameworks.

airbreather 2y ago

There is a potentially interesting standard many may not know - IEC 61499 - Standard for Distributed Automation, first ed 2005!!! and related to PLC type controllers. It was meant to be the next step forward from IEC 61131 that described the five PLC control languages, but far ahead of it's time. 61499 introduced the concept of an executable specification, and went on develop tie ins with XML and OPC. The executable specification is a particularly interesting idea, effectively emerging as no code these days But two decades on is still only starting to get traction, seemingly because it was so far ahead of it's time, and the target audience was potentially mostly only just branching out into software because they were now programming ladder logic, instead of designing panel boards with hundred of relays and timers. But, it is worth a look for some ideas in the space, particularly the executable specification concept (you know this can only lead to a DSL of some kind), but also including their diagramming that looks a lot like UML but maybe works better, along with 4Diac software package, consideration of how some of the more esoteric aspects of OPC fit into all this, and more slightly obliquely, TLA+.