6 ms·
A Sketch of the Biggest Idea in Software Architecture (2022)
- loandbehold 3y agoThis guy just gave a cool name to an old concept.
- cjk2 3y agoThat's Martin Fowler's job :)
- rwmj 3y agoI first heard about this concept in the early 1990s from netpbm (then called pbmtools). The exact language seems to have been removed from the current manual[1] but the basic idea was that if you have N image formats, you can either write NxN tools to convert between them, or you can have a few common image formats (ie. pbm - bitmap, pgm - greyscale and ppm - fullcolour) and then write 2xN tools to convert between the universe of all image formats and the common formats. You then use Unix pipes to connect the tools together. [1] https://netpbm.sourceforge.net/doc/ https://netpbm.sourceforge.net/doc/
- nottorp 3y agoExcept in that case you're converting to and from an uncompressed buffer... that could be in ram and not use a file format at all?
- rwmj 3y agoIt's much more useful to have a real file format. For example, you can generate images from your own code by doing this (with a bit more shell quoting and error checking in the real code): fp = popen ("ppmtojpeg > output.jpg", "w"); fprintf (fp, "P3 1024 1024 255\n"); for (y = 0; y < 1024; ++y) for (x = 0; x < 1024; ++x) fprintf (fp, "%d %d %d\n", 255*r[y][x], 255*g[y][x], 255*b[y][x]); fclose (fp);
- frutiger 3y agoThis is the first time I have seen anyone popen’ing a pipeline like that, and writing to its stdin. Bravo.
- ramses0 3y agoI miss a bit some of the old `xxx2yyy` programs. It's tempting to consider: FILE.csv.html.pdf.zip ...`csv2html | html2pdf | pdf2zip`, and vice-versa... how possible is it to make the pipeline reversible, eg: `zip2pdf | pdf2html | html2csv`, or in the absence of "reversibility", could you simply shortcut and get straight back to the original `*.csv`?
- deleted 3y ago[deleted]
- diego_moita 3y agoOther names for the same idea: "modularization", "loose coupling and strong cohesion", "program to interfaces, not implementations", ...
- rusk 3y agoParsimony
- chubot 3y ago(author here) I will probably write more about this, but it's not just modularity, or interfaces. Or at least those words are clouded by associations with specific mechanisms in whatever programming languages one has used One additional distinction that's relevant is strong vs. weak modules in the sense here: https://thunderseethe.dev/posts/whats-in-a-module/#strong-vs-weak-modules https://thunderseethe.dev/posts/whats-in-a-module/#strong-vs... Some of my comments here - https://lobste.rs/s/eccv1g/what_s_module https://lobste.rs/s/eccv1g/what_s_module and here - https://lobste.rs/s/u7y4lk/modules_matter_most_for_masses https://lobste.rs/s/u7y4lk/modules_matter_most_for_masses Harper doesn't use the term "strong modularity", but he says that REAL modularity is M x N, and that's closer to what I'm getting at: https://existentialtype.wordpress.com/2011/04/16/modules-matter-most/ https://existentialtype.wordpress.com/2011/04/16/modules-mat... To be most useful, it is important that the relationship between M and A be many-to-many, and not many-to-one or one-to-one. (Although I disagree with the assertions about static types -- I think he is making a very domain specific argument, nonetheless we agree on strong modularity) It needs more analysis, but I'd claim there are at least two kinds of modularity and programming to interfaces, but the difference is not clear to most people The interior vs. exterior distinction is a big one -- is the interface "owned by" one party, or does it stand alone? Are you able to do "interior" refactorings that change the interface, or is it fixed, outside the project? This is a big difference in practice
- rusk 3y agoparsimonius interfaces must have been taken
- chubot 3y ago(author here) It's an old name for an old concept :) See the previous post The Internet Was Designed With a Narrow Waist, especially the appendix with some history https://www.oilshell.org/blog/2022/02/diagrams.html https://www.oilshell.org/blog/2022/02/diagrams.html The name was apparently coined by Jon Postel in the late 1970's, but this fact is not well known. There are some misattributions in the appendix I apparently learned the term from a talk by Van Jacobsen on Named Data Networking at Google in 2006, but I forgot that I learned it that way! After I wrote the post, a reader Caleb Malchik reminded me about Van Jacobsen's talk, and then I went back and watched it, and realized that's where I got the term from! I then updated the appendix. --- I have searched Hacker News for "narrow waist", and it surprised me that it's not a very well known idea, despite being old and important. (And again - I wrote these articles specifically to gather related material from others, so I welcome any updates or corrections.) One place you can see it in passing is in a 2013 paper from Steven Kell - The operating system: should there be one? https://www.humprog.org/~stephen/research/papers/kell13operating.pdf https://www.humprog.org/~stephen/research/papers/kell13opera... Search for "waist in the hourglass" --- If you think about it, it is not surprising that the "narrow waist" concept from networking is becoming more important to software. Software has become more and more networked for the last 2 decades, and I don't think that trend has even topped out. It is hard to find any kind of software that is not networked. As an example, in 2002 I worked at EA on shrink wrapped video games. They were completely offline embedded systems. But 2004 was the first year we added networking to the game, and now pretty much all games are networked in some way. Also, the Internet is the probably the biggest and most successful thing ever engineered ... (not sure what you would compare it to, maybe nuclear reactors or aircraft carriers). It would be surprising if there weren't some lessons to learn from its engineering!
- Djeman 3y agoAircraft carriers and nuclear reactors are exactly not anything like internet. Huge physical engineering projects are like a huge jigsaw puzzle. Take one piece and picture is not complete. Internet is network, most physical thing to compare with is electric grid. But then again whole electric grid hums with one single frequency.
- samatman 3y ago
- larsrc 3y agoProtocol Buffers are the narrow waist of Google.
- selfie 3y ago[flagged]
- blowski 3y agoA comment from a previous discussion of this post is still true for me: > The text tries to introduce the idea of a “narrow waist” as an architecture pattern. But I don’t fully understand how the author distinguishes this concept from similar ones, and some of the examples even seem to contradict the principles. https://news.ycombinator.com/item?id=31777080 https://news.ycombinator.com/item?id=31777080
- mrkeen 3y agoThe article's a little meandering. I think the 'waist' in this case is just 'interface', in the general sense - i.e. where two pieces of software meet. You make the {N-waist-M} composition problem easier because not much detail can leak from N to M and vice-versa. N and M only need to know about e.g. JSON to compile, not each other. I'd consider a slightly fatter waist to be something like LLVM. My language is wide and complicated, and all the various assembler outputs for different architectures are also wide and complicated, and the idea of LLVM is to bridge the two, but it still feels like it would be a pain to couple my language to LLVM (JSON is narrower than LLVM)
- blowski 3y agoIf I’ve understood then, a narrow waist architecture follows two principles: - code to an interface not an implementation - segregate interfaces, don’t overload them with responsibilities
- jeisc 3y agobring it on the more the merrier in any endeavor to build the ultimate mousetrap. Now we just need a name for our humanity which will reverse our species suicidal and violent tendencies.
- mondrian 3y agoIs the narrow waste concept useful? Sure everything is bytes but most things overlay structure on top of bytes, and treating those structures as bytes is probably not useful. You could use “wc” to count the lines/bytes in a json file but counting bytes is likely one of the least useful/interesting things you could do with that json file.
- delusional 3y agoUnless you wanted to compress that json. If you insisted on the structure, you'd have to create a specific algorithm just for json. Because json is just bytes, you get to compress it with a generic byte compression algorithm.
- rusk 3y agoIs json just bytes tho? Doesn’t it allow for variable width encodings?
- ElevenLathe 3y agoNo matter what encoding your JSON file is, gzip will output a compressed bag of bytes that, when unzipped, will result in the same file coming out the other end. This is true of movie codecs, Word 97 files, or anything, and none of the maintainers of those formats had to be consulted about this in order to make it work. That's what is meant by "thin waist" here.
- rusk 3y agoI know, but it’s not “just bytes” as per parent comment. You cannot infer the length of the content without decoding it. “By definition” it is variable width character data. I think it’s fair to be pedantic vs a fairly dramatic oversimplification.
- ElevenLathe 3y agoLess specific interfaces let you do less interesting things, but are more resilient. It's an engineering tradeoff. Purpose-built interfaces that fully expose and understand domain-level semantics are great in certain circumstances, but other times you want a certain minimum abstraction (IP packets and 'bags-of-bytes' POSIX file semantics are good examples) that can be used to build better ones. If the rollout of HTTP had required that all the IP routers on the internet be updated to account for it, we likely would not have it. Likewise, if we required that all the classic Unix text utilities like wc, sort, paste, etc. did meaningful things to JSON before we could standardize JSON, adoption would likely have suffered.
- xuhu 3y agoSo many comments posted using dictation in this thread!
- speed_spread 3y agoThe biggest ideas in software architecture are developed while stuck in monster traffic jams. "Siri, create a new blog post."
- cjk2 3y agoOn the PTA here, the Porcelain Throne of Architecture.
- doubloon 3y agoFeature detection is better than version detection…??? Maybe for the web but jn compiled language world This has led to autoconf automake which is a gigantic waist and also a gigantic waste. Rust style cargo library version specification solves so many problems that are obvious…. We want to comple and link with the exact same version of library code that we built and tested on , because you cannot possibly test every feature nor can you test bug compatibility. This increases portability , not decreases, meanwhile C programs are not even portable between different versions of linux due to library issues. Im skeptical of some of these ideas. The most successful thinwaist of all is VBA and Excel which of course are never mentioned .
- nottorp 3y ago> JSON was an explicit design, and it's much better than CSV. Interesting. Is it better, or is it easier (and cheaper) to parse?
- bazoom42 3y agoCSV is not standardized, it might use comma or semicolon as seperator, there are different conventions for quoting and escaping, headers might be included or not. JSON is also hierarchical which makes it appropriate for a wider range of data structures. That said, I’m not sure there is any agreed upon standard for representing data tables in JSON, so it might not be any better for the CSV use case.
- rusk 3y agoI think most people hold RFC-4180 to be the CSV standard, even if it’s not officially a standard. It is sufficiently parametrised to suit all variants while at the same time providing a language and structure that is portable across all. Don’t get hung up on the fact that RFC’s aren’t officially standards. It’s a publicly accessible standard which many people fall back on as a reference, which is more than you can say for a lot of ISO standards.
- marcosdumay 3y agoOh, you can hold it all you want, popular implementations still break it.
- SkyPuncher 3y agoExactly, the files my users submit are rarely spec compliant. I have to process them properly regardless of correctness.
- lelanthran 3y agoThe text parsing[1] is not really relevant, I feel. It's what comes after parsing that's a problem. With CSV you have to guess at what the delimiter is, you have to guess if the first row are column names, you have to guess what each element is (string, date, integer, etc), you have to guess how escaped characters are represented. The good news is that from a medium sized input, all of these would be pretty simple to guess while parsing each cell. The bad news is that, if you receive only a single row, you can't very well tell WTH the types are. With JSON you at least get to read numbers as numbers and strings as strings (and special symbols as special symbols, like `null`). The downside of JSON is that, like CSV, you still have to specify what the structure of the data is to a receipient, except that, unlike CSV, it's more difficult and prone to error. [1] Although, parsing CSV is so simple you can frequently just write the parser yourself.
- tuyiown 3y agoI like this very much, but I'm not quite sure if it actually touches the full depth of the subject or if it's accidental hindsight analysis. I'm not sure I can't settle with the idea that big things gets there because they are useful simple things that people get stuck with.
- danielovichdk 3y agoLanguage standards like POSIX shell, JavaScript, and C++. ...These are narrow waists because they solve the interoperability problem of {user programs ...} × {language implementations ...} Calling those languages "narrow waist" due to interoperability might be true but it says a lot more about how stuck we are as a field than how great those languages are at interoperability. Both languages are, in each their own, complicated at best.
- cappyobvo 3y agoSo many words to say "prefer tools that deal with text protocols and formats and never break backwards compatibility." The article cites the HTTP protocol as an example of this. However only HTTP/0.9 and 1 are text format - HTTP/2 and 3 are binary.