5 ms·

A sketch of the biggest idea in software architecture

113 points by harperlee 4y ago

jeffrallen 4y ago
Small pieces, loosely coupled.
- numeromancer 4y ago
  Like a glass jar that has been dropped onto a stone floor.
  - PaulDavisThe1st 4y ago
    Or like the coins that were in the jar, whose lid was stuck. Before, they were a large mass of tightly coupled metallurgy. Now, they're actually useful.
codeflo 4y ago
The text tries to introduce the idea of a “narrow waist” as an architecture pattern. But I don’t fully understand how the author distinguishes this concept from similar ones, and some of the examples even seem to contradict the principles. It supposedly solves the N*M problem (so it’s an interface?), but it can’t be statically typed (so the interface has to be implicit?), and success stories are HTML5 and the x86 ABI (so it does require explicit specification after all, and huge amounts of it? Just look at the size of that Intel ABI book…), but also, it must be plaintext (so not like x86 at all?). I’m not trying to be mean because maybe there’s something there that I miss, but frankly, I don’t get it, at least not yet.
- zaphar 4y ago
  He's not saying it can't be statically typed. He's saying it's type specificity is very coarse grained. A stream of bytes has a well defined static type. But it's a very coarse grained one. As a method of communication a stream of bytes can express or contain almost anything. It's narrow in that it's type does not prevent any specific compositions or expressions of a wide range of things. Similarly plaintext is an example not a requirement. Like any analogy it breaks down at the extremes but that doesn't mean the analogy is a bad one.
- Diggsey 4y ago
  Yeah I think it's a bit of a stretch to try to combine all these ideas under a single name. One guideline I would follow is: Statically type everything that a service needs to inspect to do its job, and no more, but without introducing a layering violation. So, for example if a component accepts this input: { "timestamp": "2022-06-17 00:00:00", "data": { ... } } If it only uses "timestamp" to decide whether to discard the input, and then passes "data" to some other service, you would definitely want to make timestamp be a "datetime" type rather than a string. "data" should be an arbitrary JSON value, but it should still be parsed as JSON, just no additional structure enforced.
  - swatcoder 4y ago
    I wish real projects were so simple! Guidelines like this are operating on something like the Platonic ideal shape of problems. In any real world project, you need to consider the cost of handing a malformed `data` blob downstream, as it's never free. Architectural model or pattern might be a better word than guideline. You benefit by collecting a catalog of patterns more than you do by trying to internalize guidelines.
- moss2 4y ago
  What is the N*M problem? I can't find anything on google (except a scientific figure behind a paywall).
  - deleted 4y ago
    [deleted]
  - chrisshroba 4y ago
    If you have N things producing data, and M things consuming data, you have a problem where each of the M consumers needs to handle all N types of input data, meaning you have to implement N*M things. But if you have a "narrow waste", meaning a common interface like JSON that all N things are capable of producing and all M things are capable of consuming, then you only need to implement N+M things (N things that produce JSON, and M things that consume JSON).
    - deleted 4y ago
      [deleted]
    - TheOtherHobbes 4y ago
      But you still have to do some validation and presumably some type checking. So this redistributes the problem instead of solving it. JSON is a file and interchange format - more technically a protocol. It is not a data type.
      - jmt_ 4y ago
        > JSON is a file and interchange format - more technically a protocol. > It is not a data type. Although me, and I'm sure many others, have abused JSON and used it as a data type, I totally agree with you. However, GP was using JSON as an example of a more general idea. I needed to aggregate similar data across many different tables holding information collected from web scraping - typically one table corresponds to a particular service, and each service has it's own spider. My approach was to define a data model that contains all the information that I care about then write classes per service to convert its data to the format of the aforementioned data model. While the model can be serialized as JSON, it's composed of standard Python data types/classes and I don't constrain its design to be more amicable to serialization. Point being, while JSON is an intuitive example and can be a good go-to if your data matches it well, there's many ways to solve the N*M problem in this particular context.
- winternett 4y ago
  The post works hard to normalize the term "Narrow Waist" to the point that's it's burdensome. Using nondescriptive terms to define a wide range of services, languages, and protocols only alienates clients and serves to differentiate academics from the rest in the development talent pool. Development over time should become more relatable over time, not more academic unless we're talking about the machine level perhaps... It's why employers are having a really hard time trying to find angular, python, C, and rust devs despite PHP and .NET still dominating in most web based apps and sites. It would be interesting to know exactly how many no/low-code solutions are deployed in comparison to custom code solutions, and what type of business cases they serve currently... I think the results of that study would really make a lot of devs switch fields of expertise. Almost every employer is calling me about DevSecOps positions, while app design and development are an afterthought due to prevalence of low/no-code solutions because langs have become so convoluted and specialized in hopes of creating a monopolistic work dependency... Sorry to vent, but solutions simply are overcomplicated too often now, and most of the people making technical decisions are non technical, and not equipped to keep up with the ever-changing terminology and jargon used in the dev world, much less knowing the differences in languages and terminology. Over complexity and tech jargon are some of the most painfully alienating detriments to development in my experience. I think we would do a lot of good if we really got back to breaking down use cases and their individual (right-fit) solutions rather than sticking to redefining entire disciplines and forcing new terms that aren't properly descriptive frameworks, tools, and methods onto the industry as if one great new shoe fits all feet... You know, simpler jargon is better.
zaphar 4y ago
This is an excellent overview of the narrow waist concept which is a highly zoomed out. I think it's hard to see sometimes because when you are working within a narrow waist you are typically working on something that looks more like a series of filters in one or both ends of the hour glass. The fundamental compromise that the narrow waist is much more general than the specific domain you are working in means it fades to the background and instead you are working to handle all the ways that too general a space needs to be specialized.
bluesnowmonkey 4y ago
It seems like a restatement of the end-to-end principle. He does call it out as being different but… still seems pretty similar to me. Build apps on simple reusable abstractions like byte streams and message passing.
kabdib 4y ago
Describing Win32 as a "narrow waist" (hundreds of interfaces, thousands of calls, no idea how many structures) seems off the mark. I don't get it. It seems like the target got painted where the arrow landed.
- dotopotoro 4y ago
  > target got painted where the arrow landed. Commenting mainly to say that I like this phrase very very much. Creative, Witty and at the same time so descriptive. On the win32 point, it is adressed in the article: > I also realized that there are two distinct senses of the word "narrow": > - In terms of architectural connection (topology). For example, applications and physical networks are decoupled by the the Internet's narrow waist. They don't interface directly with each other. > - In terms of the size of the concept. <snip> But the web is a large set of concepts (HTTP, HTML, SVG, etc.).
readthenotes1 4y ago
I was hoping for a ... sketch
- dcow 4y ago
  Me too. This is a jumbled stream of conscious style mess of information, even if the topic is interesting.
dcow 4y ago
I don't really know what I'm supposed to derive from this style of "blog post". It feels like the author wrote down a bunch of bullet points over the last 6 months of software development on Oil and just decided to publish them. There are lots of good links and cool trinkets (like projecting json onto a filesystem), and I'm now, having read between the lines, familiar with the term "narrow waist", but I'm not convinced of anything in particular because there's not a strongly argued thesis. There are a bunch of small assertions, some contradictory. I really struggled to get a "sketch" of.. anything.
chriswarbo 4y ago
Somewhat related is the Lispy approach of using generic data structures (e.g. cons lists); compared to the OOP approach of defining new (hierachies of) datastructures/types/interfaces (usually classes) for each use-case. With the former, there's an ever-growing set of functions we call on our data; whilst the latter is incompatible with all existing code.
- convolvatron 4y ago
  having a common syntax gets you a crude kind of type-generic behavior. you can can write functions that just deal with 'stuff', including functions to merge data and to serialize/unseriaze. maybe you also get a base type system. maybe include some transport functions like idempotency...write a little data viewer. you can build stuff on top of this - including additional semantics, like adding a schema with the same underlying representation in order to do validation. or transactions - nothing to say say that the body can't be another one of these generic data items wrapped up in a transaction envelope - or signed like a cert. I dislike 'narrow waist', although I understand the image. it seems to me to really be about what everything always has been - separation of concerns and low level functions that support reuse.
al2o3cr 4y ago
[flagged]
Zababa 4y ago
Another obvious narrow waist is money. Language might be one, especially English today.
- thgs 4y ago
  language is one! Thinking of that though, makes me lose the barrier between narrow waist and any interface whatsoever. All of them become a narrow waist all of a sudden.
analog31 4y ago
Note for laypeople like me the glossary accompanying the article is really super.
rawgabbit 4y ago
ELI5? The ideas are interesting but I need help understanding.
- dotopotoro 4y ago
  https://news.ycombinator.com/item?id=31778505

113 points by harperlee 4y ago

jeffrallen 4y ago

Small pieces, loosely coupled.

numeromancer 4y ago
Like a glass jar that has been dropped onto a stone floor.
- PaulDavisThe1st 4y ago
  Or like the coins that were in the jar, whose lid was stuck. Before, they were a large mass of tightly coupled metallurgy. Now, they're actually useful.

codeflo 4y ago

The text tries to introduce the idea of a “narrow waist” as an architecture pattern. But I don’t fully understand how the author distinguishes this concept from similar ones, and some of the examples even seem to contradict the principles. It supposedly solves the N*M problem (so it’s an interface?), but it can’t be statically typed (so the interface has to be implicit?), and success stories are HTML5 and the x86 ABI (so it does require explicit specification after all, and huge amounts of it? Just look at the size of that Intel ABI book…), but also, it must be plaintext (so not like x86 at all?). I’m not trying to be mean because maybe there’s something there that I miss, but frankly, I don’t get it, at least not yet.

zaphar 4y ago
He's not saying it can't be statically typed. He's saying it's type specificity is very coarse grained. A stream of bytes has a well defined static type. But it's a very coarse grained one. As a method of communication a stream of bytes can express or contain almost anything. It's narrow in that it's type does not prevent any specific compositions or expressions of a wide range of things. Similarly plaintext is an example not a requirement. Like any analogy it breaks down at the extremes but that doesn't mean the analogy is a bad one.
Diggsey 4y ago
Yeah I think it's a bit of a stretch to try to combine all these ideas under a single name. One guideline I would follow is: Statically type everything that a service needs to inspect to do its job, and no more, but without introducing a layering violation. So, for example if a component accepts this input: { "timestamp": "2022-06-17 00:00:00", "data": { ... } } If it only uses "timestamp" to decide whether to discard the input, and then passes "data" to some other service, you would definitely want to make timestamp be a "datetime" type rather than a string. "data" should be an arbitrary JSON value, but it should still be parsed as JSON, just no additional structure enforced.
- swatcoder 4y ago
  I wish real projects were so simple! Guidelines like this are operating on something like the Platonic ideal shape of problems. In any real world project, you need to consider the cost of handing a malformed `data` blob downstream, as it's never free. Architectural model or pattern might be a better word than guideline. You benefit by collecting a catalog of patterns more than you do by trying to internalize guidelines.
moss2 4y ago
What is the N*M problem? I can't find anything on google (except a scientific figure behind a paywall).
- deleted 4y ago
  [deleted]
- chrisshroba 4y ago
  If you have N things producing data, and M things consuming data, you have a problem where each of the M consumers needs to handle all N types of input data, meaning you have to implement N*M things. But if you have a "narrow waste", meaning a common interface like JSON that all N things are capable of producing and all M things are capable of consuming, then you only need to implement N+M things (N things that produce JSON, and M things that consume JSON).
  - deleted 4y ago
    [deleted]
  - TheOtherHobbes 4y ago
    But you still have to do some validation and presumably some type checking. So this redistributes the problem instead of solving it. JSON is a file and interchange format - more technically a protocol. It is not a data type.
    - jmt_ 4y ago
      > JSON is a file and interchange format - more technically a protocol. > It is not a data type. Although me, and I'm sure many others, have abused JSON and used it as a data type, I totally agree with you. However, GP was using JSON as an example of a more general idea. I needed to aggregate similar data across many different tables holding information collected from web scraping - typically one table corresponds to a particular service, and each service has it's own spider. My approach was to define a data model that contains all the information that I care about then write classes per service to convert its data to the format of the aforementioned data model. While the model can be serialized as JSON, it's composed of standard Python data types/classes and I don't constrain its design to be more amicable to serialization. Point being, while JSON is an intuitive example and can be a good go-to if your data matches it well, there's many ways to solve the N*M problem in this particular context.
winternett 4y ago
The post works hard to normalize the term "Narrow Waist" to the point that's it's burdensome. Using nondescriptive terms to define a wide range of services, languages, and protocols only alienates clients and serves to differentiate academics from the rest in the development talent pool. Development over time should become more relatable over time, not more academic unless we're talking about the machine level perhaps... It's why employers are having a really hard time trying to find angular, python, C, and rust devs despite PHP and .NET still dominating in most web based apps and sites. It would be interesting to know exactly how many no/low-code solutions are deployed in comparison to custom code solutions, and what type of business cases they serve currently... I think the results of that study would really make a lot of devs switch fields of expertise. Almost every employer is calling me about DevSecOps positions, while app design and development are an afterthought due to prevalence of low/no-code solutions because langs have become so convoluted and specialized in hopes of creating a monopolistic work dependency... Sorry to vent, but solutions simply are overcomplicated too often now, and most of the people making technical decisions are non technical, and not equipped to keep up with the ever-changing terminology and jargon used in the dev world, much less knowing the differences in languages and terminology. Over complexity and tech jargon are some of the most painfully alienating detriments to development in my experience. I think we would do a lot of good if we really got back to breaking down use cases and their individual (right-fit) solutions rather than sticking to redefining entire disciplines and forcing new terms that aren't properly descriptive frameworks, tools, and methods onto the industry as if one great new shoe fits all feet... You know, simpler jargon is better.

zaphar 4y ago

This is an excellent overview of the narrow waist concept which is a highly zoomed out. I think it's hard to see sometimes because when you are working within a narrow waist you are typically working on something that looks more like a series of filters in one or both ends of the hour glass. The fundamental compromise that the narrow waist is much more general than the specific domain you are working in means it fades to the background and instead you are working to handle all the ways that too general a space needs to be specialized.

bluesnowmonkey 4y ago

It seems like a restatement of the end-to-end principle. He does call it out as being different but… still seems pretty similar to me. Build apps on simple reusable abstractions like byte streams and message passing.

kabdib 4y ago

Describing Win32 as a "narrow waist" (hundreds of interfaces, thousands of calls, no idea how many structures) seems off the mark. I don't get it. It seems like the target got painted where the arrow landed.

dotopotoro 4y ago
> target got painted where the arrow landed. Commenting mainly to say that I like this phrase very very much. Creative, Witty and at the same time so descriptive. On the win32 point, it is adressed in the article: > I also realized that there are two distinct senses of the word "narrow": > - In terms of architectural connection (topology). For example, applications and physical networks are decoupled by the the Internet's narrow waist. They don't interface directly with each other. > - In terms of the size of the concept. <snip> But the web is a large set of concepts (HTTP, HTML, SVG, etc.).

readthenotes1 4y ago

I was hoping for a ... sketch

dcow 4y ago
Me too. This is a jumbled stream of conscious style mess of information, even if the topic is interesting.

dcow 4y ago

I don't really know what I'm supposed to derive from this style of "blog post". It feels like the author wrote down a bunch of bullet points over the last 6 months of software development on Oil and just decided to publish them. There are lots of good links and cool trinkets (like projecting json onto a filesystem), and I'm now, having read between the lines, familiar with the term "narrow waist", but I'm not convinced of anything in particular because there's not a strongly argued thesis. There are a bunch of small assertions, some contradictory. I really struggled to get a "sketch" of.. anything.

chriswarbo 4y ago

Somewhat related is the Lispy approach of using generic data structures (e.g. cons lists); compared to the OOP approach of defining new (hierachies of) datastructures/types/interfaces (usually classes) for each use-case. With the former, there's an ever-growing set of functions we call on our data; whilst the latter is incompatible with all existing code.

convolvatron 4y ago
having a common syntax gets you a crude kind of type-generic behavior. you can can write functions that just deal with 'stuff', including functions to merge data and to serialize/unseriaze. maybe you also get a base type system. maybe include some transport functions like idempotency...write a little data viewer. you can build stuff on top of this - including additional semantics, like adding a schema with the same underlying representation in order to do validation. or transactions - nothing to say say that the body can't be another one of these generic data items wrapped up in a transaction envelope - or signed like a cert. I dislike 'narrow waist', although I understand the image. it seems to me to really be about what everything always has been - separation of concerns and low level functions that support reuse.

al2o3cr 4y ago

[flagged]

Zababa 4y ago

Another obvious narrow waist is money. Language might be one, especially English today.

thgs 4y ago
language is one! Thinking of that though, makes me lose the barrier between narrow waist and any interface whatsoever. All of them become a narrow waist all of a sudden.

analog31 4y ago

Note for laypeople like me the glossary accompanying the article is really super.

rawgabbit 4y ago

ELI5? The ideas are interesting but I need help understanding.

dotopotoro 4y ago
https://news.ycombinator.com/item?id=31778505