4 ms·

Sorry, I still don’t buy it. How often do you really have a set of features that all need to interact directly with every other feature in a well-designed syste

Sorry, I still don’t buy it. How often do you really have a set of features that all need to interact directly with every other feature in a well-designed system, other than for small N like, say, an MVC triangle? Very often, a combination of better software design and standardisation can reduce the actual complexity of an implementation below the apparent complexity. For example, suppose you have a system where N components can produce a certain type of data and another M components can then consume that data. This might appear to be a system with N×M connections between components. However, these components don’t necessarily need to be directly glued together in the implementation. If you define a standard, canonical representation for this type of data, you only need each supplier to convert into that format and each consumer to convert from it, and now you have an N+M glue problem instead of the apparently N×M problem you started with.

tobr 5y ago
Of course you can do better than just blindly gluing everything onto everything. But as a general strategy you’re not really escaping N×M. The data format needs to support everything, and all of the consumers need to be adapted to any change in any producer. That’s your glue!
- Chris_Newton 5y ago
  If your design is data-driven, and the data specification in turn comes from your requirements, why would you be updating your common, standardised data format just because any particular supplier or consumer of that data changed? The dependencies appear to be the wrong way around in that situation. If there is a material change in requirements, you update your data interchange standard accordingly, and then you still only need to adapt N+M producers and consumers to work with the updated standard. Either way, I still don’t see how we end up back with N×M. The consumers and suppliers are still communicating via the intermediary format and not all talking directly to each other.
  - tobr 5y ago
    But this is a “let them eat cake” argument. It doesn’t matter that there are strategies to mitigate the problem. Most parts of most systems don’t communicate through a standardized data format, but an ad-hoc interface that addresses the immediate needs of the current requirements.
    - Chris_Newton 5y ago
      Most parts of most systems don’t communicate through a standardized data format, but an ad-hoc interface that addresses the immediate needs of the current requirements. That’s actually a very big claim, and I don’t know how we could possibly test it. Counterexamples are easy to find, however. Consider that you’re reading this in a web browser that is communicating with a web server hosting HN. That communication, as indeed the article itself observes, happens via several standard protocols layered atop one another. I didn’t really understand the argument the article made at that point that this doesn’t solve the N² problem because it doesn’t take into account the application data at the top layer, as in practice almost everything sent over the Internet is also using standard application protocols like HTTP or SMTP as well. This has enabled literally billions of devices to communicate many types of data effectively on a global scale with a wide variety of software implementations on both ends of each connection, so IMHO it’s a pretty convincing example of the benefits of effective standardisation. Of course it’s still true that some software systems have connections all over the place using ad-hoc data formats and protocols, but often that is simply bad design. Software design is a skill and like any other skill you don’t get good at it until you’ve worked on it for a while. My argument here, in a nutshell, is that if you learn good design skills and make the effort to establish good software architecture and useful standards for a project, very often the N² problem described in the article isn’t such a problem after all.
drewcoo 5y ago
That's also a big O, so it's worst-case bounded by something on that order. By saying "how often do you," you seem to be agreeing.
- Chris_Newton 5y ago
  Sorry, I’m not following your argument. What is also big-O? And why is suggesting that the relevant scenario rarely if ever happens in practice suggesting agreement with anything?
wcarss 5y ago
I've heard this kind of expression differently formulated (I think often by Steven Sinofsky while he was the head of Windows) as the complexity of the software unavoidably increasing quadratically with either time or with feature-count. So maybe it's less about the # of systems here specifically, more about just the # of "things it does", where most of what it does is connect systems together.

tobr 5y ago

Of course you can do better than just blindly gluing everything onto everything. But as a general strategy you’re not really escaping N×M. The data format needs to support everything, and all of the consumers need to be adapted to any change in any producer. That’s your glue!

Chris_Newton 5y ago
If your design is data-driven, and the data specification in turn comes from your requirements, why would you be updating your common, standardised data format just because any particular supplier or consumer of that data changed? The dependencies appear to be the wrong way around in that situation. If there is a material change in requirements, you update your data interchange standard accordingly, and then you still only need to adapt N+M producers and consumers to work with the updated standard. Either way, I still don’t see how we end up back with N×M. The consumers and suppliers are still communicating via the intermediary format and not all talking directly to each other.
- tobr 5y ago
  But this is a “let them eat cake” argument. It doesn’t matter that there are strategies to mitigate the problem. Most parts of most systems don’t communicate through a standardized data format, but an ad-hoc interface that addresses the immediate needs of the current requirements.
  - Chris_Newton 5y ago
    Most parts of most systems don’t communicate through a standardized data format, but an ad-hoc interface that addresses the immediate needs of the current requirements. That’s actually a very big claim, and I don’t know how we could possibly test it. Counterexamples are easy to find, however. Consider that you’re reading this in a web browser that is communicating with a web server hosting HN. That communication, as indeed the article itself observes, happens via several standard protocols layered atop one another. I didn’t really understand the argument the article made at that point that this doesn’t solve the N² problem because it doesn’t take into account the application data at the top layer, as in practice almost everything sent over the Internet is also using standard application protocols like HTTP or SMTP as well. This has enabled literally billions of devices to communicate many types of data effectively on a global scale with a wide variety of software implementations on both ends of each connection, so IMHO it’s a pretty convincing example of the benefits of effective standardisation. Of course it’s still true that some software systems have connections all over the place using ad-hoc data formats and protocols, but often that is simply bad design. Software design is a skill and like any other skill you don’t get good at it until you’ve worked on it for a while. My argument here, in a nutshell, is that if you learn good design skills and make the effort to establish good software architecture and useful standards for a project, very often the N² problem described in the article isn’t such a problem after all.

drewcoo 5y ago

That's also a big O, so it's worst-case bounded by something on that order. By saying "how often do you," you seem to be agreeing.

Chris_Newton 5y ago
Sorry, I’m not following your argument. What is also big-O? And why is suggesting that the relevant scenario rarely if ever happens in practice suggesting agreement with anything?

wcarss 5y ago

I've heard this kind of expression differently formulated (I think often by Steven Sinofsky while he was the head of Windows) as the complexity of the software unavoidably increasing quadratically with either time or with feature-count. So maybe it's less about the # of systems here specifically, more about just the # of "things it does", where most of what it does is connect systems together.