5 ms·

Except I never dismissed this point. The problem with your post is that you assume I categorize design as something useless. I have not. Obviously there are ton

Except I never dismissed this point. The problem with your post is that you assume I categorize design as something useless. I have not. Obviously there are tons and tons of things within the universe where the only possible solution is design. I am not arguing against this. I am talking about a very specific aspect of the usage of "design" within software. I am specifically complaining about the endless iterations of exposes on software design patterns and architecture. The trends where history continuously repeats itself with FP becoming popular OOP becoming more popular than FP becoming popular again. What about the whole microservices/monoliths argument where monoliths started out more popular than microservices became popular and now monoliths are coming back into vogue again? Endless loops where nobody knows what is the optimal solution for specific contexts. These are all methods of software organization AND my point is that THIS specific aspect of design is ripe for formalization especially with the endless deluge of metaphor drenched pointless exposes on "design" that are inundating the HN front page feed. It's obviously an endless circle of history repeating itself. I am proposing a way to break the loop for a specific aspect of software by pointing out the distinction between "design" and "formal theory." We all know the loop exists because people confuse the two concepts and fail to actually even know what to do to optimize something. System architects are artisans not scientists and they will as a result suffer from the exact same pointless shifts in artistic styles/trends decade after decade and year after year as their artisan peers do. Totally ok for styles to shift, but our goal in software is to converge at an optimum as well and that's not currently happening in terms of software patterns and design architecture. The path out of this limbo is to definitively identify the method for optimization formally, not add to the teeming millions of articles talking about software design metaphors.

zekrioca 6y ago
Any idea on how to formalize this area? Is anyone even trying to do that?
- Geminidog 6y ago
  There's many topics in this area. Ones that are well known in industry are algorithmic complexity theory and type theory. Ones that are less well known include the two resources below. http://www4.di.uminho.pt/~jno/ps/pdbc.pdf http://www4.di.uminho.pt/~jno/ps/pdbc.pdf https://softwarefoundations.cis.upenn.edu https://softwarefoundations.cis.upenn.edu I suggest you get use to ML style languages before diving into those two resources (Haskell is a good choice) as it's not easy to learn this stuff and I think it's also part of the reason why it hasn't been so popular in industry. The first resource builds towards a prolog like programming style where you feed the computer a specification and the computer produces a program that fits the specification. The second resource involves utilizing a language with a type checker so powerful that the compiler can fully prove your program correct outside of just types. Both are far away from the ideal that the industry is searching for but in terms of optimizing and formalizing design these two resources are examples of the right approach to improving software design. I haven't found anything specifically on the organization of software modules so as far as I know none exists. But given the wide scope of software research I'm sure at least one paper has talked about this concept.
  - Twisol 6y ago
    > I haven't found anything specifically on the organization of software modules so as far as I know none exists. Parnas (1971) is seminal on this topic. https://apps.dtic.mil/sti/pdfs/AD0773837.pdf https://apps.dtic.mil/sti/pdfs/AD0773837.pdf
    - Geminidog 6y ago
      Sure this paper introduces the concept of modules in software back when modules were non-existent. As far as I know, there's nothing on the optimal way to organize (keyword) modules.
      - Twisol 6y ago
        > Sure this paper introduces the concept of modules in software back when modules were non-existent. No... that's not correct. The very first non-clerical page quotes from a textbook discussing modular systems, and Parnas himself notes a distinct lack of material on how to actually organize and break down the system into modules. The paper is literally called "On the criteria to be used in decomposing systems into modules." It is prudent to at least read the preexisting material if you are going to dismiss it.
        Geminidog 6y ago
        My mistake, I skimmed it and saw assembly language and I assumed it was that really early seminal paper that introduced modules to the programming world. Obviously my guess was wrong.
- Twisol 6y ago
  Personally, I'm nursing a thesis that the study of concurrency is fertile ground for a formalization of modular design. Where parallelism is the optimization of a software system by running parts of it simultaneously, concurrency has much more to do with the assumptions held by individual parts of the program, and how knowledge is communicated between them. Parallelism requires understanding these facets insofar as the assumptions need to be protected from foreign action -- or insfar as we try to reduce the need for those assumptions in the first place -- but I expect that concurrency goes much further. Concurrent constraint programming is a nifty approach in this vein -- it builds on a logic programming foundation where knowledge only increases monotonically, and replaces get/set on registers with ask/tell on lattice-valued cells. LVars is a related (but much more recent) approach. A different approach, "session types", works at the type system level. Both ends of a half-duplex (i.e. turn-taking) channel have compatible (dual) signatures, such that one side may send when the other side may receive. Not everything can be modeled with half-duplex communications, but the ideas are pretty useful to keep in mind. I try to keep my software systems as functional as possible (where "functional" here means "no explicit state"). But there are always places where it makes sense to think in terms of state, and so I try to model that state monotonically whenever possible. At least subjectively, it's usually a lot simpler (and easier to follow) than unrestricted state. (Note, of course, that local variables are local in the truest sense: other programmatic agents cannot make assumptions about them or change them. Short-lived, local state is as good as functional non-state in most cases.)
  - yowlingcat 6y ago
    > I try to keep my software systems as functional as possible (where "functional" here means "no explicit state"). But there are always places where it makes sense to think in terms of state, and so I try to model that state monotonically whenever possible. At least subjectively, it's usually a lot simpler (and easier to follow) than unrestricted state. Agreed. You mention LVars so I'm curious what you think about MVars and STM in general. I've always been fond of STM because relational databases and their transactions are a familiar and well understood concept historically used by the industry to keep state sane and maintain data integrity. SQLite is great, but having something that's even closer the core language or standard library is even better. It's part of why I like using SQL to do the heavy lifting when possible. I like that SQL is a purely functional language that naturally structures state mutations as transactions through the write-ahead log protocol. My flavor of choice (Postgres) makes different levels of efficient read and write available through read isolation levels that can give me up to ACID consistency without having to reinvent the wheel with my read and write semantics. If I structure my data model keys, relations and constraints properly, I get a production strength implementation with a lot of the nice properties you talk about. And that's regardless of my service layer choice for my language that I can trust to stand up. There's one exception in particular that I've seen begin to gain steam in the industry which I think is interesting, and that's Elixir. Because Elixir wraps around Erlang's venerable OTP (and distributed database mnesia), users can build on the top of something that's already solved a lot of the hard distributed systems problems in the wild in a very challenging use case (telecom switches). Of course, mnesia has its own issues so most of the folks I know using Elixir are using it with Phoenix + SQL. They seem to like it, but I worry about ecosystem collapse risk with any transpiled language -- no one wants to see another CoffeeScript.

zekrioca 6y ago

Any idea on how to formalize this area? Is anyone even trying to do that?

Geminidog 6y ago
There's many topics in this area. Ones that are well known in industry are algorithmic complexity theory and type theory. Ones that are less well known include the two resources below. http://www4.di.uminho.pt/~jno/ps/pdbc.pdf http://www4.di.uminho.pt/~jno/ps/pdbc.pdf https://softwarefoundations.cis.upenn.edu https://softwarefoundations.cis.upenn.edu I suggest you get use to ML style languages before diving into those two resources (Haskell is a good choice) as it's not easy to learn this stuff and I think it's also part of the reason why it hasn't been so popular in industry. The first resource builds towards a prolog like programming style where you feed the computer a specification and the computer produces a program that fits the specification. The second resource involves utilizing a language with a type checker so powerful that the compiler can fully prove your program correct outside of just types. Both are far away from the ideal that the industry is searching for but in terms of optimizing and formalizing design these two resources are examples of the right approach to improving software design. I haven't found anything specifically on the organization of software modules so as far as I know none exists. But given the wide scope of software research I'm sure at least one paper has talked about this concept.
- Twisol 6y ago
  > I haven't found anything specifically on the organization of software modules so as far as I know none exists. Parnas (1971) is seminal on this topic. https://apps.dtic.mil/sti/pdfs/AD0773837.pdf https://apps.dtic.mil/sti/pdfs/AD0773837.pdf
  - Geminidog 6y ago
    Sure this paper introduces the concept of modules in software back when modules were non-existent. As far as I know, there's nothing on the optimal way to organize (keyword) modules.
    - Twisol 6y ago
      > Sure this paper introduces the concept of modules in software back when modules were non-existent. No... that's not correct. The very first non-clerical page quotes from a textbook discussing modular systems, and Parnas himself notes a distinct lack of material on how to actually organize and break down the system into modules. The paper is literally called "On the criteria to be used in decomposing systems into modules." It is prudent to at least read the preexisting material if you are going to dismiss it.
      - Geminidog 6y ago
        My mistake, I skimmed it and saw assembly language and I assumed it was that really early seminal paper that introduced modules to the programming world. Obviously my guess was wrong.
Twisol 6y ago
Personally, I'm nursing a thesis that the study of concurrency is fertile ground for a formalization of modular design. Where parallelism is the optimization of a software system by running parts of it simultaneously, concurrency has much more to do with the assumptions held by individual parts of the program, and how knowledge is communicated between them. Parallelism requires understanding these facets insofar as the assumptions need to be protected from foreign action -- or insfar as we try to reduce the need for those assumptions in the first place -- but I expect that concurrency goes much further. Concurrent constraint programming is a nifty approach in this vein -- it builds on a logic programming foundation where knowledge only increases monotonically, and replaces get/set on registers with ask/tell on lattice-valued cells. LVars is a related (but much more recent) approach. A different approach, "session types", works at the type system level. Both ends of a half-duplex (i.e. turn-taking) channel have compatible (dual) signatures, such that one side may send when the other side may receive. Not everything can be modeled with half-duplex communications, but the ideas are pretty useful to keep in mind. I try to keep my software systems as functional as possible (where "functional" here means "no explicit state"). But there are always places where it makes sense to think in terms of state, and so I try to model that state monotonically whenever possible. At least subjectively, it's usually a lot simpler (and easier to follow) than unrestricted state. (Note, of course, that local variables are local in the truest sense: other programmatic agents cannot make assumptions about them or change them. Short-lived, local state is as good as functional non-state in most cases.)
- yowlingcat 6y ago
  > I try to keep my software systems as functional as possible (where "functional" here means "no explicit state"). But there are always places where it makes sense to think in terms of state, and so I try to model that state monotonically whenever possible. At least subjectively, it's usually a lot simpler (and easier to follow) than unrestricted state. Agreed. You mention LVars so I'm curious what you think about MVars and STM in general. I've always been fond of STM because relational databases and their transactions are a familiar and well understood concept historically used by the industry to keep state sane and maintain data integrity. SQLite is great, but having something that's even closer the core language or standard library is even better. It's part of why I like using SQL to do the heavy lifting when possible. I like that SQL is a purely functional language that naturally structures state mutations as transactions through the write-ahead log protocol. My flavor of choice (Postgres) makes different levels of efficient read and write available through read isolation levels that can give me up to ACID consistency without having to reinvent the wheel with my read and write semantics. If I structure my data model keys, relations and constraints properly, I get a production strength implementation with a lot of the nice properties you talk about. And that's regardless of my service layer choice for my language that I can trust to stand up. There's one exception in particular that I've seen begin to gain steam in the industry which I think is interesting, and that's Elixir. Because Elixir wraps around Erlang's venerable OTP (and distributed database mnesia), users can build on the top of something that's already solved a lot of the hard distributed systems problems in the wild in a very challenging use case (telecom switches). Of course, mnesia has its own issues so most of the folks I know using Elixir are using it with Phoenix + SQL. They seem to like it, but I worry about ecosystem collapse risk with any transpiled language -- no one wants to see another CoffeeScript.