4 ms·

Doug McIlroy, the inventor of UNIX pipes is quoted as "Write programs that do one thing and do it well. Write programs to work together. Write programs to handl

Doug McIlroy, the inventor of UNIX pipes is quoted as "Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.". I know it's probably not exactly what you're looking for but "Do one thing and do it well" tools and UNIX pipes to connect such single-purpose tools in my opinion is quite close to a "pluggable building blocks" metaphor. In terms of web applications the REST paradigm and data represented as JSON structures is a similar smallest common denominator. Someone else has already mentioned Christopher Alexander's pattern language approach. Design patterns and Atomic Design are similar concepts for the software development domain.

mlthoughts2018 8y ago
This is the real value of microservices, in my mind. It allows you to create services that "do one thing and do it well," including deployment considerations like the degree of autoscaling or load balancing required by some components and not others. Totally unrelated, it is also why I think "full stack developer" jobs are just business junk and should be totally avoided. Business people wishing for "full stack" employees are like a giant red flag of a business full of dysfunctional complexity mismanagement, without a clear vision of what needs to be solved. Inside of a software system, if we organized one monolith class or one monolith webservice that was "full stack" in the sense that it had responsibilities cutting across lots of disparate systems or layers of the stack, we would rightly think this is a bad software design, rife with unnecessary complexity from trying to make certain components do too many things. And probably it would be reflected in how hard it is to refactor, extend, or write clean tests for such a system. I argue that popping up one layer outside of the software system and into the realm of the organization of engineers who design and implement it, we should still think the same way. Different employees should be isolated specialists in certain parts of the system, and should not be expected to have cross-cutting "full stack" job duties, which is really just a way for companies to artificially save money on headcount (artificially, because the lack of clean separation of concerns ends up costing them more money in the end). This is not a statement that a given engineer should not learn about broad aspects of system engineering. It's always good to diversify your knowledge, have a better picture of the meaning of the work of other teams, or be able to fill in during an emergency or something. Only a statement that an employee's nominal job duties should not include vague bottomless complexity buckets reflected by "cross functional" or "full stack" responsibilities. This dysfunction is also often described with the "wear many hats" bullshit, where the same person who is expected to design a performant machine learning backend system is also expected to create a responsive single page website and write any CRUD implementation, ecommerce shop logic, auth logic, etc. etc., all while also functioning as a devops engineer and a data pipelines engineer because "wear many hats." It's the same mistake that Single Responsibility Principle is meant to guard against, just at the level of programming teams of people instead of programming components of a software system.
- BjoernKW 8y ago
  From my point of view, the main benefit of microservices is risk reduction. Microservices allow you to iterate rapidly and roll out new features without potentially affecting your application as a whole. "Do one thing and do it well." can also be an argument for using microservices. However, your application has to be of a certain size in order for that to be a valid point. If your application merely has a few endpoints, most of which are rarely - if ever - used as each other's inputs, then using microservices only adds development overhead without yielding considerable benefit. This is also why I'm critical of the surprisingly common one-size-fits-all "Use microservices for everything" approach. Microservices are just another design pattern. They're not a cure-all for every software development problem. I, for one, am doubtful about the front-end / back-end developer dichotomy because it tends to create new silos. I don't particularly like "full-stack developer" either because I think engineers should aim to be problem solvers rather than people who merely turn someone else's solution into code. Solving a particular problem in some cases might just involve the back-end or the front-end but in most cases it not only involves both but other areas such as requirements engineering or design as well.
  - mlthoughts2018 8y ago
    > “From my point of view, the main benefit of microservices is risk reduction. Microservices allow you to iterate rapidly and roll out new features without potentially affecting your application as a whole.” I think this is the same reason why “do one thing and do it well” is useful in software components or shell utilities too. Basically, what you said about microservices is just a restatement of Single Responsibility Principle / decoupling, in general, and it’s not unique to microservices. In this sense, I’d say microservices are not a design pattern. Rather, it is an organizational strategy for minimizing the risks of coupling in other design patterns, which is also why microservices are explicitly a good idea in cases when a service might only have a few independent endpoints. I don’t buy the argument that it adds overhead. In fact I’d say it explicitly reduces overhead, because the boilerplate of wrapping it in a web service is just too tiny to care about compared with the overhead required to manage the complexity when a system tries to be responsible for too many things. In other words, I think good hygiene would say defaulting to microservices is the right idea, and consolidating should require evidence that it provides the expected benefits. Personally I am reminded of the funny Andrew Gelman quote, “Just because something is counter-intuitive doesn’t make it true.” I think it’s fashionable to criticize microservice designs and people jump on the bandwagon of pretending like criticizing microservice designs is somehow more well-rounded or more nuanced, but really it’s not. It’s just an attempt to make something fashionable via contrarianism. > “I think engineers should aim to be problem solvers” I agree with this very much and it is why I think cross-silo / cross-functional engineering is junk. The best way to organize a team of problem solvers is to clearly delineate the boundaries between what the problem solving responsibilities are. If people use the term “problem solver” to just be an ambiguous bucket of unlimited complexity by virtue of expecting people to “just solve the problem” no matter what the impedance mismatch between their skill area and the problem area, then it renders the term “problem solver” meaningless, and will just lead to a big mess of dysfunction. As an engineering manager, one of my most important jobs is to empower engineers to say “no.” To reject inappropriately allocated work or inappropriate cross-functional pivoting driven by ineffective short term business thinking. The engineers have to be empowered to say, “Hey, that work should not be assigned to us” because it signals whether the business overall has structured itself and staffed itself appropriately to actually respond to the business problems sent by Mother Nature at us. If we are structured wrongly, yet try to paper over it by pressuring engineers to be HR-approved “problem solvers” in the sense of more just doing what they are told instead of doing what would actually work, it will be medium-term detrimental and counter-productive. In this sense, I think “silo” is a good word, and indicates a good engineering culture where different silos have the power to say “no” and where specializations are respected and planned for by the management. Yes, of course that can be subverted into an anti-pattern in poorly run businesses, but that’s a property of bad management, not of “silos” in and of themselves.

mlthoughts2018 8y ago

This is the real value of microservices, in my mind. It allows you to create services that "do one thing and do it well," including deployment considerations like the degree of autoscaling or load balancing required by some components and not others. Totally unrelated, it is also why I think "full stack developer" jobs are just business junk and should be totally avoided. Business people wishing for "full stack" employees are like a giant red flag of a business full of dysfunctional complexity mismanagement, without a clear vision of what needs to be solved. Inside of a software system, if we organized one monolith class or one monolith webservice that was "full stack" in the sense that it had responsibilities cutting across lots of disparate systems or layers of the stack, we would rightly think this is a bad software design, rife with unnecessary complexity from trying to make certain components do too many things. And probably it would be reflected in how hard it is to refactor, extend, or write clean tests for such a system. I argue that popping up one layer outside of the software system and into the realm of the organization of engineers who design and implement it, we should still think the same way. Different employees should be isolated specialists in certain parts of the system, and should not be expected to have cross-cutting "full stack" job duties, which is really just a way for companies to artificially save money on headcount (artificially, because the lack of clean separation of concerns ends up costing them more money in the end). This is not a statement that a given engineer should not learn about broad aspects of system engineering. It's always good to diversify your knowledge, have a better picture of the meaning of the work of other teams, or be able to fill in during an emergency or something. Only a statement that an employee's nominal job duties should not include vague bottomless complexity buckets reflected by "cross functional" or "full stack" responsibilities. This dysfunction is also often described with the "wear many hats" bullshit, where the same person who is expected to design a performant machine learning backend system is also expected to create a responsive single page website and write any CRUD implementation, ecommerce shop logic, auth logic, etc. etc., all while also functioning as a devops engineer and a data pipelines engineer because "wear many hats." It's the same mistake that Single Responsibility Principle is meant to guard against, just at the level of programming teams of people instead of programming components of a software system.

BjoernKW 8y ago
From my point of view, the main benefit of microservices is risk reduction. Microservices allow you to iterate rapidly and roll out new features without potentially affecting your application as a whole. "Do one thing and do it well." can also be an argument for using microservices. However, your application has to be of a certain size in order for that to be a valid point. If your application merely has a few endpoints, most of which are rarely - if ever - used as each other's inputs, then using microservices only adds development overhead without yielding considerable benefit. This is also why I'm critical of the surprisingly common one-size-fits-all "Use microservices for everything" approach. Microservices are just another design pattern. They're not a cure-all for every software development problem. I, for one, am doubtful about the front-end / back-end developer dichotomy because it tends to create new silos. I don't particularly like "full-stack developer" either because I think engineers should aim to be problem solvers rather than people who merely turn someone else's solution into code. Solving a particular problem in some cases might just involve the back-end or the front-end but in most cases it not only involves both but other areas such as requirements engineering or design as well.
- mlthoughts2018 8y ago
  > “From my point of view, the main benefit of microservices is risk reduction. Microservices allow you to iterate rapidly and roll out new features without potentially affecting your application as a whole.” I think this is the same reason why “do one thing and do it well” is useful in software components or shell utilities too. Basically, what you said about microservices is just a restatement of Single Responsibility Principle / decoupling, in general, and it’s not unique to microservices. In this sense, I’d say microservices are not a design pattern. Rather, it is an organizational strategy for minimizing the risks of coupling in other design patterns, which is also why microservices are explicitly a good idea in cases when a service might only have a few independent endpoints. I don’t buy the argument that it adds overhead. In fact I’d say it explicitly reduces overhead, because the boilerplate of wrapping it in a web service is just too tiny to care about compared with the overhead required to manage the complexity when a system tries to be responsible for too many things. In other words, I think good hygiene would say defaulting to microservices is the right idea, and consolidating should require evidence that it provides the expected benefits. Personally I am reminded of the funny Andrew Gelman quote, “Just because something is counter-intuitive doesn’t make it true.” I think it’s fashionable to criticize microservice designs and people jump on the bandwagon of pretending like criticizing microservice designs is somehow more well-rounded or more nuanced, but really it’s not. It’s just an attempt to make something fashionable via contrarianism. > “I think engineers should aim to be problem solvers” I agree with this very much and it is why I think cross-silo / cross-functional engineering is junk. The best way to organize a team of problem solvers is to clearly delineate the boundaries between what the problem solving responsibilities are. If people use the term “problem solver” to just be an ambiguous bucket of unlimited complexity by virtue of expecting people to “just solve the problem” no matter what the impedance mismatch between their skill area and the problem area, then it renders the term “problem solver” meaningless, and will just lead to a big mess of dysfunction. As an engineering manager, one of my most important jobs is to empower engineers to say “no.” To reject inappropriately allocated work or inappropriate cross-functional pivoting driven by ineffective short term business thinking. The engineers have to be empowered to say, “Hey, that work should not be assigned to us” because it signals whether the business overall has structured itself and staffed itself appropriately to actually respond to the business problems sent by Mother Nature at us. If we are structured wrongly, yet try to paper over it by pressuring engineers to be HR-approved “problem solvers” in the sense of more just doing what they are told instead of doing what would actually work, it will be medium-term detrimental and counter-productive. In this sense, I think “silo” is a good word, and indicates a good engineering culture where different silos have the power to say “no” and where specializations are respected and planned for by the management. Yes, of course that can be subverted into an anti-pattern in poorly run businesses, but that’s a property of bad management, not of “silos” in and of themselves.