3 ms·

I'm also generally micro-service skeptical, but I'll do my best to give what I believe an answer is. The industry had a huge micro service phase for like the la

I'm also generally micro-service skeptical, but I'll do my best to give what I believe an answer is. The industry had a huge micro service phase for like the last decade, and I think in the end most of the same arguments given for micro services could be answered with just modules. Someone else can correct me where I'm wrong, but I think maybe the one argument for a separate service vs module is security, where you want literal process isolation for data. I guess another benefit is micro services make it easier for teams to work in whatever language/tool they're comfortable with. I think that's it really. Better defined ownership could be done with modules. Testability could be done with modules. Most of the usual arguments could just be answered basically with well defined code, and when the interfaces are all async then it even means even those modules could have micro services as their underlying implementation.

asimpletune 3y ago
As a side note, I think even a lot of the data/process isolation stuff could be better mitigated by capabilities-based technology. It's kind of crazy to me that stuff isn't used more.
DrScientist 3y ago
There are two aspects here I think. Modularity in terms of code, and modularity in terms of deployment lifecycle. As you say most dependency problems can be handled at the level of the code, depending on how you structure the builds etc. ( personally I think automatically pulling in dependencies is a mistake - it should be slightly painful to add a new dependency ). For me, the reason for microservice like deployment is having a separately lifecycle - being able to start,stop, update separately from other services. Another aspect which I don't think is talked about enough, is that every deployment has a copy of the code ( in whatever executable form ) - this means I can deploy version 1 of code A in service X and version 2 in service Y at the same time if I need to - ie I can incrementally update the system. So one way to think about what should be deployed together ( or not ) is thinking not about the code per se - but also the natural deployment cycle.
movpasd 3y ago
It's not just about code -- microservices can offer a degree of modularity on an infrastructure level. For example, a microservice can own its own database. Being able to experiment with languages and tools (as you mention) is also a pretty big advantage. HTTP offers a universal communications interface beyond language-level APIs and ABIs. When we think about an entire department with many teams with a diversity of expertise, at least we can guarantee everyone understands HTTP.
- deleted 3y ago
  [deleted]
pravus 3y ago
Having a separate process also allows for decoupling pieces of application logic so that they can be deployed independently of each other. You can build this into a monolith but it's typically hard and error-prone. A pipeline I work on right now ingests large amounts of data from an FTP site and dumps it all onto a Kafka topic which is handled by another process on the consuming end. If this were a monolith it would run faster, but any errors or crashes would either require reprocessing the entire file (not really possible given its size) or keeping state ... which is what we are using Kafka for. We can restart pieces of the pipeline at any time and they just resume where they left off. Having a monolith has no overwhelming advantage to us in this use case.
- jeltz 3y ago
  That is a sensible split into services but when companies try to split services based on teams then there rarely are any reliability improvements since team difference may not match actual reliability concerns.
- asimpletune 3y ago
  Wouldn’t writing/reading to/from Kafka sort of be orthogonal to micro services? I say this not to argue one way of another for services, just to say that I think your example is just Kafka, or more generally event driven architecture. One thing that defines services is a request/response messaging pattern. In other words a service makes a request and then they wait for a response, like http. What you’re talking about resembles a service because the producer/consumer are separate processes, but they don’t really have much else in common with services, due to a different messaging pattern.
  - pravus 3y ago
    No? Why are micro-services only defined by synchronous/socket communication? Even monolithic threaded apps rely on asynchronous workflows. The fact that a 3rd party is used to relay the request doesn't really matter. My point was that I can build a monolith or micro service around Kafka and they end up being equivalent but the multi process approach works better for my needs. The original question was asking what advantages a separate process would have. I gave one.
asimpletune 3y ago
In response to all the people who are talking about the lifecycle of services, or independent deployments, wouldn’t something like small talk essentially do this? It’s super old tech but that doesn’t mean that what’s new is automatically better. And in this case something like small talk sort of seems like it’s from the future when compared to the hodgepodge of tools that are sort of all duck taped together that form the basis of modern infrastructure. I think in a way our ideas as modern developers have been held back by the present day status quo. In a large way that’s in part because it’s many of our jobs to build these systems. Sorry, I know it’s a bit of a rant, and I don’t mean any offense at all. I’ve worked on platform before, so maybe I’m projecting, but I really do think it’s good or even necessary to question the present day wisdom.

asimpletune 3y ago

As a side note, I think even a lot of the data/process isolation stuff could be better mitigated by capabilities-based technology. It's kind of crazy to me that stuff isn't used more.

DrScientist 3y ago

There are two aspects here I think. Modularity in terms of code, and modularity in terms of deployment lifecycle. As you say most dependency problems can be handled at the level of the code, depending on how you structure the builds etc. ( personally I think automatically pulling in dependencies is a mistake - it should be slightly painful to add a new dependency ). For me, the reason for microservice like deployment is having a separately lifecycle - being able to start,stop, update separately from other services. Another aspect which I don't think is talked about enough, is that every deployment has a copy of the code ( in whatever executable form ) - this means I can deploy version 1 of code A in service X and version 2 in service Y at the same time if I need to - ie I can incrementally update the system. So one way to think about what should be deployed together ( or not ) is thinking not about the code per se - but also the natural deployment cycle.

movpasd 3y ago

It's not just about code -- microservices can offer a degree of modularity on an infrastructure level. For example, a microservice can own its own database. Being able to experiment with languages and tools (as you mention) is also a pretty big advantage. HTTP offers a universal communications interface beyond language-level APIs and ABIs. When we think about an entire department with many teams with a diversity of expertise, at least we can guarantee everyone understands HTTP.

deleted 3y ago
[deleted]

pravus 3y ago

Having a separate process also allows for decoupling pieces of application logic so that they can be deployed independently of each other. You can build this into a monolith but it's typically hard and error-prone. A pipeline I work on right now ingests large amounts of data from an FTP site and dumps it all onto a Kafka topic which is handled by another process on the consuming end. If this were a monolith it would run faster, but any errors or crashes would either require reprocessing the entire file (not really possible given its size) or keeping state ... which is what we are using Kafka for. We can restart pieces of the pipeline at any time and they just resume where they left off. Having a monolith has no overwhelming advantage to us in this use case.

jeltz 3y ago
That is a sensible split into services but when companies try to split services based on teams then there rarely are any reliability improvements since team difference may not match actual reliability concerns.
asimpletune 3y ago
Wouldn’t writing/reading to/from Kafka sort of be orthogonal to micro services? I say this not to argue one way of another for services, just to say that I think your example is just Kafka, or more generally event driven architecture. One thing that defines services is a request/response messaging pattern. In other words a service makes a request and then they wait for a response, like http. What you’re talking about resembles a service because the producer/consumer are separate processes, but they don’t really have much else in common with services, due to a different messaging pattern.
- pravus 3y ago
  No? Why are micro-services only defined by synchronous/socket communication? Even monolithic threaded apps rely on asynchronous workflows. The fact that a 3rd party is used to relay the request doesn't really matter. My point was that I can build a monolith or micro service around Kafka and they end up being equivalent but the multi process approach works better for my needs. The original question was asking what advantages a separate process would have. I gave one.

asimpletune 3y ago

In response to all the people who are talking about the lifecycle of services, or independent deployments, wouldn’t something like small talk essentially do this? It’s super old tech but that doesn’t mean that what’s new is automatically better. And in this case something like small talk sort of seems like it’s from the future when compared to the hodgepodge of tools that are sort of all duck taped together that form the basis of modern infrastructure. I think in a way our ideas as modern developers have been held back by the present day status quo. In a large way that’s in part because it’s many of our jobs to build these systems. Sorry, I know it’s a bit of a rant, and I don’t mean any offense at all. I’ve worked on platform before, so maybe I’m projecting, but I really do think it’s good or even necessary to question the present day wisdom.