3 ms·

Having a separate process also allows for decoupling pieces of application logic so that they can be deployed independently of each other. You can build this i

by pravus 3y ago

Having a separate process also allows for decoupling pieces of application logic so that they can be deployed independently of each other. You can build this into a monolith but it's typically hard and error-prone. A pipeline I work on right now ingests large amounts of data from an FTP site and dumps it all onto a Kafka topic which is handled by another process on the consuming end. If this were a monolith it would run faster, but any errors or crashes would either require reprocessing the entire file (not really possible given its size) or keeping state ... which is what we are using Kafka for. We can restart pieces of the pipeline at any time and they just resume where they left off. Having a monolith has no overwhelming advantage to us in this use case.

jeltz 3y ago
That is a sensible split into services but when companies try to split services based on teams then there rarely are any reliability improvements since team difference may not match actual reliability concerns.
asimpletune 3y ago
Wouldn’t writing/reading to/from Kafka sort of be orthogonal to micro services? I say this not to argue one way of another for services, just to say that I think your example is just Kafka, or more generally event driven architecture. One thing that defines services is a request/response messaging pattern. In other words a service makes a request and then they wait for a response, like http. What you’re talking about resembles a service because the producer/consumer are separate processes, but they don’t really have much else in common with services, due to a different messaging pattern.
- pravus 3y ago
  No? Why are micro-services only defined by synchronous/socket communication? Even monolithic threaded apps rely on asynchronous workflows. The fact that a 3rd party is used to relay the request doesn't really matter. My point was that I can build a monolith or micro service around Kafka and they end up being equivalent but the multi process approach works better for my needs. The original question was asking what advantages a separate process would have. I gave one.
  - asimpletune 3y ago
    > Why are micro-services only defined by synchronous/socket communication? In the name is the word "service", which means a process that waits and responds to a client's request. This act of request followed by response is fundamentally synchronous.
    - pravus 3y ago
      That very much depends on your definition of service. OASIS defines a service as "a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description". [0] I see nothing that requires synchronous communication only. Virtually every large micro-service architecture I have ever seen communicates through queues asynchronously. [0] https://en.wikipedia.org/wiki/Service_(systems_architecture) https://en.wikipedia.org/wiki/Service_(systems_architecture)

by pravus 3y ago

jeltz 3y ago

That is a sensible split into services but when companies try to split services based on teams then there rarely are any reliability improvements since team difference may not match actual reliability concerns.

asimpletune 3y ago

Wouldn’t writing/reading to/from Kafka sort of be orthogonal to micro services? I say this not to argue one way of another for services, just to say that I think your example is just Kafka, or more generally event driven architecture. One thing that defines services is a request/response messaging pattern. In other words a service makes a request and then they wait for a response, like http. What you’re talking about resembles a service because the producer/consumer are separate processes, but they don’t really have much else in common with services, due to a different messaging pattern.

pravus 3y ago
No? Why are micro-services only defined by synchronous/socket communication? Even monolithic threaded apps rely on asynchronous workflows. The fact that a 3rd party is used to relay the request doesn't really matter. My point was that I can build a monolith or micro service around Kafka and they end up being equivalent but the multi process approach works better for my needs. The original question was asking what advantages a separate process would have. I gave one.
- asimpletune 3y ago
  > Why are micro-services only defined by synchronous/socket communication? In the name is the word "service", which means a process that waits and responds to a client's request. This act of request followed by response is fundamentally synchronous.
  - pravus 3y ago
    That very much depends on your definition of service. OASIS defines a service as "a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description". [0] I see nothing that requires synchronous communication only. Virtually every large micro-service architecture I have ever seen communicates through queues asynchronously. [0] https://en.wikipedia.org/wiki/Service_(systems_architecture) https://en.wikipedia.org/wiki/Service_(systems_architecture)