9 ms·

Reactive Cloud Actors: No-nonsense micro-services

41 points by aliostad 13y ago

bananas 13y ago
My bullshit detector was tripped by the number of buzzwords. What does this actually solve versus say traditional MQ (EIP stuff), service bus systems and normal SOA? I've dealt with NServiceBus extensively and really big scale .Net stuff and if it's anything remotely close to that architecture, I'd rather hang myself than use it again.
- aliostad 13y ago
  Yes, so many buzzwords together - I admit. The difference is replacing Commnad/Query with events. Replacing Imperative with Reactive. Did you read the whole post or you were put off by the buzzwords?
  - bananas 13y ago
    Perhaps it's because it is Sunday but I phased out after the first couple of paragraphs. I've read it now and am still unaware of why this is fundamentally better. Command/Query pattern is an imeplementation concern. The architectural pattern that backs it up isn't necessarily related (for example event sourcing). At that point it isn't imperative so why is this different? It just appears to be fragmenting lots of concerns into even smaller ones which are vastly more complicated to debug and manage. The bold highlighted text of: "in a reactive actor system, each actor only knows about its own Step and what itself does and has no clue about the next steps or the rest of the system" Isn't that the same for a message channel but without a transactional guarantee? (going back a decade or more to EIP).
    - aliostad 13y ago
      No it is not the same. First of all, CQRS is applying CQS to the architecture, so it is an architectural decision. When you use commands, you know about the interface of the endpoint receiving your message. In case of an event, you do not have to know anything - and this is the whole premise of this post. In the article to come, I have explained the differences more with regard to EIP as well. A half-draft available at https://gist.github.com/aliostad/55b652e44dfe87d44444 https://gist.github.com/aliostad/55b652e44dfe87d44444
      - bananas 13y ago
        Thanks. I'm sceptical (having been through several similar systems and ripped them out) but I will read your draft later.
mml 13y ago
He's certainly on to something. As he pointed out, it's been around since about '73, and many people are happy with the design of such systems. Amazon even has an actual for-real implementation of one (see Simple Workflow), he should probably try it out before rolling his own. It really is an elegant way to do things from what I can tell. It forces you to keep your state in one, carefully managed place (hairball), rather than sprinkled around god-knows where, being poked and prodded by god knows what.... IMHO, There should be more talk about the design of actor based systems than the glut of OO yammering we are all exposed to.
th0br0 12y ago
Isn't this what http://akka.io http://akka.io does?
- aliostad 12y ago
  Yes, but they do them differently. BeeHive is much more opinionated about the message (must be event and has an enveloper) and the Basic Data Structures. Also BeeHive avoids stateful actors. State is always persistent in HA queues or Basic Data Structure stores.
  - platz 12y ago
    Any comments about the recent Orleans announcement? There must be differences but also I assume a lot of similarities As you mention there in no state in beehive actors, so that is one difference. http://blogs.msdn.com/b/dotnet/archive/2014/04/02/available-now-preview-of-project-orleans-cloud-services-at-scale.aspx http://blogs.msdn.com/b/dotnet/archive/2014/04/02/available-...
  - th0br0 12y ago
    How exactly do you mean "stateful actor"? Whether an Akka actor has a state is actually up to you!? E.g. you could have your router fire up a new actor to process each request up to a maximum number of n concurrent actors etc...
  - hibikir 12y ago
    I thought Rails had taught us that extremely opinionated infrastructures are doomed to at best be influential, but not widely used. Sometimes actors have to be stateful. Sometimes they should be the interface to large, complex computations that take 20 minutes to run. The market is choke full of techs that solve very narrow cloud problems. What we need is something that isn't so terribly narrow. Microservices is very, very narrow. Even people working for Fred George admit to that.
    - karmajunkie 12y ago
      I'm having a hard time parsing this—Rails is something I would consider to be both highly opinionated and widely used, but its unclear where your statement falls. I'm not crazy about some (most?) of the opinions, but that's a matter of some debate. > Microservices is very, very narrow. Even people working for Fred George admit to that. Its hard to make this statement without first agreeing to a definition of microservices. I'm not convinced there's wide agreement to such a definition yet.
orasis 12y ago
Sorry, I couldn't get past the terrible whitish text on blackish background. Instant headache.
mamcx 12y ago
The big problem with this kind of stuff is how debug them. Also, that despite the intention, all is really coupled: Don't make sense to cancel a order before create it, so the order of the actions are important. The benefit is (only ?) easy of scaling, IMHO. How solve this? I understand that GO keep the ordering of the calls and is not harder than doing imperative...
- aliostad 12y ago
  Well, first of all, debugging while developing an actor is actually easier: you just have to debug an actor in isolation and make sure it does the right thing and sends the right event in response to the incoming event. In terms of debugging issues when in live, you have to rely on tracing and let something such as Logstash handle search, etc. The ordering of the workflow is preserved. As I explained, FraudCheck is only ever done after an order's payment has been authorised. Hope this makes sense.
  - twic 12y ago
    Well, first of all, debugging while developing an actor is actually easier: you just have to debug an actor in isolation and make sure it does the right thing and sends the right event in response to the incoming event. No. No, this is terrifyingly incorrect. Making each component do the right thing in isolation is easy. Most of the difficulty in systems - in developing, testing, and debugging them - arises from what the components do when they're together, because it turns out your ideas about how they need to behave in isolation were mistaken.
    - mamcx 12y ago
      Exactly. Sometimes is necessary to step into the call of another function/actor, see what is is doing, note is necessary to step into more into a deeper function, etc. Or stop when a certain value arrive, and inspect the code. If this could be done with actor/csp, then is good idea to jump onto it.
mercurial 12y ago
Fun. I accidentally used this pattern (minus the timestamp, since it wasn't relevant for my use case) to implement a non-distributed data processing framework on top of POE. It does work pretty well, however, you need to have a way for the actor to block until enough messages have arrived (sometimes buffered operations are the most efficient). I'm not sure how you would do that with the model proposed here.
mantrax4 12y ago
This guy seems uneasy with Actors. His example is wrong and then he tries to solve it with a solution that's worse than the problem he thinks he has. Let's untangle that stuff. Actors are about conversations between folks organized, like you would find in a good company. So let's use the conversation metaphor. His example of "classical" actors: - PaymentActor (to herself): Someone told me to process a payment from this order. I'm done. - PaymentActor: Hey, FraudCheckActor, check if this order I'm giving you is a fraud. - FraudCheckActor (to herself): Damn, this is a fraud. - FraudCheckActor: Hey, CancelOrderActor, cancel that order I'm giving you! - CancelOrderActor (to himself): I canceled the order. Now I see an immediate problem here. This is anarchy. No one is in charge of the whole process, everyone runs to talk to someone else, and passing around that order data and there's no conversation happening here. Would someone in a real company organize things this way? Not in a good company, for sure! Let's see what his solution is: - PaymentActor (to herself): Someone told me to process a payment for this order. - PaymentActor (screaming): Everyone! Payment is complete for this order! (screams the order info to everyone) - FraudCheckActor (to herself): I heard that, I'll check it. Damn, this is a fraud. - FraudCheckActor (screaming): Everyone! This order is a fraud! (screams the order info to everyone) - CancelOrderActor (to himself): I heard that, I'll cancel that order. - CancelOrderActor (to himself): I cancelled it. Now I see an even bigger problem here. It's still a god damn anarchy, but now also everyone is screaming at everyone, including the entire order data, or talking to themselves. How is this better again? Plus notice, there's still state in the system - the subscription setup. The only thing achieved here is that it'll be nearly impossible to reason about how this thing works once it becomes more like a real-world scenario. Here's how you design it with Actors, once you have some experience: - OrderActor: Hey, PaymentActor, process this payment and tell me what happened. - PaymentActor: Hey, OrderActor, I processed it. It's good! - OrderActor: Hey, FraudCheckActor, check this order for fraud and tell me what you found. - FraudCheckActor: Hey, OrderActor, damn! This is a fraud! - OrderActor: Hey, CancelOrderActor, cancel that order and tell me when done. - CancelOrderActor: Hey, OrderActor, I canceled it! Now what do we see here? We see structure. We see clear responsibilities. We see teamwork. No screaming and no chaos. Only OrderActor has the entire order data. He's passing along only the needed bits to each actor according to their responsibilities. And best of all, PaymentActor, FraudCheckActor and CancelOrderActor are not coupled to each other. They just respond to their events, limited to their own responsibilities. Good job OrderActor & co. OrderActor is a supervising actor (that's an actual term). In my example, he's managing the workflow and he is stateful and as you see it's not that scary and out-of-order as the author is telling you. After all, he needs to know the payment went through, and then remember to ask for a fraud check and know what to do after that. (Note: technically the supervisor in this example doesn't have to have state, he can just react to the events by Payment, FraudCheck, and Cancel, but this only works in this very simple example. Real world scenarios never fit into a completely stateless world. We're talking about the order state here!) The other actors have no state (that is seen in this example). So proper design isolates concerns, and may have stateless actors, but stateful actors are essential to the model. Of course, the linked article demonstrates how you can shoot yourself in the foot even with the best model out there. I find it ironic that the author was unhappy with over-complications and coupling, yet coupled everything in a complicated way with his "subscriptions" model, instead of simply using a supervisor.
zenbowman 12y ago
Another case of reinventing the wheel and calling it something new. This is just a reimplementation of what used to be called the "blackboard architecture". http://en.wikipedia.org/wiki/Blackboard_system http://en.wikipedia.org/wiki/Blackboard_system I don't see how broadcast is more "reactive" than singlecast. And the solution to tight coupling in actor systems has already been thought of - it is a message broker. This is just an actor system where every actor shares a single message broker. Its cool that you built your own system for actors, and I certainly wouldn't discourage people from reimplementing something for the sake of learning, but every criticism you have of traditional actor systems is inaccurate. Strangely enough, you criticize Orleans in particular, which is especially good about tackling the coupling problem by making actor references virtual. "Perhaps the central problem we face in all of computer science is how we are to get to the situation where we build on top of the work of others rather than redoing so much of it in a trivially different way." -Richard Hamming

41 points by aliostad 13y ago

My bullshit detector was tripped by the number of buzzwords. What does this actually solve versus say traditional MQ (EIP stuff), service bus systems and normal SOA? I've dealt with NServiceBus extensively and really big scale .Net stuff and if it's anything remotely close to that architecture, I'd rather hang myself than use it again.

aliostad 13y ago
Yes, so many buzzwords together - I admit. The difference is replacing Commnad/Query with events. Replacing Imperative with Reactive. Did you read the whole post or you were put off by the buzzwords?
- bananas 13y ago
  Perhaps it's because it is Sunday but I phased out after the first couple of paragraphs. I've read it now and am still unaware of why this is fundamentally better. Command/Query pattern is an imeplementation concern. The architectural pattern that backs it up isn't necessarily related (for example event sourcing). At that point it isn't imperative so why is this different? It just appears to be fragmenting lots of concerns into even smaller ones which are vastly more complicated to debug and manage. The bold highlighted text of: "in a reactive actor system, each actor only knows about its own Step and what itself does and has no clue about the next steps or the rest of the system" Isn't that the same for a message channel but without a transactional guarantee? (going back a decade or more to EIP).
  - aliostad 13y ago
    No it is not the same. First of all, CQRS is applying CQS to the architecture, so it is an architectural decision. When you use commands, you know about the interface of the endpoint receiving your message. In case of an event, you do not have to know anything - and this is the whole premise of this post. In the article to come, I have explained the differences more with regard to EIP as well. A half-draft available at https://gist.github.com/aliostad/55b652e44dfe87d44444 https://gist.github.com/aliostad/55b652e44dfe87d44444
    - bananas 13y ago
      Thanks. I'm sceptical (having been through several similar systems and ripped them out) but I will read your draft later.

mml 13y ago

He's certainly on to something. As he pointed out, it's been around since about '73, and many people are happy with the design of such systems. Amazon even has an actual for-real implementation of one (see Simple Workflow), he should probably try it out before rolling his own. It really is an elegant way to do things from what I can tell. It forces you to keep your state in one, carefully managed place (hairball), rather than sprinkled around god-knows where, being poked and prodded by god knows what.... IMHO, There should be more talk about the design of actor based systems than the glut of OO yammering we are all exposed to.

th0br0 12y ago

Isn't this what http://akka.io http://akka.io does?

aliostad 12y ago
Yes, but they do them differently. BeeHive is much more opinionated about the message (must be event and has an enveloper) and the Basic Data Structures. Also BeeHive avoids stateful actors. State is always persistent in HA queues or Basic Data Structure stores.
- platz 12y ago
  Any comments about the recent Orleans announcement? There must be differences but also I assume a lot of similarities As you mention there in no state in beehive actors, so that is one difference. http://blogs.msdn.com/b/dotnet/archive/2014/04/02/available-now-preview-of-project-orleans-cloud-services-at-scale.aspx http://blogs.msdn.com/b/dotnet/archive/2014/04/02/available-...
- th0br0 12y ago
  How exactly do you mean "stateful actor"? Whether an Akka actor has a state is actually up to you!? E.g. you could have your router fire up a new actor to process each request up to a maximum number of n concurrent actors etc...
- hibikir 12y ago
  I thought Rails had taught us that extremely opinionated infrastructures are doomed to at best be influential, but not widely used. Sometimes actors have to be stateful. Sometimes they should be the interface to large, complex computations that take 20 minutes to run. The market is choke full of techs that solve very narrow cloud problems. What we need is something that isn't so terribly narrow. Microservices is very, very narrow. Even people working for Fred George admit to that.
  - karmajunkie 12y ago
    I'm having a hard time parsing this—Rails is something I would consider to be both highly opinionated and widely used, but its unclear where your statement falls. I'm not crazy about some (most?) of the opinions, but that's a matter of some debate. > Microservices is very, very narrow. Even people working for Fred George admit to that. Its hard to make this statement without first agreeing to a definition of microservices. I'm not convinced there's wide agreement to such a definition yet.

orasis 12y ago

Sorry, I couldn't get past the terrible whitish text on blackish background. Instant headache.

mamcx 12y ago

The big problem with this kind of stuff is how debug them. Also, that despite the intention, all is really coupled: Don't make sense to cancel a order before create it, so the order of the actions are important. The benefit is (only ?) easy of scaling, IMHO. How solve this? I understand that GO keep the ordering of the calls and is not harder than doing imperative...

aliostad 12y ago
Well, first of all, debugging while developing an actor is actually easier: you just have to debug an actor in isolation and make sure it does the right thing and sends the right event in response to the incoming event. In terms of debugging issues when in live, you have to rely on tracing and let something such as Logstash handle search, etc. The ordering of the workflow is preserved. As I explained, FraudCheck is only ever done after an order's payment has been authorised. Hope this makes sense.
- twic 12y ago
  Well, first of all, debugging while developing an actor is actually easier: you just have to debug an actor in isolation and make sure it does the right thing and sends the right event in response to the incoming event. No. No, this is terrifyingly incorrect. Making each component do the right thing in isolation is easy. Most of the difficulty in systems - in developing, testing, and debugging them - arises from what the components do when they're together, because it turns out your ideas about how they need to behave in isolation were mistaken.
  - mamcx 12y ago
    Exactly. Sometimes is necessary to step into the call of another function/actor, see what is is doing, note is necessary to step into more into a deeper function, etc. Or stop when a certain value arrive, and inspect the code. If this could be done with actor/csp, then is good idea to jump onto it.

mercurial 12y ago

Fun. I accidentally used this pattern (minus the timestamp, since it wasn't relevant for my use case) to implement a non-distributed data processing framework on top of POE. It does work pretty well, however, you need to have a way for the actor to block until enough messages have arrived (sometimes buffered operations are the most efficient). I'm not sure how you would do that with the model proposed here.

mantrax4 12y ago

This guy seems uneasy with Actors. His example is wrong and then he tries to solve it with a solution that's worse than the problem he thinks he has. Let's untangle that stuff. Actors are about conversations between folks organized, like you would find in a good company. So let's use the conversation metaphor. His example of "classical" actors: - PaymentActor (to herself): Someone told me to process a payment from this order. I'm done. - PaymentActor: Hey, FraudCheckActor, check if this order I'm giving you is a fraud. - FraudCheckActor (to herself): Damn, this is a fraud. - FraudCheckActor: Hey, CancelOrderActor, cancel that order I'm giving you! - CancelOrderActor (to himself): I canceled the order. Now I see an immediate problem here. This is anarchy. No one is in charge of the whole process, everyone runs to talk to someone else, and passing around that order data and there's no conversation happening here. Would someone in a real company organize things this way? Not in a good company, for sure! Let's see what his solution is: - PaymentActor (to herself): Someone told me to process a payment for this order. - PaymentActor (screaming): Everyone! Payment is complete for this order! (screams the order info to everyone) - FraudCheckActor (to herself): I heard that, I'll check it. Damn, this is a fraud. - FraudCheckActor (screaming): Everyone! This order is a fraud! (screams the order info to everyone) - CancelOrderActor (to himself): I heard that, I'll cancel that order. - CancelOrderActor (to himself): I cancelled it. Now I see an even bigger problem here. It's still a god damn anarchy, but now also everyone is screaming at everyone, including the entire order data, or talking to themselves. How is this better again? Plus notice, there's still state in the system - the subscription setup. The only thing achieved here is that it'll be nearly impossible to reason about how this thing works once it becomes more like a real-world scenario. Here's how you design it with Actors, once you have some experience: - OrderActor: Hey, PaymentActor, process this payment and tell me what happened. - PaymentActor: Hey, OrderActor, I processed it. It's good! - OrderActor: Hey, FraudCheckActor, check this order for fraud and tell me what you found. - FraudCheckActor: Hey, OrderActor, damn! This is a fraud! - OrderActor: Hey, CancelOrderActor, cancel that order and tell me when done. - CancelOrderActor: Hey, OrderActor, I canceled it! Now what do we see here? We see structure. We see clear responsibilities. We see teamwork. No screaming and no chaos. Only OrderActor has the entire order data. He's passing along only the needed bits to each actor according to their responsibilities. And best of all, PaymentActor, FraudCheckActor and CancelOrderActor are not coupled to each other. They just respond to their events, limited to their own responsibilities. Good job OrderActor & co. OrderActor is a supervising actor (that's an actual term). In my example, he's managing the workflow and he is stateful and as you see it's not that scary and out-of-order as the author is telling you. After all, he needs to know the payment went through, and then remember to ask for a fraud check and know what to do after that. (Note: technically the supervisor in this example doesn't have to have state, he can just react to the events by Payment, FraudCheck, and Cancel, but this only works in this very simple example. Real world scenarios never fit into a completely stateless world. We're talking about the order state here!) The other actors have no state (that is seen in this example). So proper design isolates concerns, and may have stateless actors, but stateful actors are essential to the model. Of course, the linked article demonstrates how you can shoot yourself in the foot even with the best model out there. I find it ironic that the author was unhappy with over-complications and coupling, yet coupled everything in a complicated way with his "subscriptions" model, instead of simply using a supervisor.

zenbowman 12y ago

Another case of reinventing the wheel and calling it something new. This is just a reimplementation of what used to be called the "blackboard architecture". http://en.wikipedia.org/wiki/Blackboard_system http://en.wikipedia.org/wiki/Blackboard_system I don't see how broadcast is more "reactive" than singlecast. And the solution to tight coupling in actor systems has already been thought of - it is a message broker. This is just an actor system where every actor shares a single message broker. Its cool that you built your own system for actors, and I certainly wouldn't discourage people from reimplementing something for the sake of learning, but every criticism you have of traditional actor systems is inaccurate. Strangely enough, you criticize Orleans in particular, which is especially good about tackling the coupling problem by making actor references virtual. "Perhaps the central problem we face in all of computer science is how we are to get to the situation where we build on top of the work of others rather than redoing so much of it in a trivially different way." -Richard Hamming