3 ms·

Can't speak for all potential users, but the license is in fact a complete deal-breaker for me and any client I've worked with given the FOSS tools available in

Can't speak for all potential users, but the license is in fact a complete deal-breaker for me and any client I've worked with given the FOSS tools available in the ecosystem. But then, there's also the "Java-only" which is a complete deal-breaker in any client I've worked with doing {micro,}services. Then there's the "what the hell does this actually do" deal-breaker when trying to explain it to some decision makers, and the "we already have queues and K8s to solve all of those issues" deal-breaker when explaining it to most fellow SWEs/SREs.

stolsvik 4y ago
Hahaha, that's rough! :-) I'll tell you one thing: "What the hell does this actually do?!" is extremely spot on! I am close to amazed to how hard it is to explain this library. It really does provide value, but it is evidently exceptionally hard to explain. I first and foremost believe that this is due to the massive prevalence of sync REST/RPC style coding, and that messaging is only pulled up as a solution when you get massive influxes of e.g. inbound reports - where you actually want the queue aspect of a message broker. Not the async-ness. I've tried to lay this out multiple times, e.g. here: https://mats3.io/docs/message-oriented-rpc/ https://mats3.io/docs/message-oriented-rpc/, and in the link for this post itself.
- mixedCase 4y ago
  I think I got it, but I'm inclined to believe I'm missing something. From what I understood, it boils down to "distributed CPS" (continuation-passing style), where the state instead of being automatically closed over it is explicitly assigned to, and the continuation execution is distributed. The problem about this specific approachh from where I see it is that something like this should be a few days' work to implement using an off-the shelf persistent queue (which could be abstracted away), something like Protobuf for defining your shared state and auto-generating serializers/deserializers, and then some reusable glue code to push/pull from the queue, and it would support multiple languages almost out of the box. With that said, I am not sure if I'd want to define pipelines as a shared "mutable" state passed around, it sounds like classic object oriented design which makes it easy to create buggy code. It'd be much more robust to clearly define several steps which very strict, well-defined inputs and outputs every step of the way. You're paying for the copies in each serialization anyway (and much more, since this is a Remote Procedure Call), so what reason could there be for choosing a fixed mutable model?
  - stolsvik 4y ago
    I think you're onto it, but not quite? The "distributed CPS" explanation is quite good. There is no "shared mutable state" per se. The "state object" is passed through the stages of the same endpoint, not shared with stages of other invoked endpoints. It is meant to emulate the method local variables that you "get for free" if you code within a method. If you have String firstname set before you invoke HTTP endpoint X, you of course have that firstname available after the HTTP invocation returns, right? This is so obvious that one really doesn't think about it. When you throw a message onto a queue (and then the processor goes back listening on its incoming queue), you of course loose that String firstname, unless you pass it along (But the next service might not need it, so he then just have to pass it on further, so that the downstream service that actually needs it, can get it). But with Mats' state object, if you assigned the firstname to the state object, it is "magically present" on the next stage too. It is passed within the message, this is what I mean by "messaging with a call stack": The call stack (called MatsTrace) holds the Reply-queues (where an Endpoint's Reply should go to), and the state object which will appear for that replied-to stage. There is no external storage for this. The point of Mats is to leverage normal developers' innate understanding of normal, straight-down, sequential method-invocation-based coding. You can reason exactly like that, while still being able to code fully async messaging. I love the Loom project, and I feel that there is a massive comparison here: Instead of having to embrace async/await, or Promises, or whatnots, Loom instead "hacks" the threading model. The result is exactly the same as if you were using async/await, but with a bit more overhead, you can code linearly, not having to warp your brain around the mess of async/await/Promises. This is exactly the same rationale for me making Mats.

stolsvik 4y ago

Hahaha, that's rough! :-) I'll tell you one thing: "What the hell does this actually do?!" is extremely spot on! I am close to amazed to how hard it is to explain this library. It really does provide value, but it is evidently exceptionally hard to explain. I first and foremost believe that this is due to the massive prevalence of sync REST/RPC style coding, and that messaging is only pulled up as a solution when you get massive influxes of e.g. inbound reports - where you actually want the queue aspect of a message broker. Not the async-ness. I've tried to lay this out multiple times, e.g. here: https://mats3.io/docs/message-oriented-rpc/ https://mats3.io/docs/message-oriented-rpc/, and in the link for this post itself.

mixedCase 4y ago
I think I got it, but I'm inclined to believe I'm missing something. From what I understood, it boils down to "distributed CPS" (continuation-passing style), where the state instead of being automatically closed over it is explicitly assigned to, and the continuation execution is distributed. The problem about this specific approachh from where I see it is that something like this should be a few days' work to implement using an off-the shelf persistent queue (which could be abstracted away), something like Protobuf for defining your shared state and auto-generating serializers/deserializers, and then some reusable glue code to push/pull from the queue, and it would support multiple languages almost out of the box. With that said, I am not sure if I'd want to define pipelines as a shared "mutable" state passed around, it sounds like classic object oriented design which makes it easy to create buggy code. It'd be much more robust to clearly define several steps which very strict, well-defined inputs and outputs every step of the way. You're paying for the copies in each serialization anyway (and much more, since this is a Remote Procedure Call), so what reason could there be for choosing a fixed mutable model?
- stolsvik 4y ago
  I think you're onto it, but not quite? The "distributed CPS" explanation is quite good. There is no "shared mutable state" per se. The "state object" is passed through the stages of the same endpoint, not shared with stages of other invoked endpoints. It is meant to emulate the method local variables that you "get for free" if you code within a method. If you have String firstname set before you invoke HTTP endpoint X, you of course have that firstname available after the HTTP invocation returns, right? This is so obvious that one really doesn't think about it. When you throw a message onto a queue (and then the processor goes back listening on its incoming queue), you of course loose that String firstname, unless you pass it along (But the next service might not need it, so he then just have to pass it on further, so that the downstream service that actually needs it, can get it). But with Mats' state object, if you assigned the firstname to the state object, it is "magically present" on the next stage too. It is passed within the message, this is what I mean by "messaging with a call stack": The call stack (called MatsTrace) holds the Reply-queues (where an Endpoint's Reply should go to), and the state object which will appear for that replied-to stage. There is no external storage for this. The point of Mats is to leverage normal developers' innate understanding of normal, straight-down, sequential method-invocation-based coding. You can reason exactly like that, while still being able to code fully async messaging. I love the Loom project, and I feel that there is a massive comparison here: Instead of having to embrace async/await, or Promises, or whatnots, Loom instead "hacks" the threading model. The result is exactly the same as if you were using async/await, but with a bit more overhead, you can code linearly, not having to warp your brain around the mess of async/await/Promises. This is exactly the same rationale for me making Mats.