3 ms·

> One is smart messaging; dumb clients. The other is dumb messaging; smart clients. All the smartness of the messaging can be implemented in the smart clients.

> One is smart messaging; dumb clients. The other is dumb messaging; smart clients. All the smartness of the messaging can be implemented in the smart clients. Then you can expose that as a smart messaging api to dumb clients. The most obvious example is kafka streams which exposes a "simple" api rather than dealing directly with kafka, but obviously you could create a less featurefull wrapper than that.

debadyutirc 3y ago
That's a neat way to put it! - RabbitMQ: Smart messaging, dumb client - Kafka: Dumb messaging, smart client Have you heard of Fluvio? Fluvio: Smart messaging, Smart Client, Stateful Streaming Kafka + Flink in Rust + WASM Git Repo - https://github.com/infinyon/fluvio https://github.com/infinyon/fluvio
waynesonfire 3y ago
And reimplement rabbitmq? Great idea. Let's do it in rust too.
- deleted 3y ago
  [deleted]
eddythompson80 3y ago
I can't help but think that this just gives you the worst of both worlds. You are now on the hook managing that non-standard "smart" wrapper which will quickly just become the status quo for the project. Anyone wanting to change how it works needs to understand exactly how "smart" you made it and all the side effects that will come with making a change there. I pushed against knative in our company particularly for that reason. Like we wanna use kafka because [Insert kafka sales pitch], but we don't want our developers to utilize any of the kafka features. We're just gonna define the kafka client in some yaml format and have our clients handle an http request per message. It didn't make sense to me.
- robertlagrant 3y ago
  This would be my instinct too.
- supermatt 3y ago
  Thats kind of like saying dont use any software libraries because they all use the standard lib indirectly so you may as well just use that? Its just an abstraction layer to make things less effort.
  - eddythompson80 3y ago
    yeah, don't wrap all calls to a standard lib in another homegrown or non-standard single-digit user lib that makes changes in all sort of subtle ways. There are plenty of C++ projects that make their own or wrap stdlib and they are always a big wtf. It's one thing to have an abstraction for kafka in your code, it's another to wrap the client in a smart client that reimplements something like rabbitmq, and much worse a smart service.
    - supermatt 3y ago
      > don't wrap all calls to a standard lib Im not saying to expose the same primitives - what would be the point in that? I am saying that EVERY lib you use will be using the standard lib or some abstraction of it to perform its own utility. > It's one thing to have an abstraction for kafka in your code, it's another to wrap the client in a smart client, and much worse a smart service. That abstraction is exactly what i am talking about. Why write 50 lines of boilerplate multiple times throughout your code when you can wrap that up in a single function call and expose THAT as your client. You know thats exactly what you will end up doing on any non-trivial project. Or you could use a lib that already does that - such as the "official" kafka streams lib.
  - hamandcheese 3y ago
    > Thats kind of like saying dont use any software libraries because they all use the standard lib indirectly so you may as well just use that? This is decent advice, IMO. The cost of dependency management is often vastly understated.
NovemberWhiskey 3y ago
>All the smartness of the messaging can be implemented in the smart clients. How do you do, for example, a queue with priorities client side without it being insanity? That's a relatively basic AMQP thing. Or managing the number of redeliveries for a message that's being repeatedly rejected. You can absolutely try to build some of this with a look-aside shared data store that all clients have to depend on in order to emulate having the capability in the broker, but you just introduced another common point of failure in addition to the messaging infrastructure. Life is too short for this.
- supermatt 3y ago
  I totally agree that you cant do a lot of AMQP stuff. As you noted, you can build some of it by managing state via transactional producers, etc - but you definitely cant do everything. The biggest gripe for me is actually dynamic "queue" creation, patterns for topics, etc. So I use an MQ for an MQ ;) I'm just saying you can "dumb down" the client side on kafka by creating an abstraction layer (or one of the many higher level libs that already do that).
- robertlagrant 3y ago
  Those requirements would definitely be examples of those that are fulfilled by smart messaging.

debadyutirc 3y ago

That's a neat way to put it! - RabbitMQ: Smart messaging, dumb client - Kafka: Dumb messaging, smart client Have you heard of Fluvio? Fluvio: Smart messaging, Smart Client, Stateful Streaming Kafka + Flink in Rust + WASM Git Repo - https://github.com/infinyon/fluvio https://github.com/infinyon/fluvio

waynesonfire 3y ago

And reimplement rabbitmq? Great idea. Let's do it in rust too.

deleted 3y ago
[deleted]

eddythompson80 3y ago

I can't help but think that this just gives you the worst of both worlds. You are now on the hook managing that non-standard "smart" wrapper which will quickly just become the status quo for the project. Anyone wanting to change how it works needs to understand exactly how "smart" you made it and all the side effects that will come with making a change there. I pushed against knative in our company particularly for that reason. Like we wanna use kafka because [Insert kafka sales pitch], but we don't want our developers to utilize any of the kafka features. We're just gonna define the kafka client in some yaml format and have our clients handle an http request per message. It didn't make sense to me.

robertlagrant 3y ago
This would be my instinct too.
supermatt 3y ago
Thats kind of like saying dont use any software libraries because they all use the standard lib indirectly so you may as well just use that? Its just an abstraction layer to make things less effort.
- eddythompson80 3y ago
  yeah, don't wrap all calls to a standard lib in another homegrown or non-standard single-digit user lib that makes changes in all sort of subtle ways. There are plenty of C++ projects that make their own or wrap stdlib and they are always a big wtf. It's one thing to have an abstraction for kafka in your code, it's another to wrap the client in a smart client that reimplements something like rabbitmq, and much worse a smart service.
  - supermatt 3y ago
    > don't wrap all calls to a standard lib Im not saying to expose the same primitives - what would be the point in that? I am saying that EVERY lib you use will be using the standard lib or some abstraction of it to perform its own utility. > It's one thing to have an abstraction for kafka in your code, it's another to wrap the client in a smart client, and much worse a smart service. That abstraction is exactly what i am talking about. Why write 50 lines of boilerplate multiple times throughout your code when you can wrap that up in a single function call and expose THAT as your client. You know thats exactly what you will end up doing on any non-trivial project. Or you could use a lib that already does that - such as the "official" kafka streams lib.
- hamandcheese 3y ago
  > Thats kind of like saying dont use any software libraries because they all use the standard lib indirectly so you may as well just use that? This is decent advice, IMO. The cost of dependency management is often vastly understated.

NovemberWhiskey 3y ago

>All the smartness of the messaging can be implemented in the smart clients. How do you do, for example, a queue with priorities client side without it being insanity? That's a relatively basic AMQP thing. Or managing the number of redeliveries for a message that's being repeatedly rejected. You can absolutely try to build some of this with a look-aside shared data store that all clients have to depend on in order to emulate having the capability in the broker, but you just introduced another common point of failure in addition to the messaging infrastructure. Life is too short for this.

supermatt 3y ago
I totally agree that you cant do a lot of AMQP stuff. As you noted, you can build some of it by managing state via transactional producers, etc - but you definitely cant do everything. The biggest gripe for me is actually dynamic "queue" creation, patterns for topics, etc. So I use an MQ for an MQ ;) I'm just saying you can "dumb down" the client side on kafka by creating an abstraction layer (or one of the many higher level libs that already do that).
robertlagrant 3y ago
Those requirements would definitely be examples of those that are fulfilled by smart messaging.