4 ms·

Messaging platforms have been an ever present in my career. Tibco, IBM MQ, back to Tibco, RabbitMQ then Kafka. I like how on the surface they can be quite sim

Messaging platforms have been an ever present in my career. Tibco, IBM MQ, back to Tibco, RabbitMQ then Kafka. I like how on the surface they can be quite simple to use, but the optimisation and management of them can be fiendishly subtle. Some of my most interesting projects have been trying to squeeze more messages through a pipe with lower latency, changing the way messages are sent and flow through these platforms, or digging into why one in a billion messages are dropped. There was also an interesting phase of trying to containerise and infra-as-code Kafka. It’s all like plumbing for data infrastructure. An interesting corner of the IT industry.

hardwaresofton 4y ago
Don't forget the NATS-based solutions - NATS Core[0] as an ephemeral message exchange (personally what I would use RabbitMQ for) - NATS Jeststream[1] as a persistent, queue-focused kafka alternative - Liftbridge as an alternate implementation of a persistent kafka alternative[2] Liftbridge has a decent comparison page[3] but unfortunately it's still missing NATS JetStream[4]. I want to see more members of the community use and write about the NATS ecosystem -- I rarely hear complaints. [0]: https://docs.nats.io/nats-concepts/core-nats https://docs.nats.io/nats-concepts/core-nats [1]: https://docs.nats.io/nats-concepts/jetstream https://docs.nats.io/nats-concepts/jetstream [2]: https://liftbridge.io/ https://liftbridge.io/ [3]: https://liftbridge.io/docs/feature-comparison.html https://liftbridge.io/docs/feature-comparison.html [4]: https://github.com/liftbridge-io/liftbridge/issues/104 https://github.com/liftbridge-io/liftbridge/issues/104
- TedDoesntTalk 4y ago
  There are dozens of other solutions you are leaving out. There was an entire industry around “enterprise messaging bus and orchestration“ (or a similar name) in Java land in the early 2000s.
  - hardwaresofton 4y ago
    Right, and with all due respect I'd like to leave those in the past which is why I didn't mention not forgetting them! NATS is a lively project, has never had anything to do with mules or camels, and isn't hard-tied to concepts like ESB. It deserves to be mentioned in 2022. But here are some projects I left out that do deserve to be mentioned I think: - RedPanda[0] - Apache Pulsar[1] [0]: https://redpanda.com/ https://redpanda.com/ [1]: https://pulsar.apache.org/ https://pulsar.apache.org/
EdwardDiego 4y ago
I'm a contributor to Strimzi, a K8s operator to manage Kafka, and I'm a fan of the "define your cluster in YAML".

hardwaresofton 4y ago

Don't forget the NATS-based solutions - NATS Core[0] as an ephemeral message exchange (personally what I would use RabbitMQ for) - NATS Jeststream[1] as a persistent, queue-focused kafka alternative - Liftbridge as an alternate implementation of a persistent kafka alternative[2] Liftbridge has a decent comparison page[3] but unfortunately it's still missing NATS JetStream[4]. I want to see more members of the community use and write about the NATS ecosystem -- I rarely hear complaints. [0]: https://docs.nats.io/nats-concepts/core-nats https://docs.nats.io/nats-concepts/core-nats [1]: https://docs.nats.io/nats-concepts/jetstream https://docs.nats.io/nats-concepts/jetstream [2]: https://liftbridge.io/ https://liftbridge.io/ [3]: https://liftbridge.io/docs/feature-comparison.html https://liftbridge.io/docs/feature-comparison.html [4]: https://github.com/liftbridge-io/liftbridge/issues/104 https://github.com/liftbridge-io/liftbridge/issues/104

TedDoesntTalk 4y ago
There are dozens of other solutions you are leaving out. There was an entire industry around “enterprise messaging bus and orchestration“ (or a similar name) in Java land in the early 2000s.
- hardwaresofton 4y ago
  Right, and with all due respect I'd like to leave those in the past which is why I didn't mention not forgetting them! NATS is a lively project, has never had anything to do with mules or camels, and isn't hard-tied to concepts like ESB. It deserves to be mentioned in 2022. But here are some projects I left out that do deserve to be mentioned I think: - RedPanda[0] - Apache Pulsar[1] [0]: https://redpanda.com/ https://redpanda.com/ [1]: https://pulsar.apache.org/ https://pulsar.apache.org/

EdwardDiego 4y ago

I'm a contributor to Strimzi, a K8s operator to manage Kafka, and I'm a fan of the "define your cluster in YAML".