3 ms·

Is there any sort of 'single node' version of these frameworks? I'm very interested in building event-driven solutions, but I don't need the scale offered by ka

by SmooL 6y ago

Is there any sort of 'single node' version of these frameworks? I'm very interested in building event-driven solutions, but I don't need the scale offered by kakfa/pulsar, and I really don't want all the complexity. Is there any reason nobody has made a smaller, less distributed event-centric DB?

staticassertion 6y ago
Redis?
leourbina 6y ago
Redis Streams is a pretty good lightweight replacement for Kafka. Of course, what Kafka gets you are durability, being able to pick among at-least-once, at-most-once and exactly-once delivery of messages, among other things
- SmooL 6y ago
  Thank you, I'll check it out
Tsarbomb 6y ago
You can run kafka on a single node no problem. Give this a try: https://www.digitalocean.com/community/tutorials/how-to-install-apache-kafka-on-ubuntu-18-04 https://www.digitalocean.com/community/tutorials/how-to-inst...
- SmooL 6y ago
  Oh cool, thank you!
- oweiler 6y ago
  Single node Kafka still needs Zookeeper, though.
gerbal 6y ago
Postgres can be used as an effective event centric database and pub/sub queue with NOTIFY and LISTEN
itestyourcode 6y ago
Pulsar does offer a standalone mode. It has a vertical stack of Pulsar broker, bookkeeper, and zookeeper in one process. The standalone mode also comes in a single docker image. However, the complexity you refer to is essential in a reliable messaging framework. Use of zookeeper or any log consensus algorithm requires multiple nodes ( 3 or more) to achieve durability and high availability goal. It is out of necessity. This is actually essential complexity. There is another messaging framework called Nats.io. It is not persistent so architecturally relatively simpler. You might want to investigate.
lymitshn 6y ago
nats streaming worth checking out, or if you dont care about the streaming part nats/zeromq should be simple enough (simple pub/sub infra tools).
addisonj 6y ago
Pulsar offers a standalone mode, which runs everything inside a single JVM and you can just backup the data files. Pulsar also can be made to run with multiple components, but not distributed.

by SmooL 6y ago

staticassertion 6y ago

Redis?

leourbina 6y ago

Redis Streams is a pretty good lightweight replacement for Kafka. Of course, what Kafka gets you are durability, being able to pick among at-least-once, at-most-once and exactly-once delivery of messages, among other things

SmooL 6y ago
Thank you, I'll check it out

Tsarbomb 6y ago

You can run kafka on a single node no problem. Give this a try: https://www.digitalocean.com/community/tutorials/how-to-install-apache-kafka-on-ubuntu-18-04 https://www.digitalocean.com/community/tutorials/how-to-inst...

SmooL 6y ago
Oh cool, thank you!
oweiler 6y ago
Single node Kafka still needs Zookeeper, though.

gerbal 6y ago

Postgres can be used as an effective event centric database and pub/sub queue with NOTIFY and LISTEN

itestyourcode 6y ago

Pulsar does offer a standalone mode. It has a vertical stack of Pulsar broker, bookkeeper, and zookeeper in one process. The standalone mode also comes in a single docker image. However, the complexity you refer to is essential in a reliable messaging framework. Use of zookeeper or any log consensus algorithm requires multiple nodes ( 3 or more) to achieve durability and high availability goal. It is out of necessity. This is actually essential complexity. There is another messaging framework called Nats.io. It is not persistent so architecturally relatively simpler. You might want to investigate.

lymitshn 6y ago

nats streaming worth checking out, or if you dont care about the streaming part nats/zeromq should be simple enough (simple pub/sub infra tools).

addisonj 6y ago

Pulsar offers a standalone mode, which runs everything inside a single JVM and you can just backup the data files. Pulsar also can be made to run with multiple components, but not distributed.