4 ms·
It really depends on what you need to do. That's around 1 request per second. I think Kafka might be overkill, but if you plan on needing to increase capacity
by rockostrich 10y ago
It really depends on what you need to do. That's around 1 request per second. I think Kafka might be overkill, but if you plan on needing to increase capacity then it might be worth it. It might just be easier to use RabbitMQ or even Redis at that scale though.
- lekker 10y agoOkay, thank you. I've been looking into Rabbit quite a bit. I like the stream processing aspect of Kafka, where you can introspectively peek into things as they happen rather than a straight message queue. Some of our stuff is on the GCP, so have also been investigating Pub/Sub and DataFlow, which seem more viable since we don't have the up-front cost of hardware and maintenance but pay for resources used. If we don't use much, we should not pay much.
- captn3m0 10y agoI'd also recommend taking a look at nsq.io. Super easy to setup and supports almost everything.
- lekker 10y agoOh, that looks really interesting. I'll definitely give this a bash
- kasey_junk 10y agoOne thing to note about Pub/Sub vs Kafka. Pub/Sub does not offer as strict of guarantees around ordering as Kafka. If you don't need that strictness I'd definitely recommend Pub/Sub over Kafka but if you do I don't believe any of the Google Cloud Offerings will provide you that (Kinesis on AWS will).
- krallistic 10y agoIts worth mentioning that kafka only supports guarantees for ordering inside a partition, not globally. (nifty little detail :) )
- deleted 10y ago[deleted]
- kajecounterhack 10y agoOrdering guarantees are offered by Dataflow.
- lclarkmichalek 10y agowithin a window