2 ms·
If you are on GCP I think the choice is simple, use Cloud Pub/Sub. Extremely simple, extremely reliable, extremely performant, fairly inexpensive, multi-region
by antoncohen 6y ago
If you are on GCP I think the choice is simple, use Cloud Pub/Sub. Extremely simple, extremely reliable, extremely performant, fairly inexpensive, multi-region (global). No maintenance, no scaling, almost no tunables, it just works.
Google provides a Pub/Sub emulator for local development.
I don't really buy the vendor lock-in thing for Pub/Sub-like systems. The Cloud Pub/Sub usage pattern is basically the same as Kafka, you can have a library that abstracts away the differences. There are open source libraries that do that[1]. If you ever need to switch cloud providers, or want a messaging system to span cloud providers, you can switch without changing lots of code.
[1] https://github.com/google/go-cloud/tree/master/pubsub https://github.com/google/go-cloud/tree/master/pubsub
- peterhunt 6y agoI don't think it's that simple unless I misunderstand how GCP Pubsub works. I don't think GCP PubSub will give you deterministic delivery order within a partition the way Kafka will: https://cloud.google.com/pubsub/docs/ordering https://cloud.google.com/pubsub/docs/ordering
- batter 6y agoWe have Kafka and GCP pubsub. Kafka is the way to go for us. In terms of reliability, performance, load, etc.
- skyde 6y agoAre you saying GCP pubsub have bad reliability? I never used it but would be surprised if it was down for several hours each weak or would just drop your data. I would love to know your experience
- batter 6y agoOne of the teams was complaining about behavior on load. Now another team will migrate that flow to Kafka. Another team was complaining about reliability/speed. I think eventually they stopped relying on that.