3 ms·

I've been watching this with some interest as the successor of NATS streaming. One question I have is how does it fair in potential-high-latency, geographically

I've been watching this with some interest as the successor of NATS streaming. One question I have is how does it fair in potential-high-latency, geographically distributed situations? E.g. what if I had dozens of servers at the edge worldwide wanting to receive events from one another?

tylertreat 6y ago
Good question. It's early days, so there's still a lot of work to be done around geo-replication/geo-aware. It depends on the specific use case, but there are a few approaches you could take. You could run a Liftbridge and NATS cluster in each DC with the NATS cluster connected to a global NATS supercluster. Alternatively, you could just run separate Liftbridge clusters and do replication between them (sort of like the MirrorMaker approach in Kafka, though the tooling here is still lacking for Liftbridge). Longer term, I'd like to take advantage of the recent work in NATS around superclusters and geo-aware subscribers to provide better geo-replication primitives directly inside Liftbridge (https://www.slideshare.net/nats_io/deploy-secure-and-scalable-services-across-kubernetes-clusters-with-nats https://www.slideshare.net/nats_io/deploy-secure-and-scalabl...).
paddybyers 6y ago
We've been experimenting with Liftbridge at a range of scales and workloads, and clusters that are distributed across a single region perform very well. However, it's not really suitable for operating a cluster distributed across multiple regions because the latency adversely impacts the bandwidth of the Raft protocol that's used for cluster-wide coordination; this impacts the rate at which streams can be created, and the rate at which changes to In-Sync Replica sets (ISRs) can be propagated. As suggested, a federation of clusters, each covering a single region, would be required.

3 ms·

I've been watching this with some interest as the successor of NATS streaming. One question I have is how does it fair in potential-high-latency, geographically

by kodablah 6y ago

tylertreat 6y ago
Good question. It's early days, so there's still a lot of work to be done around geo-replication/geo-aware. It depends on the specific use case, but there are a few approaches you could take. You could run a Liftbridge and NATS cluster in each DC with the NATS cluster connected to a global NATS supercluster. Alternatively, you could just run separate Liftbridge clusters and do replication between them (sort of like the MirrorMaker approach in Kafka, though the tooling here is still lacking for Liftbridge). Longer term, I'd like to take advantage of the recent work in NATS around superclusters and geo-aware subscribers to provide better geo-replication primitives directly inside Liftbridge (https://www.slideshare.net/nats_io/deploy-secure-and-scalable-services-across-kubernetes-clusters-with-nats https://www.slideshare.net/nats_io/deploy-secure-and-scalabl...).
paddybyers 6y ago
We've been experimenting with Liftbridge at a range of scales and workloads, and clusters that are distributed across a single region perform very well. However, it's not really suitable for operating a cluster distributed across multiple regions because the latency adversely impacts the bandwidth of the Raft protocol that's used for cluster-wide coordination; this impacts the rate at which streams can be created, and the rate at which changes to In-Sync Replica sets (ISRs) can be propagated. As suggested, a federation of clusters, each covering a single region, would be required.