3 ms·

Does anyone have recommendations on books/papers/articles which cover gossip protocols? I have been more interested in learning about gossip protocols and how

Does anyone have recommendations on books/papers/articles which cover gossip protocols? I have been more interested in learning about gossip protocols and how they are used, different tradeoffs, etc.

rishabhaiover 1y ago
https://thesecretlivesofdata.com/raft/ https://thesecretlivesofdata.com/raft/
- rdtsc 1y ago
  > https://thesecretlivesofdata.com/raft/ https://thesecretlivesofdata.com/raft/ Are you suggesting to use raft as a gossip protocol? Run a replicated state machine with leader election, replicated logs and stable storage?
- the_duke 1y ago
  Raft is a consensus protocol, which is very different from a gossip protocol.
  - rishabhaiover 1y ago
    I'm sorry, I got confused.
the_duke 1y ago
Two interesting papers: * Epidemic broadcast trees: https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf * HyParView: https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf The iroh-gossip implementation is based on those: https://docs.rs/iroh-gossip/latest/iroh_gossip/ https://docs.rs/iroh-gossip/latest/iroh_gossip/
- __turbobrew__ 1y ago
  Thank you
deleted 1y ago
[deleted]
John23832 1y ago
While not a book/paper/article, this is good implementation practice: https://fly.io/dist-sys/ https://fly.io/dist-sys/
just_mc 1y ago
Check out SWIM and its extensions: https://en.wikipedia.org/wiki/SWIM_Protocol https://en.wikipedia.org/wiki/SWIM_Protocol https://arxiv.org/abs/1707.00788 https://arxiv.org/abs/1707.00788
radiator 1y ago
Small example from CZMQ - High-level C Binding for ZeroMQ: http://czmq.zeromq.org/manual:zgossip http://czmq.zeromq.org/manual:zgossip . Perhaps not self-contained, one might have to read the manual of other resources, such as zactor, zsock or of the application Zyre - Local Area Clustering for Peer-to-Peer Applications https://github.com/zeromq/zyre https://github.com/zeromq/zyre .
lokar 1y ago
I’ve worked with a few, and seen a few built. IME they are never really worth it. They make the system much more complex, fragile and hard to debug. The scale limits they aim to help are just not a big deal anymore. A small group of “coordinator” nodes to do this is much simpler and easy to build and scale.
- __turbobrew__ 1y ago
  > The scale limits they aim to help are just not a big deal anymore. A small group of “coordinator” nodes to do this is much simpler and easy to build and scale. What changed that made it easier?
  - lokar 11mo ago
    Faster hardware (cpu and nic) and better low overhead networking APIs in Linux. One machine can manage really a ton of low volume connections.
cipherself 11mo ago
10 years ago I've implemented SCAMP (a gossip protocol) in Clojure, you might find it interesting, the implementation is quite small https://github.com/cipherself/gossip https://github.com/cipherself/gossip

Does anyone have recommendations on books/papers/articles which cover gossip protocols? I have been more interested in learning about gossip protocols and how they are used, different tradeoffs, etc.

rishabhaiover 1y ago

https://thesecretlivesofdata.com/raft/ https://thesecretlivesofdata.com/raft/

rdtsc 1y ago
> https://thesecretlivesofdata.com/raft/ https://thesecretlivesofdata.com/raft/ Are you suggesting to use raft as a gossip protocol? Run a replicated state machine with leader election, replicated logs and stable storage?
the_duke 1y ago
Raft is a consensus protocol, which is very different from a gossip protocol.
- rishabhaiover 1y ago
  I'm sorry, I got confused.

the_duke 1y ago

Two interesting papers: * Epidemic broadcast trees: https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf * HyParView: https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf The iroh-gossip implementation is based on those: https://docs.rs/iroh-gossip/latest/iroh_gossip/ https://docs.rs/iroh-gossip/latest/iroh_gossip/

__turbobrew__ 1y ago
Thank you

deleted 1y ago

[deleted]

John23832 1y ago

While not a book/paper/article, this is good implementation practice: https://fly.io/dist-sys/ https://fly.io/dist-sys/

just_mc 1y ago

Check out SWIM and its extensions: https://en.wikipedia.org/wiki/SWIM_Protocol https://en.wikipedia.org/wiki/SWIM_Protocol https://arxiv.org/abs/1707.00788 https://arxiv.org/abs/1707.00788

radiator 1y ago

Small example from CZMQ - High-level C Binding for ZeroMQ: http://czmq.zeromq.org/manual:zgossip http://czmq.zeromq.org/manual:zgossip . Perhaps not self-contained, one might have to read the manual of other resources, such as zactor, zsock or of the application Zyre - Local Area Clustering for Peer-to-Peer Applications https://github.com/zeromq/zyre https://github.com/zeromq/zyre .

lokar 1y ago

I’ve worked with a few, and seen a few built. IME they are never really worth it. They make the system much more complex, fragile and hard to debug. The scale limits they aim to help are just not a big deal anymore. A small group of “coordinator” nodes to do this is much simpler and easy to build and scale.

__turbobrew__ 1y ago
> The scale limits they aim to help are just not a big deal anymore. A small group of “coordinator” nodes to do this is much simpler and easy to build and scale. What changed that made it easier?
- lokar 11mo ago
  Faster hardware (cpu and nic) and better low overhead networking APIs in Linux. One machine can manage really a ton of low volume connections.

cipherself 11mo ago

10 years ago I've implemented SCAMP (a gossip protocol) in Clojure, you might find it interesting, the implementation is quite small https://github.com/cipherself/gossip https://github.com/cipherself/gossip