5 ms·

Show HN: A reference use of Hashicorp's Raft implementation

60 points by otoolep 10y ago

scrollaway 10y ago
Nice work. What prompted you to write this? Are you using it anywhere?
- jpgvm 10y ago
  Not OP but I am pretty sure OP has worked with/on InfluxDB which used it as part of the clustering component (which I am not sure if exists in the open source release anymore). The hashicorp/raft library actually seems to have a bunch of production users, namely Consul ofcourse but also Flynn (used in service discovery and metadata store called discoverd) and I would assume still InfluxDB. It's currently seeing a resurgence in development as the author of the Raft paper has begun contributing to the library also. Disclaimer: I work on the Flynn project.
  - lifty 10y ago
    Do you think Hashicorp's Raft implementation is the best among the Go implementations? Just curious what is the perception among Raft users.
    - otoolep 10y ago
      Best? I don't know. The etcd implementation is pretty well known too: https://github.com/coreos/etcd/tree/master/raft https://github.com/coreos/etcd/tree/master/raft I'm just more familiar with the one from Hashicorp, and it's used in a few production systems.
    - jpgvm 10y ago
      I think so yes. I haven't used the others to the same degree as the Hashicorp implementation but I have done some research and played around with toy KV stores and have formed the opinion that it's the easiest to get going with and provides the highest performance out of the box due to pipelining. These things may have changed but were my experience when learning Raft. I also expect it to become much better now that development has picked up pace again.
zazibar 10y ago
For those unfamiliar with how the Raft protocol works (as I was) this site[1] provides a great step by step visualization. 1. http://thesecretlivesofdata.com/raft/ http://thesecretlivesofdata.com/raft/
- 0x006A 10y ago
  https://raft.github.io/ https://raft.github.io/ does at better job at giving an overview.
  - kbenson 10y ago
    Which itself says: The Secret Lives[1] of Data is a different visualization of Raft. It's more guided and less interactive, so it may be a gentler starting point. :) 1: The original link.
amelius 10y ago
After taking a quick glance, this implementation seems quite large, and cluttered with details, such as dealing with tcp connections. Wouldn't it be possible to have a much smaller reference implementation?
- otoolep 10y ago
  This is the smallest possible implementation in terms of TCP connections. Those connections are required for the nodes to communicate, and achieve consensus. The HTTP interface is technically optional, but you need some way to interact with the system as a user. You could probably multiplex the use of the TCP connections to do everything, but that would require even more complexity.
  - amelius 10y ago
    I'd really prefer it if the TCP connections were not in the main code, so people could add their own network protocols. In fact, I think the code should not even call any kernel function. Raft is an algorithm after all.
    - otoolep 10y ago
      You can add whatever network connection you like (within reason), as the basis of an implementation. All the Hashicorp implementation requires is that it implements a certain interface. Check the docs: https://godoc.org/github.com/hashicorp/raft https://godoc.org/github.com/hashicorp/raft
zackmorris 10y ago
>Because any node will answer a GET request, and nodes may "fall behind" updates, stale reads are possible. Again, hraftd is a simple program, for the purpose of demonstrating a distributed key-value store. These shortcomings can be addressed by enhancements to the existing source. Late to the conversation, but would you mind updating the README to elaborate upon this? I'm looking for the simplest implementation of Raft but need to be certain that I'm getting the "most recent" value for a key (in other words the last value that passed consensus). I'd like to be able to either: 1) turn on that functionality without modifying the code 2) see a code snippet that achieves it by reading from some number of nodes or whatever 3) be directed to a link explaining if/why it's not possible Otherwise, cool project, congrats!
- otoolep 10y ago
  Sure, I'll add some details, but hraft would require a chunk of work to support other modes of operation. You might like to read a fuller description of this issue for another project based on Raft, called rqlite: https://github.com/rqlite/rqlite/blob/master/doc/CONSISTENCY.md https://github.com/rqlite/rqlite/blob/master/doc/CONSISTENCY... https://github.com/rqlite/rqlite/issues/5 https://github.com/rqlite/rqlite/issues/5 I'll include a link to this from the README.

60 points by otoolep 10y ago

scrollaway 10y ago

Nice work. What prompted you to write this? Are you using it anywhere?

jpgvm 10y ago
Not OP but I am pretty sure OP has worked with/on InfluxDB which used it as part of the clustering component (which I am not sure if exists in the open source release anymore). The hashicorp/raft library actually seems to have a bunch of production users, namely Consul ofcourse but also Flynn (used in service discovery and metadata store called discoverd) and I would assume still InfluxDB. It's currently seeing a resurgence in development as the author of the Raft paper has begun contributing to the library also. Disclaimer: I work on the Flynn project.
- lifty 10y ago
  Do you think Hashicorp's Raft implementation is the best among the Go implementations? Just curious what is the perception among Raft users.
  - otoolep 10y ago
    Best? I don't know. The etcd implementation is pretty well known too: https://github.com/coreos/etcd/tree/master/raft https://github.com/coreos/etcd/tree/master/raft I'm just more familiar with the one from Hashicorp, and it's used in a few production systems.
  - jpgvm 10y ago
    I think so yes. I haven't used the others to the same degree as the Hashicorp implementation but I have done some research and played around with toy KV stores and have formed the opinion that it's the easiest to get going with and provides the highest performance out of the box due to pipelining. These things may have changed but were my experience when learning Raft. I also expect it to become much better now that development has picked up pace again.

zazibar 10y ago

For those unfamiliar with how the Raft protocol works (as I was) this site[1] provides a great step by step visualization. 1. http://thesecretlivesofdata.com/raft/ http://thesecretlivesofdata.com/raft/

0x006A 10y ago
https://raft.github.io/ https://raft.github.io/ does at better job at giving an overview.
- kbenson 10y ago
  Which itself says: The Secret Lives[1] of Data is a different visualization of Raft. It's more guided and less interactive, so it may be a gentler starting point. :) 1: The original link.

amelius 10y ago

After taking a quick glance, this implementation seems quite large, and cluttered with details, such as dealing with tcp connections. Wouldn't it be possible to have a much smaller reference implementation?

otoolep 10y ago
This is the smallest possible implementation in terms of TCP connections. Those connections are required for the nodes to communicate, and achieve consensus. The HTTP interface is technically optional, but you need some way to interact with the system as a user. You could probably multiplex the use of the TCP connections to do everything, but that would require even more complexity.
- amelius 10y ago
  I'd really prefer it if the TCP connections were not in the main code, so people could add their own network protocols. In fact, I think the code should not even call any kernel function. Raft is an algorithm after all.
  - otoolep 10y ago
    You can add whatever network connection you like (within reason), as the basis of an implementation. All the Hashicorp implementation requires is that it implements a certain interface. Check the docs: https://godoc.org/github.com/hashicorp/raft https://godoc.org/github.com/hashicorp/raft

zackmorris 10y ago

>Because any node will answer a GET request, and nodes may "fall behind" updates, stale reads are possible. Again, hraftd is a simple program, for the purpose of demonstrating a distributed key-value store. These shortcomings can be addressed by enhancements to the existing source. Late to the conversation, but would you mind updating the README to elaborate upon this? I'm looking for the simplest implementation of Raft but need to be certain that I'm getting the "most recent" value for a key (in other words the last value that passed consensus). I'd like to be able to either: 1) turn on that functionality without modifying the code 2) see a code snippet that achieves it by reading from some number of nodes or whatever 3) be directed to a link explaining if/why it's not possible Otherwise, cool project, congrats!

otoolep 10y ago
Sure, I'll add some details, but hraft would require a chunk of work to support other modes of operation. You might like to read a fuller description of this issue for another project based on Raft, called rqlite: https://github.com/rqlite/rqlite/blob/master/doc/CONSISTENCY.md https://github.com/rqlite/rqlite/blob/master/doc/CONSISTENCY... https://github.com/rqlite/rqlite/issues/5 https://github.com/rqlite/rqlite/issues/5 I'll include a link to this from the README.