4 ms·

I've been working through a basic implementation over the weekend [0] in Go. I am skipping dynamic cluster management and log compaction/snapshots. I mostly got

I've been working through a basic implementation over the weekend [0] in Go. I am skipping dynamic cluster management and log compaction/snapshots. I mostly got leader election working and the basics of log replication. But it's not yet correct because nodes that are down when new messages come in are not correctly brought up to date. :D Still working through issues. When it's less buggy I'll write a post. If you haven't read the paper, just try it out [1]! It's pretty approachable as far as these things go. It even includes its own cheatsheet inside the paper at Figure 2 (page 4). [0] https://twitter.com/phil_eaton/status/1576281031367221249 https://twitter.com/phil_eaton/status/1576281031367221249 [1] https://raft.github.io/raft.pdf https://raft.github.io/raft.pdf

AtNightWeCode 4y ago
I believe Hashicorp has an OSS GO implementation of raft if I remember correctly.
- eatonphil 4y ago
  Yup! etcd [0] and hashicorp [1] have Raft libraries for Go. I've written about using the hashicorp one before [2, 3]. [0] https://pkg.go.dev/go.etcd.io/etcd/raft/v3 https://pkg.go.dev/go.etcd.io/etcd/raft/v3 [1] https://github.com/hashicorp/raft https://github.com/hashicorp/raft [2] https://notes.eatonphil.com/distributed-postgres.html https://notes.eatonphil.com/distributed-postgres.html [3] https://notes.eatonphil.com/minimal-key-value-store-with-hashicorp-raft.html https://notes.eatonphil.com/minimal-key-value-store-with-has...
  - candiddevmike 4y ago
    Why did you pick HC raft over etcd?
    - eatonphil 4y ago
      The Hashicorp one came up first in a google search months ago. So, basically no reason.
      - asadawadia 4y ago
        love it
    - didip 4y ago
      IMHO, the code is easier to follow than etcd's.
      - skyde 4y ago
        yes it’s more modular with clear contract between the parts
    - withinboredom 4y ago
      Etcd will spam your logs on quorum loss. It can easily fill your disk before the node comes back online or you wake up to fix it. Etcd logging is playing with fire. Just be careful.
    - eatonphil 4y ago
      To defend etcd's implementation though, it is what cockroachdb uses. https://github.com/cockroachdb/cockroach/blob/master/go.mod#L155 https://github.com/cockroachdb/cockroach/blob/master/go.mod#...

AtNightWeCode 4y ago

I believe Hashicorp has an OSS GO implementation of raft if I remember correctly.

eatonphil 4y ago
Yup! etcd [0] and hashicorp [1] have Raft libraries for Go. I've written about using the hashicorp one before [2, 3]. [0] https://pkg.go.dev/go.etcd.io/etcd/raft/v3 https://pkg.go.dev/go.etcd.io/etcd/raft/v3 [1] https://github.com/hashicorp/raft https://github.com/hashicorp/raft [2] https://notes.eatonphil.com/distributed-postgres.html https://notes.eatonphil.com/distributed-postgres.html [3] https://notes.eatonphil.com/minimal-key-value-store-with-hashicorp-raft.html https://notes.eatonphil.com/minimal-key-value-store-with-has...
- candiddevmike 4y ago
  Why did you pick HC raft over etcd?
  - eatonphil 4y ago
    The Hashicorp one came up first in a google search months ago. So, basically no reason.
    - asadawadia 4y ago
      love it
  - didip 4y ago
    IMHO, the code is easier to follow than etcd's.
    - skyde 4y ago
      yes it’s more modular with clear contract between the parts
  - withinboredom 4y ago
    Etcd will spam your logs on quorum loss. It can easily fill your disk before the node comes back online or you wake up to fix it. Etcd logging is playing with fire. Just be careful.
  - eatonphil 4y ago
    To defend etcd's implementation though, it is what cockroachdb uses. https://github.com/cockroachdb/cockroach/blob/master/go.mod#L155 https://github.com/cockroachdb/cockroach/blob/master/go.mod#...