3 ms·

I had a fun time recently implementing Raft leader election and log replication (i.e. I didn't get to snapshotting/checkpointing). One of the most challenging p

I had a fun time recently implementing Raft leader election and log replication (i.e. I didn't get to snapshotting/checkpointing). One of the most challenging projects I've tried to do. The Raft paper is very gentle to read, and gives you a great intuition on its own. Even if you don't want to implement it, you probably use software that uses it: like etcd or consul or cockroach or tidb, etc. I collected all the resources I found useful while doing it here: https://github.com/eatonphil/goraft#references https://github.com/eatonphil/goraft#references. This includes Diego Ongaro's thesis and his TLA+ spec. Some people say Figure 2 of the Raft paper has everything you need but I'm pretty sure that's just not true. It's a little bit more vague than looking at the TLA+ spec to me anyway.

kevdev 3y ago
Figure 2 is great, but I would agree the entire paper is needed if you are implementing raft. There are a few specifics in the paper that you need when implementing it. P.S. love your blog :)
- eatonphil 3y ago
  Cheers! > Figure 2 is great, but I would agree the entire paper is needed if you are implementing raft. There are a few specifics in the paper that you need when implementing it. It was more than that. I'm blanking on what it was but there were parts where I really couldn't find anything about the intended behavior in the paper (let alone in Figure 2) except for in Diego's thesis or in the TLA+ spec. Though maybe I was just not reading the paper correctly.
agonz253 3y ago
This looks great. However, without a truly comprehensive test suite it’s guaranteed to have a great many subtle bugs :). I recommend trying to hook it up with the tests from MIT’s distributed systems course: https://pdos.csail.mit.edu/6.824/labs/lab-raft.html https://pdos.csail.mit.edu/6.824/labs/lab-raft.html. It looks as though it would take only a minor refactoring at least for the leader election and log replication.
skowalak 3y ago
I had to implement Raft for a network programming course during my bachelors and I had the same experience regarding how gentle the paper was. Especially for people new to distributed algorithms, I can really recommend it. My implementation is probably not that great, but I put it online anyway if anyone is interested: https://github.com/skowalak/fastapi-raft/ https://github.com/skowalak/fastapi-raft/

kevdev 3y ago

Figure 2 is great, but I would agree the entire paper is needed if you are implementing raft. There are a few specifics in the paper that you need when implementing it. P.S. love your blog :)

eatonphil 3y ago
Cheers! > Figure 2 is great, but I would agree the entire paper is needed if you are implementing raft. There are a few specifics in the paper that you need when implementing it. It was more than that. I'm blanking on what it was but there were parts where I really couldn't find anything about the intended behavior in the paper (let alone in Figure 2) except for in Diego's thesis or in the TLA+ spec. Though maybe I was just not reading the paper correctly.

agonz253 3y ago

This looks great. However, without a truly comprehensive test suite it’s guaranteed to have a great many subtle bugs :). I recommend trying to hook it up with the tests from MIT’s distributed systems course: https://pdos.csail.mit.edu/6.824/labs/lab-raft.html https://pdos.csail.mit.edu/6.824/labs/lab-raft.html. It looks as though it would take only a minor refactoring at least for the leader election and log replication.

skowalak 3y ago

I had to implement Raft for a network programming course during my bachelors and I had the same experience regarding how gentle the paper was. Especially for people new to distributed algorithms, I can really recommend it. My implementation is probably not that great, but I put it online anyway if anyone is interested: https://github.com/skowalak/fastapi-raft/ https://github.com/skowalak/fastapi-raft/