6 ms·
It took me months to do a reliable Raft implementation. The author of the Raft paper, Diego Ongaro, wrote in a comment on a Raft mailing list that it could take
by ccleve 8y ago
It took me months to do a reliable Raft implementation. The author of the Raft paper, Diego Ongaro, wrote in a comment on a Raft mailing list that it could take weeks.
The Raft paper is really easy to read and understand, but the actual code is very tricky.
- justinjlynn 8y agoWhat did you find most difficult about the implementation?
- Matheus28 8y agoNot OP, but it is very easy to make mistakes while implementing it and end up with something that is not actually reliable. You have to have the paper opened and write code line by line looking at it. I implemented it a while ago so that's the most I remember, but it did a bit of fuzzing to nail down every bug.
- isbvhodnvemrwvn 8y agoAlso not OP, I think I went through the paper 10 times over during the implementation. There's a lot of cross-references all over the place.
- karl-police 8y agoCould I ask you something, or maybe on Steam?
- karl-police 8y agoOr I could ask you here one of the questions. Do you host diep.io on cloudflare, heroku, linode etc.? or do you have own servers? Because I'm interested about developing something.
- ccleve 8y agoTo make it work with any kind of performance, all message passing needs to be asynchronous, and you need an event loop to process messages one at a time. Things are much simpler if your code is effectively single-threaded when processing an inbound message. That took me a while to figure out. Futures and promises are very helpful. It took a while to get the building blocks together: the log file, which can be traversed, read, and rolled back/truncated (use a kv store for this, trust me), and the transport mechanism to deal with sending complex objects and handling errors and timeouts. Tracking terms, states (leader, follower, candidate), rejecting out of date terms, voting, elections -- it's all very complex. You'll have to build some really, really good multi-threaded tests that simulate out-of-order and dropped messages, then really pound away at the system. And good logging to find that odd corner case that keeps coming up and throwing mysterious messages. I haven't even tried membership changes yet.
- bch 8y ago> use a kv store for this, trust me Order is important here - is this inherent in your key?
- ccleve 8y agoYes. I did two implementations, one with plain text files and one with a kv store. For plain text files, I had to maintain file offsets of the records so I could traverse backward and truncate records that had to be replaced, and then break the files into segments so I could retire segments that are no longer needed. It had all kinds of little bugs. Then I reimplemented using a kv store, and it took a tiny fraction of the time. And surprisingly had better performance. Just create a good interface at the start, use the kv store underneath, and then later if you don't want the dependency swap it out for the more complicated code.