I led the storage engine prototyping for Kinesis in 2012 (the best time in my career so far). Kinesis uses Chain Replication, a dead simple fault tolerante sto

I led the storage engine prototyping for Kinesis in 2012 (the best time in my career so far). Kinesis uses Chain Replication, a dead simple fault tolerante storage algorithm: machines formed a chain, data flow from head to tail in one direction, writes always start at head, and read at tail, new nodes always join at tail, but nodes can be kicked out at any position. The membership management of chain node is done through a paxos-based consensus service like chubby or zookeeper. Allan [2] (the best engineer I personally worked with so far, way better than anyone I encountered) wrote that system. The Java code quality shows itself after the first glance. Not mentioning the humbleness and openness in sharing his knowledge during early design meetings. I am not sure what protocol is actually used now. But I would be surprised it's different, given the protocol's simplicity and performance. [1] https://www.cs.cornell.edu/home/rvr/papers/OSDI04.pdf https://www.cs.cornell.edu/home/rvr/papers/OSDI04.pdf [2] https://www.linkedin.com/in/allan-vermeulen-58835b/ https://www.linkedin.com/in/allan-vermeulen-58835b/