3 ms·

Ask HN: What are some good projects/resources to learn distributed systems?

• jxub 8y ago
  I'd recommend the book Designing Data Intensive Applications, by Martin Kleppmann and as for online resources, check out http://book.mixu.net/distsys/single-page.html http://book.mixu.net/distsys/single-page.html for a throughout overview.
  • gyani95 8y ago
    Thanks for the suggestion. I've added it to my list. Do you recommend some good first apps for learning distributed systems?
• iampims 8y ago
  This Github repo is a pretty good start to get familiar with distributed systems: https://github.com/donnemartin/system-design-primer https://github.com/donnemartin/system-design-primer
  • gyani95 8y ago
    I've been through this. It's a good starting point and can be fairly useful for interviews. It gives a good high level overview of systems, enough for you to talk about them. Maybe I can go through this again and dive deeper into topics that interest me.
• convolvatron 8y ago
  do you want to learn how to design distributed systems from first principles or build systems from pre-existing distributed components? unfortunately these are both called 'distributed systems' #1 is alot more interesting. #2 is what people are hiring for
  • gyani95 8y ago
    I'd like to say both! I guess I can get started with the second one and then move onto the first one? Maybe after playing with a few components I can look into their internals.
    • dsnuh 8y ago
      For #2 - Maybe try setting up your own Kubernetes cluster on a cloud service provider, without using shortcuts? Perhaps etcd and the Raft protocol would be interesting and helpful in some cases, but leans more towards #1. For #1 - CAP Theorem, read through aphyr's blog, google concepts/details/jargon you find in white papers on different of the various distributed consensus protocols, etc.
• daoran 8y ago
  Beside Martin Kleppmann's wonderful book, I'd also recommend MIT 6.824 distributed system course, you can find lots of resources on the course website.
• brudgers 8y ago
  Erlang?
• lambda_func 8y ago
  Some stuff you can build to learn more about distributed systems (you can start simple, do research, and then iterate to build a more comprehensive solution): build a distributed file system, a distributed lock server, a consensus algorithm (paxos, raft, etc.), two/three phase commit protocol, distributed hash table, etc.
• simon_acca 8y ago
  Aphyr, of Jepsen fame, published an outline for a distributed systems class that makes for a good reference on what is relevant for most modern distributed systems: https://github.com/aphyr/distsys-class https://github.com/aphyr/distsys-class
  • gyani95 8y ago
    Thanks for this! This sounds lovely and is 12-16 hours long. Fits my schedule :D
• srkiranraj 8y ago
  MapReduce paper, though published in 2004, it is foundational, relevant and very well written paper. https://ai.google/research/pubs/pub62 https://ai.google/research/pubs/pub62 P.S: it has close to 25k citations.
• cottonseed 8y ago
  You might find some of these references useful: Resources for Getting Started with Distributed Systems https://caitiem.com/2017/09/07/getting-started-with-distributed-systems/ https://caitiem.com/2017/09/07/getting-started-with-distribu... Distributed systems theory for the distributed systems engineer http://www.the-paper-trail.org/post/2014-08-09-distributed-systems-theory-for-the-distributed-systems-engineer/ http://www.the-paper-trail.org/post/2014-08-09-distributed-s... Distributed Systems for Fun and Profit http://book.mixu.net/distsys/ http://book.mixu.net/distsys/ Notes on the Theory of Distributed Systems http://cs-www.cs.yale.edu/homes/aspnes/classes/465/notes.pdf http://cs-www.cs.yale.edu/homes/aspnes/classes/465/notes.pdf