4 ms·

Principles of Distributed Computing

268 points by olalonde 12y ago

olalonde 12y ago
The whole book is available for download here: http://dcg.ethz.ch/lectures/podc_allstars/lecture/podc.pdf http://dcg.ethz.ch/lectures/podc_allstars/lecture/podc.pdf Has anyone followed the course or read the book? I was actually reading "Distributed Systems - Concepts and Design (Third Edition)" and thought it felt a bit outdated and not enough focused for my taste (there are whole chapters on networking and operating systems for example). Then I found this course/book which seems a lot more in depth and modern but I haven't had time to read it yet and couldn't find any reviews.
- shadeslayer 12y ago
  Would you recommend this for someone who is just beginning with the basics of Distributed Computing and does not have a CS background? If not, any recommendations? :) I'm very much interested in distributed computing and was recently reading up about things like the Raft consensus algorithm.
  - oinksoft 12y ago
    Elements of Distributed by Computing by Vijay Garg is great: http://www.amazon.com/Elements-Distributed-Computing-Vijay-Garg/dp/0471036005 http://www.amazon.com/Elements-Distributed-Computing-Vijay-G...
  - deleted 12y ago
    [deleted]
- jandrewrogers 12y ago
  The book is quite good for elementary distributed systems theory. You would not want to design sophisticated systems from nothing but the principles in that book but it does a nice job of outlining the areas of theory that you need to understand in order to excel at it. Each of the chapters hints at topic areas that require a much deeper dive if you want to master distributed systems. Like Wikipedia, it is merely a starting point. For example, the discussion of greedy routing theory barely scrapes the surface of what is known about designing distributed system with approximately optimal aggregate system throughput. You can get by in many cases without fully understanding the topics in this book. Most popular distributed systems that exist today ignore fundamental concepts in at least one of the theoretical areas discussed. But that is also why distributed systems tend to be brittle or exhibit mediocre scalability. The book doesn't fully explain why some of these areas are important but if you study each of the subject areas, it will become obvious.
  - subbu 12y ago
    What books/resources do you recommend for advanced topics?
- FracMat 12y ago
  Having followed the course in 2010 I can fully recommend the script. It gives an exceptionally comprehensive explanation into the most important distributed computing principles without requiring much (if any) previous knowledge on the topic. The course helped me a lot to get a grasp on the complexity of the various underlying challenges the distributed computing and routing world is facing (and with it an idea of the speed achievable). I can recommend anyone enrolled to visit the course in person (most algorithms and solutions are being "discovered" together with the class on the blackboard, which helped me a lot with a deeper understanding), but I found that the script alone already gives a very good foundation of the principles of distributed computing.
  - bluelu 12y ago
    I also followed the course in 2005/2006. It was one of the better lectures and I really much enjoyed it at that time.
jonpress 12y ago
Looks like good reading, thanks! I'm always interested in new techniques for use in my project http://socketcluster.io/ http://socketcluster.io/
javajosh 12y ago
I'm actually rather curious to know how HN people use resources like this. Do you set aside a few hours a week and do a self-course on the content? Do you skim it and smile and node? Do you bookmark it and never get around to reading it? Something else...?
- olalonde 12y ago
  I usually try to go through the whole thing quickly in a weekend (usually skipping exercises unless I'm not sure I understood something correctly). This allows me to get rid of a chunk of my "unkown unkowns" (e.g. if I ever face a problem related to the topic, I will know a solution exists). I will sometime do a more in depth study a few months or even years later if the topic truly interests me or if I need the knowledge for a specific problem I'm trying to solve. (see just-in-time vs. just-in-case learning for pros and cons) If I really want to master the topic, I will try to only study a few hours a week and do the exercises, but I rarely get the discipline required.
- jonpress 12y ago
  I usually bookmark the page then later when I have the time I will read the parts that could be useful for my project.
- jackgolding 12y ago
  Smile and nod :)
- walrus 12y ago
  If it's something I can read in a couple days' worth of free time (~12 hours) and it really interests me, then I'll read it and work the exercises. Otherwise I save a copy and never read it. I don't recommend my approach.
- hiq 12y ago
  It depends on the topic (especially if it can be helpful for work in the near future), but the best approach I have found so far is to put everything at least mildly interesting in a single place, and to process every item one by one whenever I have some free time. That way you always have something to read when on a plane etc. It is also easier to get rid of the urge of reading every paper in the world when you're no longer on HN but with a folder full of files you're supposed to process, so when you realize you're not really interested in one of the items you just throw it away.
krat0sprakhar 12y ago
Shameless plug: If you're interested in more such university courses on systems that make their lectures, assignments available checkout https://github.com/prakhar1989/awesome-courses#systems https://github.com/prakhar1989/awesome-courses#systems
YesThatTom2 12y ago
Shameless plug... "The Practice of Cloud Computing" is a more balanced approach. "Unsatisfied with books that cover either design or operations in isolation, the authors created this authoritative reference centered around a comprehensive approach." (quote from the back cover) Ladies and gentlemen, I present exhibit A: the ToC of "Principles of Distributed Computing". All design. No operations. http://the-cloud-book.com http://the-cloud-book.com
- rev_bird 12y ago
  Seconding this recommendation, and not just because I suspect YesThatTom2 is Mr. Limoncelli. It might look at first like it just goes over the basics (what a load balancer is, what config management is, etc.), but it covers a huge amount of material in very accessible terminology and does a lot to clear up best practices that might not be obvious at first. Just bought a copy for a new junior member of our team who is terrified by all this new stuff; I'm totally confident he'll be able to get through the book and feel much better about things.
wstrange 12y ago
I am curious why Jini gets no mention in a book on distributed computing. Jini was not commercially successful, but some of the concepts were quite interesting: mobile code, downloadable client proxies, leasing, look-up by interface, etc. "The end of protocols" is a short but good read: http://www.cc.gatech.edu/~keith/classes/ubicomplexity/pdfs/infra/waldo-end-of-proto.pdf http://www.cc.gatech.edu/~keith/classes/ubicomplexity/pdfs/i...
- justin66 12y ago
  It was never popular and has been a dead technology for over a decade. (I can only go by mentions in publications. For all I know it has millions of deployments in some gadget or another. I suspect not.)
  - seanmcdirmid 12y ago
    Not entirely true; many of the concepts from Linda (Jini/Javaspaces) are around in the form of KV stores.
    - justin66 12y ago
      Could you elaborate on that? I'm pretty sure you mean something more sophisticated than a key-value store, which is about as old and dead simple a concept as we have.
      - seanmcdirmid 12y ago
        No, I mean, tuple spaces were always quite simple as well as a simplification, and the use of KV stores in many distributed computing tasks continued that trend under another less soiled name.

268 points by olalonde 12y ago

The whole book is available for download here: http://dcg.ethz.ch/lectures/podc_allstars/lecture/podc.pdf http://dcg.ethz.ch/lectures/podc_allstars/lecture/podc.pdf Has anyone followed the course or read the book? I was actually reading "Distributed Systems - Concepts and Design (Third Edition)" and thought it felt a bit outdated and not enough focused for my taste (there are whole chapters on networking and operating systems for example). Then I found this course/book which seems a lot more in depth and modern but I haven't had time to read it yet and couldn't find any reviews.

shadeslayer 12y ago
Would you recommend this for someone who is just beginning with the basics of Distributed Computing and does not have a CS background? If not, any recommendations? :) I'm very much interested in distributed computing and was recently reading up about things like the Raft consensus algorithm.
- oinksoft 12y ago
  Elements of Distributed by Computing by Vijay Garg is great: http://www.amazon.com/Elements-Distributed-Computing-Vijay-Garg/dp/0471036005 http://www.amazon.com/Elements-Distributed-Computing-Vijay-G...
- deleted 12y ago
  [deleted]
jandrewrogers 12y ago
The book is quite good for elementary distributed systems theory. You would not want to design sophisticated systems from nothing but the principles in that book but it does a nice job of outlining the areas of theory that you need to understand in order to excel at it. Each of the chapters hints at topic areas that require a much deeper dive if you want to master distributed systems. Like Wikipedia, it is merely a starting point. For example, the discussion of greedy routing theory barely scrapes the surface of what is known about designing distributed system with approximately optimal aggregate system throughput. You can get by in many cases without fully understanding the topics in this book. Most popular distributed systems that exist today ignore fundamental concepts in at least one of the theoretical areas discussed. But that is also why distributed systems tend to be brittle or exhibit mediocre scalability. The book doesn't fully explain why some of these areas are important but if you study each of the subject areas, it will become obvious.
- subbu 12y ago
  What books/resources do you recommend for advanced topics?
FracMat 12y ago
Having followed the course in 2010 I can fully recommend the script. It gives an exceptionally comprehensive explanation into the most important distributed computing principles without requiring much (if any) previous knowledge on the topic. The course helped me a lot to get a grasp on the complexity of the various underlying challenges the distributed computing and routing world is facing (and with it an idea of the speed achievable). I can recommend anyone enrolled to visit the course in person (most algorithms and solutions are being "discovered" together with the class on the blackboard, which helped me a lot with a deeper understanding), but I found that the script alone already gives a very good foundation of the principles of distributed computing.
- bluelu 12y ago
  I also followed the course in 2005/2006. It was one of the better lectures and I really much enjoyed it at that time.

jonpress 12y ago

Looks like good reading, thanks! I'm always interested in new techniques for use in my project http://socketcluster.io/ http://socketcluster.io/

javajosh 12y ago

I'm actually rather curious to know how HN people use resources like this. Do you set aside a few hours a week and do a self-course on the content? Do you skim it and smile and node? Do you bookmark it and never get around to reading it? Something else...?

olalonde 12y ago
I usually try to go through the whole thing quickly in a weekend (usually skipping exercises unless I'm not sure I understood something correctly). This allows me to get rid of a chunk of my "unkown unkowns" (e.g. if I ever face a problem related to the topic, I will know a solution exists). I will sometime do a more in depth study a few months or even years later if the topic truly interests me or if I need the knowledge for a specific problem I'm trying to solve. (see just-in-time vs. just-in-case learning for pros and cons) If I really want to master the topic, I will try to only study a few hours a week and do the exercises, but I rarely get the discipline required.
jonpress 12y ago
I usually bookmark the page then later when I have the time I will read the parts that could be useful for my project.
jackgolding 12y ago
Smile and nod :)
walrus 12y ago
If it's something I can read in a couple days' worth of free time (~12 hours) and it really interests me, then I'll read it and work the exercises. Otherwise I save a copy and never read it. I don't recommend my approach.
hiq 12y ago
It depends on the topic (especially if it can be helpful for work in the near future), but the best approach I have found so far is to put everything at least mildly interesting in a single place, and to process every item one by one whenever I have some free time. That way you always have something to read when on a plane etc. It is also easier to get rid of the urge of reading every paper in the world when you're no longer on HN but with a folder full of files you're supposed to process, so when you realize you're not really interested in one of the items you just throw it away.

krat0sprakhar 12y ago

Shameless plug: If you're interested in more such university courses on systems that make their lectures, assignments available checkout https://github.com/prakhar1989/awesome-courses#systems https://github.com/prakhar1989/awesome-courses#systems

YesThatTom2 12y ago

Shameless plug... "The Practice of Cloud Computing" is a more balanced approach. "Unsatisfied with books that cover either design or operations in isolation, the authors created this authoritative reference centered around a comprehensive approach." (quote from the back cover) Ladies and gentlemen, I present exhibit A: the ToC of "Principles of Distributed Computing". All design. No operations. http://the-cloud-book.com http://the-cloud-book.com

rev_bird 12y ago
Seconding this recommendation, and not just because I suspect YesThatTom2 is Mr. Limoncelli. It might look at first like it just goes over the basics (what a load balancer is, what config management is, etc.), but it covers a huge amount of material in very accessible terminology and does a lot to clear up best practices that might not be obvious at first. Just bought a copy for a new junior member of our team who is terrified by all this new stuff; I'm totally confident he'll be able to get through the book and feel much better about things.

wstrange 12y ago

I am curious why Jini gets no mention in a book on distributed computing. Jini was not commercially successful, but some of the concepts were quite interesting: mobile code, downloadable client proxies, leasing, look-up by interface, etc. "The end of protocols" is a short but good read: http://www.cc.gatech.edu/~keith/classes/ubicomplexity/pdfs/infra/waldo-end-of-proto.pdf http://www.cc.gatech.edu/~keith/classes/ubicomplexity/pdfs/i...

justin66 12y ago
It was never popular and has been a dead technology for over a decade. (I can only go by mentions in publications. For all I know it has millions of deployments in some gadget or another. I suspect not.)
- seanmcdirmid 12y ago
  Not entirely true; many of the concepts from Linda (Jini/Javaspaces) are around in the form of KV stores.
  - justin66 12y ago
    Could you elaborate on that? I'm pretty sure you mean something more sophisticated than a key-value store, which is about as old and dead simple a concept as we have.
    - seanmcdirmid 12y ago
      No, I mean, tuple spaces were always quite simple as well as a simplification, and the use of KV stores in many distributed computing tasks continued that trend under another less soiled name.