4 ms·

Distributed systems for fun and profit

131 points by austengary 13y ago

dysoco 13y ago
This is just what I wanted, something that explained me all this amazing world of distributed computing and networking. If someone could suggest a good book, that'd be interesting too (I've been looking at Unix Network Programming and The TCP/IP Guide for starting with networks).
- mjb 13y ago
  That depends on what level of book, and what you want to learn from it. On "how real distributed systems are built", there isn't a really good book available in my opinion. I liked The Datacenter as a Computer, but it's scope is fairly narrow and is more focused on global infrastructure issues than software design issues. On the math behind distributed systems, Nancy Lynch's book Distributed Algorithms (http://www.amazon.com/Distributed-Algorithms-Kaufmann-Management-Systems/dp/1558603484 http://www.amazon.com/Distributed-Algorithms-Kaufmann-Manage...) is my favorite, but it's getting to be quite out-of-date. Absent a book, there are a lot of great resources online. High Scalability(http://highscalability.com/ http://highscalability.com/) has a good amount of quality links to distributed systems content, and some editorial content of varying quality. In general, it's a good site to follow if you're interested in how distributed systems are being used in industry. I also like to follow Aphyr's blog(http://aphyr.com/ http://aphyr.com/), Daniel Abadi's blog(http://dbmsmusings.blogspot.com/ http://dbmsmusings.blogspot.com/), and Henry Robinson's blog(http://the-paper-trail.org/blog/ http://the-paper-trail.org/blog/) and Peter Bailis's blog(http://www.bailis.org/ http://www.bailis.org/).
  - dysoco 13y ago
    Thanks. Basically I'd like to understand all this Redis, ZeroMQ, etc. stuff, message passing, queues, etc. Not sure if there's a book that explains all this, or if I have to do a bit of research by myself.
    - contingencies 13y ago
      Just started working on an outline. I am no expert, but have faith that I can research and simplify an explanation at the same time :) https://github.com/mixu/distsysbook/issues/2 https://github.com/mixu/distsysbook/issues/2 (Edit: On hold waiting for author to wake up .. see issues list)
  - packetslave 13y ago
    As a side note, the second edition of "Datacenter as a Computer" was recently released: http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y201306CAC024 http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y2...
  - gtani 13y ago
    don't overlook the link in Robinson's upper right page http://henryr.github.io/distributed-systems-readings/ http://henryr.github.io/distributed-systems-readings/ also: http://christophermeiklejohn.com/distributed/systems/2013/07/12/readings-in-distributed-systems.html http://christophermeiklejohn.com/distributed/systems/2013/07...
- netvarun 13y ago
  Another really good article on distributed systems along the same lines is this one: http://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/ http://www.somethingsimilar.com/2013/01/14/notes-on-distribu...
- jackowayed 13y ago
  The Architecture of Open Source Applications books[1], while eclectic, seem pretty good from the chapters I've read. They have a few chapters about distributed systems--HDFS, Riak, and a chapter about "Scalable Web Architecture and Distributed Systems" (haven't read it). 1: http://aosabook.org/en/index.html http://aosabook.org/en/index.html
- ovi256 13y ago
  Distributed Systems: Principles and Paradigms by Tannenbaum is a classic textbook. It's considered pretty dry though. I recommend strongly MIT's Distributed Systems course, both for the lectures and the labs. See http://pdos.csail.mit.edu/6.824 http://pdos.csail.mit.edu/6.824 As a bonus, the labs (in Go) give you invaluable hands-on knowledge.
- vdm 13y ago
  KTH ID2203 Distributed Systems, Advanced Course, with Youtube video lectures http://www.ict.kth.se/courses/ID2203/video_lectures.html http://www.ict.kth.se/courses/ID2203/video_lectures.html By Seif Haridi, who co-authored 'Concepts, Techniques, and Models of Computer Programming' which is well regarded among functional programmers.
peterwwillis 13y ago
While this subject is full of interesting sub-topics to investigate for real-world use, Parallel Computing is a field much wider than just a DMM. Look at systems designed more than 10 years ago for better ideas about alternative computing models (all the good papers were written by the end of the 20th century). For example, if your "big data" is just computationally-intensive normal data, you might want to look into PVM/MPI or SSI. It's not as sexy, though, because you don't use it in JavaScript.
- capkutay 13y ago
  I'd love to see tools like MPI come back into mainstream. For instance, there's some work being done optimizing MPI in java for high performance computing but not nearly as much as what's being done for tools like Hadoop.
squeed 13y ago
Years ago I watched MIT's course 6.033. It is a great way to understand the first principles at hand in distributed / non-trivial programs: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-033-computer-system-engineering-spring-2009/video-lectures/ http://ocw.mit.edu/courses/electrical-engineering-and-comput... (also on iTunesU). If you're just getting started with this stuff, having a solid understanding of these principles is critical. I can't recommend these videos enough.
contingencies 13y ago
I was skeptical to click this but it's piqued my interest. If it continues as cogently as it began, then the author has my congratulations: it's said that if you truly understand something, you can explain it eloquently to others.

131 points by austengary 13y ago

This is just what I wanted, something that explained me all this amazing world of distributed computing and networking. If someone could suggest a good book, that'd be interesting too (I've been looking at Unix Network Programming and The TCP/IP Guide for starting with networks).

mjb 13y ago
That depends on what level of book, and what you want to learn from it. On "how real distributed systems are built", there isn't a really good book available in my opinion. I liked The Datacenter as a Computer, but it's scope is fairly narrow and is more focused on global infrastructure issues than software design issues. On the math behind distributed systems, Nancy Lynch's book Distributed Algorithms (http://www.amazon.com/Distributed-Algorithms-Kaufmann-Management-Systems/dp/1558603484 http://www.amazon.com/Distributed-Algorithms-Kaufmann-Manage...) is my favorite, but it's getting to be quite out-of-date. Absent a book, there are a lot of great resources online. High Scalability(http://highscalability.com/ http://highscalability.com/) has a good amount of quality links to distributed systems content, and some editorial content of varying quality. In general, it's a good site to follow if you're interested in how distributed systems are being used in industry. I also like to follow Aphyr's blog(http://aphyr.com/ http://aphyr.com/), Daniel Abadi's blog(http://dbmsmusings.blogspot.com/ http://dbmsmusings.blogspot.com/), and Henry Robinson's blog(http://the-paper-trail.org/blog/ http://the-paper-trail.org/blog/) and Peter Bailis's blog(http://www.bailis.org/ http://www.bailis.org/).
- dysoco 13y ago
  Thanks. Basically I'd like to understand all this Redis, ZeroMQ, etc. stuff, message passing, queues, etc. Not sure if there's a book that explains all this, or if I have to do a bit of research by myself.
  - contingencies 13y ago
    Just started working on an outline. I am no expert, but have faith that I can research and simplify an explanation at the same time :) https://github.com/mixu/distsysbook/issues/2 https://github.com/mixu/distsysbook/issues/2 (Edit: On hold waiting for author to wake up .. see issues list)
- packetslave 13y ago
  As a side note, the second edition of "Datacenter as a Computer" was recently released: http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y201306CAC024 http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y2...
- gtani 13y ago
  don't overlook the link in Robinson's upper right page http://henryr.github.io/distributed-systems-readings/ http://henryr.github.io/distributed-systems-readings/ also: http://christophermeiklejohn.com/distributed/systems/2013/07/12/readings-in-distributed-systems.html http://christophermeiklejohn.com/distributed/systems/2013/07...
netvarun 13y ago
Another really good article on distributed systems along the same lines is this one: http://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/ http://www.somethingsimilar.com/2013/01/14/notes-on-distribu...
jackowayed 13y ago
The Architecture of Open Source Applications books[1], while eclectic, seem pretty good from the chapters I've read. They have a few chapters about distributed systems--HDFS, Riak, and a chapter about "Scalable Web Architecture and Distributed Systems" (haven't read it). 1: http://aosabook.org/en/index.html http://aosabook.org/en/index.html
ovi256 13y ago
Distributed Systems: Principles and Paradigms by Tannenbaum is a classic textbook. It's considered pretty dry though. I recommend strongly MIT's Distributed Systems course, both for the lectures and the labs. See http://pdos.csail.mit.edu/6.824 http://pdos.csail.mit.edu/6.824 As a bonus, the labs (in Go) give you invaluable hands-on knowledge.
vdm 13y ago
KTH ID2203 Distributed Systems, Advanced Course, with Youtube video lectures http://www.ict.kth.se/courses/ID2203/video_lectures.html http://www.ict.kth.se/courses/ID2203/video_lectures.html By Seif Haridi, who co-authored 'Concepts, Techniques, and Models of Computer Programming' which is well regarded among functional programmers.

peterwwillis 13y ago

While this subject is full of interesting sub-topics to investigate for real-world use, Parallel Computing is a field much wider than just a DMM. Look at systems designed more than 10 years ago for better ideas about alternative computing models (all the good papers were written by the end of the 20th century). For example, if your "big data" is just computationally-intensive normal data, you might want to look into PVM/MPI or SSI. It's not as sexy, though, because you don't use it in JavaScript.

capkutay 13y ago
I'd love to see tools like MPI come back into mainstream. For instance, there's some work being done optimizing MPI in java for high performance computing but not nearly as much as what's being done for tools like Hadoop.

squeed 13y ago

Years ago I watched MIT's course 6.033. It is a great way to understand the first principles at hand in distributed / non-trivial programs: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-033-computer-system-engineering-spring-2009/video-lectures/ http://ocw.mit.edu/courses/electrical-engineering-and-comput... (also on iTunesU). If you're just getting started with this stuff, having a solid understanding of these principles is critical. I can't recommend these videos enough.

contingencies 13y ago

I was skeptical to click this but it's piqued my interest. If it continues as cogently as it began, then the author has my congratulations: it's said that if you truly understand something, you can explain it eloquently to others.