5 ms·

Programming Models for Distributed Computation

174 points by simonpure 7y ago

justicezyx 7y ago
The content seems shallow, the topic is extremely broad. Overall I do not feel this topic can be described really well.
- omeze 7y ago
  I think "shallow" is being a bit uncharitable. The book here doesn't dive into ditributed systems theory very deeply but it gives a very rich overview of real industry tools used over the past few decades with plenty of context around why technologies are being used or discarded. They go into code examples of things like CRDTs so its not like its just prose. One thing that feels off to me as a backend engineer that deals with distributed systems in practice is no treatment (AFAICT) of containers/virtual machines and how tools like kubernetes are providing the primitives that the distributed languages of the past provided. In general I think chapter 4 doesn't have a cohesive narrative on its treatment of languages for distributed computation. I know this is messy topic but I think more could be done to tie things to current art. If I were to make a prediction, there's general trend towards protocols (mostly network level above layer 4) being the actual thing of interest for distributed computation, not single language runtimes. Wire-compatible DBs like AWS aurora and MySQL are an example, but also I think things like container orchestator's liveness and readiness checks over HTTP. As more primitives rally around the same core concepts, I think the general trend will be to go from conceptual similarity, to client library compatability (e.g. identical interfaces a la OpenTracing <> Lightstep, Jaeger) and eventually wire/protocol compatibility. Language runtimes are too much of a moving target as there are other pressures at play (community, tooling, typing, yes even syntax) that go into language choices over a decade.
  - riedel 7y ago
    Yes, i was also a bit disappointed reading e.g. the reference e.g. about CSPs, it should at least link some other textbooks here. However, I admire the work being put into this. It would be great if all university teachers would make the effort to publish open access. A dream would be collaboratively maintained textbooks with strong single editors for readability and consistent style. Also there should be foundations like apache, etc that market such material and put up quality control, pay lectors where the publishers are largely failing atm.
- dang 7y ago
  "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
infogulch 7y ago
Linear Logic / Session Types seem interesting. I've poked around some of the papers, but I'm not sure how to approach the subject yet. https://homepages.inf.ed.ac.uk/wadler/topics/linear-logic.html https://homepages.inf.ed.ac.uk/wadler/topics/linear-logic.ht...
- throwaway17_17 7y ago
  Session Types, AFAIU are a specific implementation of a linear logic in some type system and I am not all too familiar with them in practice. However, I would like to think I am well versed in Linear Logic and Substructural Type Theory. I think the works by Wadler are a good place to start, in particular is a sort of review paper, based in intuitionistic logic (which he also covers in said paper), on linear logic. It’s called ‘A Taste of Linear Logic’ (I actually have the paper open in Safari on my iPhone as I type this) and it is written as a sort of primer on the area. Wadler’s bibliographies on the works contained in the pages you linked are a good source as well. If you are at all coming from a place of familiarity with Haskell, a decent overview of the current discussions on Linear Types in functional languages is at [2]. And lastly, if you like videos, Simon Peyton Jones has a great talk on YouTube [3] where he discusses practical linear types for Haskell, but the theory is not Haskell specific. I will warn you that the latter two references make several arguments and then discuss uniqueness typing (which is related but very different) and that is a whole other rabbit hole of sources, but generally a good topic to dig into as well. [1] https://homepages.inf.ed.ac.uk/wadler/papers/lineartaste/lineartaste-revised.pdf https://homepages.inf.ed.ac.uk/wadler/papers/lineartaste/lin... [2] http://edsko.net/2017/01/08/linearity-in-haskell/ http://edsko.net/2017/01/08/linearity-in-haskell/ [3] https://youtu.be/t0mhvd3-60Y https://youtu.be/t0mhvd3-60Y
peatfreak 7y ago
This is going to sound snobby to some people but I don't care. I really believe this. The whole book is written in Markdown. This is a huge red flag for me and the book goes in to the "no" pile pretty much immediately. Distributed computation is a complex and complicated discipline. Moreover if you're going to write a book and you don't even realize that Markdown is completely inappropriate it calls in to question your authoring skills. At best the use of Markdown is lazy. If you are an expert in distributed computation I assume that you're probably capable of using LaTeX, DocBook, *roff, or some other appropriate technical authoring system.
- bruth 7y ago
  What does writing in Markdown have to do with the complexity of the topic? Heather Miller is an assistant professor at CMU and well respected and certainly an expert in this context. Clearly she believed Markdown was sufficient for her purpose and/or wanted to leverage the other benefits beyond just its simplicity. Did you see that book is being rendered on a site? http://dist-prog-book.com/chapter/2/futures.html http://dist-prog-book.com/chapter/2/futures.html
  - throwqwerty 7y ago
    >What does writing in Markdown have to do with the complexity of the topic? nothing. the person you're responding to just wants to feel very good about themselves without actually engaging with the material.
    - dang 7y ago
      Reacting to a bad comment with an even worse comment is not ok here. It's against the site guidelines, and you've been doing it a lot. We've had to ban you before for that, so would you please stop? If you're posting to HN, you're responsible for helping to take care of this place, to the best of your ability and awareness. If others are wrong, correct them respectfully. If they are ignorant, teach them. If you know more, share some of what you know so the rest of us can learn. Tearing others down helps no one, and worsens the very things you're complaining about. I'm sure you wouldn't litter in a city park just because others do, nor would you fail to put out a campfire just because you were mad at other campers. Please bring those values here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
throwqwerty 7y ago
lol hn never fails in it's shallow dismissal of an enormous amount of work: as of right now there's a comment on how this textbook is too shallow (I guess if it were a content marketing medium post it would be much better) and how it can't possibly be substantive because it's written in markdown. man what a bastion of intelligent discussion this place is.
- eternalban 7y ago
  It is fair to point out this is a product of a highly regarded computer science school. So the honest question is: should we subject industry e.g. blogs to the same standard as the academy. I would think we should hold academic products to a much higher standard. I do agree criticism should be constructive. (One of PRs for example noted lack of mention of CSP.)
  - throwqwerty 7y ago
    >I would think we should hold academic products to a much higher standard who is we? are you a tt prof at literally the number 1 cs school in the US? are you a reviewer for any distributed journals? the author of this book is a wildly successful researcher. why should i trust literally almost anyone's here opinion over their opinion?
  - nogabebop23 7y ago
    Maybe when you're assessing the relative quality we should consider this context with more weight, but should that excuse industry for presenting expertise via a light-weight puff piece PR posting?
  - marcosdumay 7y ago
    Hum... It does really depend. Honestly, for me right now, coming from HN into random places, no I don't have different standards. But in a course having a textbook recommendation, yes, the standards are very different. Anyway, you are out on a tangent. The work should be judged based on its quality, and for all the dimensions quality has, the presentation format is not one that matters.
- 7y ago
MUHAMMADRIZWAN 7y ago
https://bit.ly/2QTW8St https://bit.ly/2QTW8St
troelsSteegin 7y ago
The menu of papers for the associated class, https://heather.miller.am/teaching/cs7680/ https://heather.miller.am/teaching/cs7680/, looks really rich. The class is from 2016. Looks like the book dates from 2016-2017. My first impression was that this was work in progress ... I'd be curious to see what from this set Adrian Colyer https://blog.acolyer.org/ https://blog.acolyer.org/ has already digested. A perhaps interesting follow on would be to see which of these programming models are manifest today across the big cloud services .. or how they have advanced from this legacy work.
zzzcpan 7y ago
For those who didn't figure it out - instead of raw markdown you can read the book already in html format that browsers can render: http://dist-prog-book.com/chapter/ http://dist-prog-book.com/chapter/
macintux 7y ago
I haven’t checked these links for a couple of years, but here’s a collection of distributed systems reading lists I compiled. https://gist.github.com/macintux/6227368 https://gist.github.com/macintux/6227368

174 points by simonpure 7y ago

justicezyx 7y ago

The content seems shallow, the topic is extremely broad. Overall I do not feel this topic can be described really well.

omeze 7y ago
I think "shallow" is being a bit uncharitable. The book here doesn't dive into ditributed systems theory very deeply but it gives a very rich overview of real industry tools used over the past few decades with plenty of context around why technologies are being used or discarded. They go into code examples of things like CRDTs so its not like its just prose. One thing that feels off to me as a backend engineer that deals with distributed systems in practice is no treatment (AFAICT) of containers/virtual machines and how tools like kubernetes are providing the primitives that the distributed languages of the past provided. In general I think chapter 4 doesn't have a cohesive narrative on its treatment of languages for distributed computation. I know this is messy topic but I think more could be done to tie things to current art. If I were to make a prediction, there's general trend towards protocols (mostly network level above layer 4) being the actual thing of interest for distributed computation, not single language runtimes. Wire-compatible DBs like AWS aurora and MySQL are an example, but also I think things like container orchestator's liveness and readiness checks over HTTP. As more primitives rally around the same core concepts, I think the general trend will be to go from conceptual similarity, to client library compatability (e.g. identical interfaces a la OpenTracing <> Lightstep, Jaeger) and eventually wire/protocol compatibility. Language runtimes are too much of a moving target as there are other pressures at play (community, tooling, typing, yes even syntax) that go into language choices over a decade.
- riedel 7y ago
  Yes, i was also a bit disappointed reading e.g. the reference e.g. about CSPs, it should at least link some other textbooks here. However, I admire the work being put into this. It would be great if all university teachers would make the effort to publish open access. A dream would be collaboratively maintained textbooks with strong single editors for readability and consistent style. Also there should be foundations like apache, etc that market such material and put up quality control, pay lectors where the publishers are largely failing atm.
dang 7y ago
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

infogulch 7y ago

Linear Logic / Session Types seem interesting. I've poked around some of the papers, but I'm not sure how to approach the subject yet. https://homepages.inf.ed.ac.uk/wadler/topics/linear-logic.html https://homepages.inf.ed.ac.uk/wadler/topics/linear-logic.ht...

throwaway17_17 7y ago
Session Types, AFAIU are a specific implementation of a linear logic in some type system and I am not all too familiar with them in practice. However, I would like to think I am well versed in Linear Logic and Substructural Type Theory. I think the works by Wadler are a good place to start, in particular is a sort of review paper, based in intuitionistic logic (which he also covers in said paper), on linear logic. It’s called ‘A Taste of Linear Logic’ (I actually have the paper open in Safari on my iPhone as I type this) and it is written as a sort of primer on the area. Wadler’s bibliographies on the works contained in the pages you linked are a good source as well. If you are at all coming from a place of familiarity with Haskell, a decent overview of the current discussions on Linear Types in functional languages is at [2]. And lastly, if you like videos, Simon Peyton Jones has a great talk on YouTube [3] where he discusses practical linear types for Haskell, but the theory is not Haskell specific. I will warn you that the latter two references make several arguments and then discuss uniqueness typing (which is related but very different) and that is a whole other rabbit hole of sources, but generally a good topic to dig into as well. [1] https://homepages.inf.ed.ac.uk/wadler/papers/lineartaste/lineartaste-revised.pdf https://homepages.inf.ed.ac.uk/wadler/papers/lineartaste/lin... [2] http://edsko.net/2017/01/08/linearity-in-haskell/ http://edsko.net/2017/01/08/linearity-in-haskell/ [3] https://youtu.be/t0mhvd3-60Y https://youtu.be/t0mhvd3-60Y

peatfreak 7y ago

This is going to sound snobby to some people but I don't care. I really believe this. The whole book is written in Markdown. This is a huge red flag for me and the book goes in to the "no" pile pretty much immediately. Distributed computation is a complex and complicated discipline. Moreover if you're going to write a book and you don't even realize that Markdown is completely inappropriate it calls in to question your authoring skills. At best the use of Markdown is lazy. If you are an expert in distributed computation I assume that you're probably capable of using LaTeX, DocBook, *roff, or some other appropriate technical authoring system.

bruth 7y ago
What does writing in Markdown have to do with the complexity of the topic? Heather Miller is an assistant professor at CMU and well respected and certainly an expert in this context. Clearly she believed Markdown was sufficient for her purpose and/or wanted to leverage the other benefits beyond just its simplicity. Did you see that book is being rendered on a site? http://dist-prog-book.com/chapter/2/futures.html http://dist-prog-book.com/chapter/2/futures.html
- throwqwerty 7y ago
  >What does writing in Markdown have to do with the complexity of the topic? nothing. the person you're responding to just wants to feel very good about themselves without actually engaging with the material.
  - dang 7y ago
    Reacting to a bad comment with an even worse comment is not ok here. It's against the site guidelines, and you've been doing it a lot. We've had to ban you before for that, so would you please stop? If you're posting to HN, you're responsible for helping to take care of this place, to the best of your ability and awareness. If others are wrong, correct them respectfully. If they are ignorant, teach them. If you know more, share some of what you know so the rest of us can learn. Tearing others down helps no one, and worsens the very things you're complaining about. I'm sure you wouldn't litter in a city park just because others do, nor would you fail to put out a campfire just because you were mad at other campers. Please bring those values here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

throwqwerty 7y ago

lol hn never fails in it's shallow dismissal of an enormous amount of work: as of right now there's a comment on how this textbook is too shallow (I guess if it were a content marketing medium post it would be much better) and how it can't possibly be substantive because it's written in markdown. man what a bastion of intelligent discussion this place is.

eternalban 7y ago
It is fair to point out this is a product of a highly regarded computer science school. So the honest question is: should we subject industry e.g. blogs to the same standard as the academy. I would think we should hold academic products to a much higher standard. I do agree criticism should be constructive. (One of PRs for example noted lack of mention of CSP.)
- throwqwerty 7y ago
  >I would think we should hold academic products to a much higher standard who is we? are you a tt prof at literally the number 1 cs school in the US? are you a reviewer for any distributed journals? the author of this book is a wildly successful researcher. why should i trust literally almost anyone's here opinion over their opinion?
- nogabebop23 7y ago
  Maybe when you're assessing the relative quality we should consider this context with more weight, but should that excuse industry for presenting expertise via a light-weight puff piece PR posting?
- marcosdumay 7y ago
  Hum... It does really depend. Honestly, for me right now, coming from HN into random places, no I don't have different standards. But in a course having a textbook recommendation, yes, the standards are very different. Anyway, you are out on a tangent. The work should be judged based on its quality, and for all the dimensions quality has, the presentation format is not one that matters.
7y ago

MUHAMMADRIZWAN 7y ago

https://bit.ly/2QTW8St https://bit.ly/2QTW8St

troelsSteegin 7y ago

The menu of papers for the associated class, https://heather.miller.am/teaching/cs7680/ https://heather.miller.am/teaching/cs7680/, looks really rich. The class is from 2016. Looks like the book dates from 2016-2017. My first impression was that this was work in progress ... I'd be curious to see what from this set Adrian Colyer https://blog.acolyer.org/ https://blog.acolyer.org/ has already digested. A perhaps interesting follow on would be to see which of these programming models are manifest today across the big cloud services .. or how they have advanced from this legacy work.

zzzcpan 7y ago

For those who didn't figure it out - instead of raw markdown you can read the book already in html format that browsers can render: http://dist-prog-book.com/chapter/ http://dist-prog-book.com/chapter/

macintux 7y ago

I haven’t checked these links for a couple of years, but here’s a collection of distributed systems reading lists I compiled. https://gist.github.com/macintux/6227368 https://gist.github.com/macintux/6227368