8 ms·

Technical Papers Every Programmer Should Read at Least Twice (2011)

366 points by altern8 12y ago

deleted 12y ago
[deleted]
ashish01 12y ago
Can some please do a similar list for machine learning and also for maths relevant to ML?
- jcr 12y ago
  Machine Learning is a fast moving field, and many papers are incremental improvements, so an always up-to-date list will be unlikely. None the less, there are quite a few lists out there if you look. Here are some I've seen posted recently: https://news.ycombinator.com/item?id=8728112 https://news.ycombinator.com/item?id=8728112 http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Recommended_Readings http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Recomm... http://www.cs.toronto.edu/~hinton/deeprefs.html http://www.cs.toronto.edu/~hinton/deeprefs.html http://arxiv.org/abs/1404.7828 http://arxiv.org/abs/1404.7828
- wbsun 12y ago
  The OP already did this for ML too. Read the title again, "every programmer", don't you think people doing ML are programmers? :-D
- stillsut 12y ago
  Probably the most famous papers is Breimen's "The Two Cultures". The rejoinder's from Cox and Hoadley add two additional perspectives from long careers.
  - melchebo 12y ago
    Breiman, btw. Link to paper: http://projecteuclid.org/euclid.ss/1009213726 http://projecteuclid.org/euclid.ss/1009213726
platz 12y ago
There is very little all programmers should be required to have in common. The field is just that big now. 9 times out of 10 a list like this includes a treatise on floating point number representation, which while useful, probably isn't of utmost importance in the 21st century, but hey, at one time folks thought that was required for 'all programmers' to read. At least this list does seem more up to date and relevant. I just wish we'd stop with 'all X should' titles. Its demeaning and inaccurate.
- recursive 12y ago
  Floating point behavior still represents a significant portion of new questions asked on stack overflow. I'd say it's still very relevant.
  - theoh 12y ago
    Considering JavaScript makes it difficult to avoid, that's not surprising... And since js is probably one of the common denominators of much activity on the fringes of programming it seems like floating point should be widely discussed. Unrelated to numeric computing, my favourite thing would be Butler Lampson's Hints for Computer System Design.
    - grobinson 12y ago
      I've worked with code in the past where the previous developers didn't understand that there was a difference between an integer in a string (i.e. var x = '10';) and an integer (i.e. var x = 10; ). In some ways weak typing and automatic type conversion is quite dangerous because it can encourage beginners to adopt bad practice and be non the wiser. Edit: Changed 'In some ways duck typing' to 'In some ways weak typing' as rightly pointed out below.
      - klibertp 12y ago
        > duck typing and automatic type conversion It's weak typing (or equivalently "automatic type conversion" as you say) that is to blame. "Duck typing" - also "structural polymorphism", "subtyping polymorphism", know in OCaml as "polymorphic variants" and adopted as a default by Go's interfaces semantics - would only enable "'10' + 10" if String type would have a method "+(x:Number)", which most likely is not there, or if it is it's explicitly laid out. It's weak typing - ie. language trying to coerce elements of expressions to the types which make the whole expressions make sense, with implicit rules that no one ever reads - that's dangerous. Sorry for being this pedantic, but while I'd like to see less weak typing in languages (outside of some specific domains, like AWK) I'd also would like to see more "duck typing", because it's more convenient than nominative polymorphism and exactly as safe.
notacoward 12y ago
Eight out of ten are about programming languages, and strong on the functional side to boot. It's not that these topics aren't important, or that they're not great papers, but isn't that a bit too heavily skewed toward one area? Shouldn't at least one of those top ten be more directly about security, or performance, or some other kind of idea rather than the notation we use to express ideas? Yeah, I know, make your own list. Maybe I will. Nonetheless, the author specifically mentions "cover a wide-range of topics" as a goal and this list fails to meet that goal.
- frozenport 12y ago
  Perhaps these 10 are among the greater set of papers that every programmer should read. I will shamelessly plug the IRLS paper on l0 optimization: http://onlinelibrary.wiley.com/doi/10.1002/cpa.20303/abstract http://onlinelibrary.wiley.com/doi/10.1002/cpa.20303/abstrac...
  - DyslexicAtheist 12y ago
    might want to check http://blog.valbonne-consulting.com/2014/06/09/an-incomplete-list-of-classic-papers-every-software-architect-should-read/ http://blog.valbonne-consulting.com/2014/06/09/an-incomplete...
- dj-wonk 12y ago
  Yes, there is an emphasis (and bias!) towards functional languages and distributed systems. Yes, Fogus' intent is to raise awareness of these concepts. Yes, other people would have different lists. Yes, the list could have more breadth. Personally, I'd like to see a concept map of essential computer science topics combined with some of the top papers and books that cover each. It could be implemented as a curated collapsable directed graph.
  - tekacs 12y ago
    > Personally, I'd like to see a concept map of essential computer science topics combined with some of the top papers and books that cover each. It could be implemented as a curated collapsable directed graph. I've tried building one of these before, both for CS and for Maths. Funnily enough formatting, displaying and making it accessible (whilst retaining enough data) was the time-consuming issue, despite all the tech around for handling graphs. :/
jvreeland 12y ago
It's weird that "What every programmer should know about memory" Isn't on here. Even for languages that manage memory for you understanding the hard limitations and basic operations used to access and manipulate memory is certainly useful.
- Matumio 12y ago
  This is an excellent read, but I disagree with the title. It's a must-read for people interested in CPU architecture and for programmers who, after profiling, still consider doing micro-optimizations. E.g. if you seriously think about reordering your struct members for faster access. If you're building a web service, skip it.
  - EdwardCoffin 12y ago
    I think it is a useful read for people doing any kind of programming at all, in that it might discourage them from attempting optimizations that seem obviously good, and might have _been_ good, if they were running on 20 year-old architectures, but these days just make the code more convoluted to no gain. I know that this is the kind of effect the paper had on me (I spent a lot of time carefully reading it shortly after it was published), and now I almost never use what knowledge I gained from it to make performance optimizations, but it does frequently discourage me from attempting optimizations that I now know would probably not have the desired effect.
KevinEldon 12y ago
It's interesting to read the HN comments on this post now and what others have said previously. https://news.ycombinator.com/item?id=3382962 https://news.ycombinator.com/item?id=3382962 https://news.ycombinator.com/item?id=2979458 https://news.ycombinator.com/item?id=2979458
- waterlesscloud 12y ago
  Wow. That really is a quite revealing comparison.
  - Semiapies 12y ago
    Part of it might be that this was dropped on a Friday night, US time.
    - 3rd3 12y ago
      There is also the fact that the first condensation seeds of discussions on HN are largely determined by chance.
tormeh 12y ago
Nope, not going to read papers. Scientific papers are full of scientese and I'm allergic. I encountered one recently that used "ambulate" instead of "walk". It is simply insufferable. It needs to stop. Scientific papers aren't good at communicating in any way - they exist primarily as an intellectual masturbation device where the authors try to max out verbosity*vocabulary. The only reason anyone reads them is that we do not have anything better than this filth. Yes, I am angry.
dvanduzer 12y ago
Why this Hoare paper and this Lamport paper? A list of ten is a bit long considering how much background material is required reading for every single entry.
frakkingcylons 12y ago
I think Communicating Sequential Processes [0] by Hoare is another landmark paper that should be on this list for its perspective on organizing concurrent processes. This was actually required reading for the concurrency section of my undergrad operating systems course. [0]: https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf
- dvanduzer 12y ago
  "The practice of supplying proofs for nontrivial programs will not become widespread until considerably more powerful proof techniques become available, and even then will not be easy. But the practical advantages of program proving will eventually outweigh the difficulties, in view of the increasing costs of programming error." The Hoare paper actually linked is hopelessly outdated, despite being an interesting historical artifact.
  - ploxiln 12y ago
    I might call it "sadly" outdated... not that I'm a fan of provably-correct programming (I've never done it, I just fix systems and networking software), but the seemingly modern comfort with buggyness and sloppy implementation... funny or sad, who knows
jaryd 12y ago
I would recommend this one for historical purposes: http://insecure.org/stf/smashstack.html http://insecure.org/stf/smashstack.html
- grobinson 12y ago
  This is an excellent paper but it does require a prior understanding of the stack, addressing, buffers, pointers, etc. It would be really nice if there was an article or blog post designed for people who want to read this paper but don't have the prior knowledge that's required to understand it. Unfortuntely I don't know of any. HN?
hotgoldminer 12y ago
Has anyone else explored the rest of his site? Good posts, workable, no fuss design. All good work here. Keep it up!
- jcr 12y ago
  fogus is a well known HN contributor. https://news.ycombinator.com/user?id=fogus https://news.ycombinator.com/user?id=fogus
angry_octet 12y ago
I love that even HN has listicles, and I would rate this one as at least on par with http://www.lifehack.org/articles/lifestyle/21-things-you-are-doing-wrong-every-day.html http://www.lifehack.org/articles/lifestyle/21-things-you-are...
Yadi 12y ago
#oly $#it! I have read these papers, all of'm! WHAT! Haha that is so accurate LOL! Great stuff! But there are way more important whitepapers to be honest. I can't really think of the others right now, but if you go to the Digital library from ACM / IEEE, you can find really good stuff.
jpitz 12y ago
tormeh said: Nope, not going to read papers. Scientific papers are full of scientese and I'm allergic. And got downvoted to hell for it. Which is too bad, because its worth discussing. There _is_ a lot of bad writing and intellectual masturbation in academic publications. There is also a lot of good writing. In particular, I find Lamport's writing to be prosaic and easy to follow.
bdamm 12y ago
Any list without Shannon's 1948 "A Mathematical Theory of Communication" is just not a good list. Sorry. http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pd... The foundation of information theory. It is, by far, the most astonishing paper I have ever read. Far more astonishing than Lamport's famous conclusion about clocks. It is the kind of paper that causes a soul rift when read thoroughly.
- anjbe 12y ago
  I’m partial to his Master’s thesis, “A Symbolic Analysis of Relay and Switching Circuits.” It literally invented digital computing. As an electrical engineering student it blows me away that one guy’s thesis (he was 21!) can be fundamental to so much.
kashif 12y ago
Most seem to be available here - https://github.com/papers-we-love/papers-we-love https://github.com/papers-we-love/papers-we-love
picardo 12y ago
I had a dream about one of these papers tonight (seriously), so I come to HN and find this post. Pretty amazing coincidence. :)
jobenex 12y ago
Agree with platz

366 points by altern8 12y ago

deleted 12y ago

[deleted]

ashish01 12y ago

Can some please do a similar list for machine learning and also for maths relevant to ML?

jcr 12y ago
Machine Learning is a fast moving field, and many papers are incremental improvements, so an always up-to-date list will be unlikely. None the less, there are quite a few lists out there if you look. Here are some I've seen posted recently: https://news.ycombinator.com/item?id=8728112 https://news.ycombinator.com/item?id=8728112 http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Recommended_Readings http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Recomm... http://www.cs.toronto.edu/~hinton/deeprefs.html http://www.cs.toronto.edu/~hinton/deeprefs.html http://arxiv.org/abs/1404.7828 http://arxiv.org/abs/1404.7828
wbsun 12y ago
The OP already did this for ML too. Read the title again, "every programmer", don't you think people doing ML are programmers? :-D
stillsut 12y ago
Probably the most famous papers is Breimen's "The Two Cultures". The rejoinder's from Cox and Hoadley add two additional perspectives from long careers.
- melchebo 12y ago
  Breiman, btw. Link to paper: http://projecteuclid.org/euclid.ss/1009213726 http://projecteuclid.org/euclid.ss/1009213726

platz 12y ago

There is very little all programmers should be required to have in common. The field is just that big now. 9 times out of 10 a list like this includes a treatise on floating point number representation, which while useful, probably isn't of utmost importance in the 21st century, but hey, at one time folks thought that was required for 'all programmers' to read. At least this list does seem more up to date and relevant. I just wish we'd stop with 'all X should' titles. Its demeaning and inaccurate.

recursive 12y ago
Floating point behavior still represents a significant portion of new questions asked on stack overflow. I'd say it's still very relevant.
- theoh 12y ago
  Considering JavaScript makes it difficult to avoid, that's not surprising... And since js is probably one of the common denominators of much activity on the fringes of programming it seems like floating point should be widely discussed. Unrelated to numeric computing, my favourite thing would be Butler Lampson's Hints for Computer System Design.
  - grobinson 12y ago
    I've worked with code in the past where the previous developers didn't understand that there was a difference between an integer in a string (i.e. var x = '10';) and an integer (i.e. var x = 10; ). In some ways weak typing and automatic type conversion is quite dangerous because it can encourage beginners to adopt bad practice and be non the wiser. Edit: Changed 'In some ways duck typing' to 'In some ways weak typing' as rightly pointed out below.
    - klibertp 12y ago
      > duck typing and automatic type conversion It's weak typing (or equivalently "automatic type conversion" as you say) that is to blame. "Duck typing" - also "structural polymorphism", "subtyping polymorphism", know in OCaml as "polymorphic variants" and adopted as a default by Go's interfaces semantics - would only enable "'10' + 10" if String type would have a method "+(x:Number)", which most likely is not there, or if it is it's explicitly laid out. It's weak typing - ie. language trying to coerce elements of expressions to the types which make the whole expressions make sense, with implicit rules that no one ever reads - that's dangerous. Sorry for being this pedantic, but while I'd like to see less weak typing in languages (outside of some specific domains, like AWK) I'd also would like to see more "duck typing", because it's more convenient than nominative polymorphism and exactly as safe.

notacoward 12y ago

Eight out of ten are about programming languages, and strong on the functional side to boot. It's not that these topics aren't important, or that they're not great papers, but isn't that a bit too heavily skewed toward one area? Shouldn't at least one of those top ten be more directly about security, or performance, or some other kind of idea rather than the notation we use to express ideas? Yeah, I know, make your own list. Maybe I will. Nonetheless, the author specifically mentions "cover a wide-range of topics" as a goal and this list fails to meet that goal.

frozenport 12y ago
Perhaps these 10 are among the greater set of papers that every programmer should read. I will shamelessly plug the IRLS paper on l0 optimization: http://onlinelibrary.wiley.com/doi/10.1002/cpa.20303/abstract http://onlinelibrary.wiley.com/doi/10.1002/cpa.20303/abstrac...
- DyslexicAtheist 12y ago
  might want to check http://blog.valbonne-consulting.com/2014/06/09/an-incomplete-list-of-classic-papers-every-software-architect-should-read/ http://blog.valbonne-consulting.com/2014/06/09/an-incomplete...
dj-wonk 12y ago
Yes, there is an emphasis (and bias!) towards functional languages and distributed systems. Yes, Fogus' intent is to raise awareness of these concepts. Yes, other people would have different lists. Yes, the list could have more breadth. Personally, I'd like to see a concept map of essential computer science topics combined with some of the top papers and books that cover each. It could be implemented as a curated collapsable directed graph.
- tekacs 12y ago
  > Personally, I'd like to see a concept map of essential computer science topics combined with some of the top papers and books that cover each. It could be implemented as a curated collapsable directed graph. I've tried building one of these before, both for CS and for Maths. Funnily enough formatting, displaying and making it accessible (whilst retaining enough data) was the time-consuming issue, despite all the tech around for handling graphs. :/

jvreeland 12y ago

It's weird that "What every programmer should know about memory" Isn't on here. Even for languages that manage memory for you understanding the hard limitations and basic operations used to access and manipulate memory is certainly useful.

Matumio 12y ago
This is an excellent read, but I disagree with the title. It's a must-read for people interested in CPU architecture and for programmers who, after profiling, still consider doing micro-optimizations. E.g. if you seriously think about reordering your struct members for faster access. If you're building a web service, skip it.
- EdwardCoffin 12y ago
  I think it is a useful read for people doing any kind of programming at all, in that it might discourage them from attempting optimizations that seem obviously good, and might have _been_ good, if they were running on 20 year-old architectures, but these days just make the code more convoluted to no gain. I know that this is the kind of effect the paper had on me (I spent a lot of time carefully reading it shortly after it was published), and now I almost never use what knowledge I gained from it to make performance optimizations, but it does frequently discourage me from attempting optimizations that I now know would probably not have the desired effect.

KevinEldon 12y ago

It's interesting to read the HN comments on this post now and what others have said previously. https://news.ycombinator.com/item?id=3382962 https://news.ycombinator.com/item?id=3382962 https://news.ycombinator.com/item?id=2979458 https://news.ycombinator.com/item?id=2979458

waterlesscloud 12y ago
Wow. That really is a quite revealing comparison.
- Semiapies 12y ago
  Part of it might be that this was dropped on a Friday night, US time.
  - 3rd3 12y ago
    There is also the fact that the first condensation seeds of discussions on HN are largely determined by chance.

tormeh 12y ago

Nope, not going to read papers. Scientific papers are full of scientese and I'm allergic. I encountered one recently that used "ambulate" instead of "walk". It is simply insufferable. It needs to stop. Scientific papers aren't good at communicating in any way - they exist primarily as an intellectual masturbation device where the authors try to max out verbosity*vocabulary. The only reason anyone reads them is that we do not have anything better than this filth. Yes, I am angry.

dvanduzer 12y ago

Why this Hoare paper and this Lamport paper? A list of ten is a bit long considering how much background material is required reading for every single entry.

frakkingcylons 12y ago

I think Communicating Sequential Processes [0] by Hoare is another landmark paper that should be on this list for its perspective on organizing concurrent processes. This was actually required reading for the concurrency section of my undergrad operating systems course. [0]: https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf

dvanduzer 12y ago
"The practice of supplying proofs for nontrivial programs will not become widespread until considerably more powerful proof techniques become available, and even then will not be easy. But the practical advantages of program proving will eventually outweigh the difficulties, in view of the increasing costs of programming error." The Hoare paper actually linked is hopelessly outdated, despite being an interesting historical artifact.
- ploxiln 12y ago
  I might call it "sadly" outdated... not that I'm a fan of provably-correct programming (I've never done it, I just fix systems and networking software), but the seemingly modern comfort with buggyness and sloppy implementation... funny or sad, who knows

jaryd 12y ago

I would recommend this one for historical purposes: http://insecure.org/stf/smashstack.html http://insecure.org/stf/smashstack.html

grobinson 12y ago
This is an excellent paper but it does require a prior understanding of the stack, addressing, buffers, pointers, etc. It would be really nice if there was an article or blog post designed for people who want to read this paper but don't have the prior knowledge that's required to understand it. Unfortuntely I don't know of any. HN?

hotgoldminer 12y ago

Has anyone else explored the rest of his site? Good posts, workable, no fuss design. All good work here. Keep it up!

jcr 12y ago
fogus is a well known HN contributor. https://news.ycombinator.com/user?id=fogus https://news.ycombinator.com/user?id=fogus

angry_octet 12y ago

I love that even HN has listicles, and I would rate this one as at least on par with http://www.lifehack.org/articles/lifestyle/21-things-you-are-doing-wrong-every-day.html http://www.lifehack.org/articles/lifestyle/21-things-you-are...

Yadi 12y ago

#oly $#it! I have read these papers, all of'm! WHAT! Haha that is so accurate LOL! Great stuff! But there are way more important whitepapers to be honest. I can't really think of the others right now, but if you go to the Digital library from ACM / IEEE, you can find really good stuff.

jpitz 12y ago

tormeh said: Nope, not going to read papers. Scientific papers are full of scientese and I'm allergic. And got downvoted to hell for it. Which is too bad, because its worth discussing. There _is_ a lot of bad writing and intellectual masturbation in academic publications. There is also a lot of good writing. In particular, I find Lamport's writing to be prosaic and easy to follow.

bdamm 12y ago

Any list without Shannon's 1948 "A Mathematical Theory of Communication" is just not a good list. Sorry. http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pd... The foundation of information theory. It is, by far, the most astonishing paper I have ever read. Far more astonishing than Lamport's famous conclusion about clocks. It is the kind of paper that causes a soul rift when read thoroughly.

anjbe 12y ago
I’m partial to his Master’s thesis, “A Symbolic Analysis of Relay and Switching Circuits.” It literally invented digital computing. As an electrical engineering student it blows me away that one guy’s thesis (he was 21!) can be fundamental to so much.

kashif 12y ago

Most seem to be available here - https://github.com/papers-we-love/papers-we-love https://github.com/papers-we-love/papers-we-love

picardo 12y ago

I had a dream about one of these papers tonight (seriously), so I come to HN and find this post. Pretty amazing coincidence. :)

jobenex 12y ago

Agree with platz