7 ms·

How to Read a Paper [pdf]

246 points by sherilm 3y ago

lubitelpospat 3y ago
Is there a DOI for this paper? Couldn’t find it
- Jtsummers 3y ago
  It's an ACM publication, when in doubt search their digital library: https://dl.acm.org/doi/10.1145/1273445.1273458 https://dl.acm.org/doi/10.1145/1273445.1273458 is their entry for it with the following: https://doi.org/10.1145/1273445.1273458 https://doi.org/10.1145/1273445.1273458
  - lubitelpospat 3y ago
    Thank you very much!
hprotagonist 3y ago
I absolutely love this paper. Its counterpart is Kording and Mensh, “Ten Simple Rules for Structuring Papers.” doi.org/10.1101/088278 And the strategies outlined in both ought to be taught much more rigorously, and at an earlier stage in an education, than they currently are.
- Jtsummers 3y ago
  https://www.biorxiv.org/content/10.1101/088278v5 https://www.biorxiv.org/content/10.1101/088278v5
nittanymount 3y ago
guess this article/approach is still useful/helpful even AI could help do a good summary... thanks for sharing...
- nittanymount 3y ago
  someone removed 1 karma for my comment above, could you explain why? thanks. haha
  - horsellama 3y ago
    If the paper is well written, the abstract is already the summary you are after. If you are then interested in the details, gpt/AI isn't helping there.
- cratermoon 3y ago
  IMNSHO, asking AI to summarize a paper is a flavor of what Umberto Eco terms the "alibi of photocopies", where he says, "There are many things that I do not know because I photocopied a text and then relaxed as if I had read it." Today I would write "There are many things that I do not know because I asked an LLM to summarize a text and then relaxed as if I had read it."
kvrck 3y ago
That's a long list of supporting organisations/institutions for a somewhat trivial paper.
- daveguy 3y ago
  I think the lesson from this is reading papers well can help your funding ability. Because it's such a general paper the author felt they should include all funding and support organizations, not necessarily those that specifically paid for this paper. Better to over-acknowledge than under-acknowledge.
r-zip 3y ago
I think the time estimates depend heavily on the field. A mathematician recently told him that it would take him 1-2 weeks (with nothing else to do in that time) to digest a paper outside his main area.
- daveguy 3y ago
  That makes sense. The proof-heavy part is the third pass and part the author says takes the longest (4-5 hours for a beginner). With math papers it's essentially all proof!
- penguin_booze 3y ago
  Makes me wonder: should it, though? The author of the paper presumably had gone through the trouble of understanding everything described in the paper, built intuition and a mental model. Instead of putting more effort in writing the latter down, it appears to me that authors are inclined to throw the proverbial baby out with the bath water: they spend more time writing out the dense proof rather than an exposition. I'm not saying proof isn't important or to exclude it. But given that more people understand plain and intuitive explanation (at the expense of accuracy, maybe), their hard work reaches broader audience that way. Isn't that what authors want, instead of "dog whistling"? Do proofs alone carry intuition? I don't think so.
  - abdullahkhalids 3y ago
    It takes many dozens of hours, sometimes hundreds, to write a paper (just the writing not the technical work). People will spend a long time trying to improve the expositions. I have seen papers where co-authors have fought for weeks about the accuracy of a single sentence. I have seen papers where there were over twenty draft-revision cycles. But there are natural limits. Usually, after working for years on a problem, you become so close to it that describing to a general technical audience is very difficult. Often, after you publish the work, someone else will do the difficult work of understanding your paper, and then write a more readable exposition as part of a review paper or book.
apokryptein 3y ago
Interesting paper. Consequently, a similar approach is outlined in "How to Read a Book" by Mortimer J. Adler, which covers reading approaches for a variety of formats.
ssn 3y ago
"The first version of this document was drafted by my students: …" So why aren't they listed as co-authors? :-)
- deleted 3y ago
  [deleted]
- disqard 3y ago
  They're in the "Acknowledgements" section, which is appropriate for this particular scheme of things. Academia is nothing if not a "street cred" game, and that cred is not portioned out easily (or equitably).
jll29 3y ago
I give all my doctoral students a copy of the following great paper (and I've used a variant of the check list at the end for years - avoids errors when working on multiple papers with multiple international teams in parallel): How to Write a Paper http://www-mech.eng.cam.ac.uk/mmd/ashby-paper-V6.pdf http://www-mech.eng.cam.ac.uk/mmd/ashby-paper-V6.pdf
- cratermoon 3y ago
  Umberto Eco wrote a little (229p) volume titled How to Write a Thesis in 1977. In the introduction to the revised 1985 edition, Eco describes the book as one that "focuses on the spirit, mentality, and research methods required to write a good thesis, rather than on its content". It's a bit longer than Ashby's paper, and some of the internet age may look disdainfully on his advice for index cards, paper notes, and photocopies, but the intent behind them and the general method translates easily to the digital age, and thankfully we have tools like Zotero to drastically reduce some of the busywork.
- beryilma 3y ago
  The moment I saw Figures 3 and 4, I knew this paper would not be useful for me at all. The author of the paper is likely a visual thinker type of person, whereas I am certainly not. The boxes, colors, shading, etc. on those figures would/does easily overwhelm me. I find simply writing an outline, section titles and introductory paragraphs, and iterating over and over much more suitable to get started with writing a paper.
- Beijinger 3y ago
  This may get downvoted, but regarding to writing a thesis: If I open a thesis and see it was done in Latex I always think this guy can't be an idiot.
  - maleldil 3y ago
    That's an interesting prior. Can you expand on it? If it's because of managing latex's complexity indicating some kind of ability, I can guarantee you that most people just copy paste and tweak stuff till it works.
svilen_dobrev 3y ago
interesting. Are there such papers - that one can check the proofs and references and etc - on IT/computing related stuff? Most papers i have stumbled upon, back in time, have been some mumbo-jumbo showing-off this or that approach or achievement, but rarely coherent or consistent (one example talks X, another talks Y, with no link inbetween). Hence i stopped reading any such papers, years ago..
- cratermoon 3y ago
  I recommend visiting "the morning paper: a random walk through Computer Science research, by Adrian Colyer". It ran from 2014 to February of 2021, covering a paper per week or so. You can read Colyer's analysis first to determine if reading the paper sounds worth your time. There are also various lists of key programming/CS papers. For example, "What Every Computer Scientist Should Know About Floating-Point Arithmetic", David Goldberg's 1990 paper, includes various theorem proofs essential to understanding how IEEE floating point is defined.
  - cratermoon 3y ago
    Forgot the link: https://blog.acolyer.org/archives/ https://blog.acolyer.org/archives/
cratermoon 3y ago
The 3-pass method seems to roughly correspond the Adler's[1] technique for reading a book. The first pass is the systematic skimming of the inspectional level, the second pass corresponds to the analytical level, and the final pass of evaluating the arguments and ideas corresponds to his syntopical level. 1 Adler, Mortimer Jerome, and Charles Van Doren. 1972. How to Read a Book. Rev. and updated ed. New York: Simon and Schuster.
cubefox 3y ago
I don't really agree with such advice. The only important thing is to read (or skim) many papers in the relevant field. Initially they will be hard to understand as they are full of difficult jargon. But after a while, you will notice the jargon that repeats across articles, be able to look it up, and ultimately understand more and more. After a while it will be fairly easy. But there is no general "paper reading ability" one could learn. Papers outside your field of expertise will always be hard to understand as long as you are not used to the relevant field, the jargon and style.
- linuxdude314 3y ago
  In the case of science and math words are more than just jargon, they have precise meaning. Often jargon contains ambiguity but you can infer a general meaning. With the language of math and science you actually need to read the precise definition of a term to understand its meaning.
niam 3y ago
It's reassuring since, having not habitually read papers, I've always wondered how (in)efficiently I read compared to readers of a professional sort.
dmillar 3y ago
I used to write research at Goldman. Our chief economist would often give this book as an end-of-year gift. I found it useful in my writing: https://press.uchicago.edu/ucp/books/book/chicago/E/bo25674588.html https://press.uchicago.edu/ucp/books/book/chicago/E/bo256745...
- mattmmatthews 3y ago
  I'm not interested in the economy, explicitly. Would you recommend this for someone who is geared towards technical writing and just getting better at it in general? I've never thought about reading a book about how to write better, to be totally honest, but how could it hurt?
  - dmillar 3y ago
    Yes, it's short and insightful for all kinds of writers--technical and otherwise. The editions subsequent to the first have decreased emphasis on economics and instead geared it toward academic-style writing.
trostaft 3y ago
My advisor gave me this paper at the beginning of my PhD; useful advice especially when google scholar dumps heaps of papers onto you every so often. Though I'd largely ignore the time estimates. That varies deeply depending on the paper and field.
mad44 3y ago
How I read a research paper https://muratbuffalo.blogspot.com/2013/07/how-i-read-research-paper.html https://muratbuffalo.blogspot.com/2013/07/how-i-read-researc...
NoToP 3y ago
But how do you read "How to Read a Paper"?
- anon115 3y ago
  Meow meow meow
sirpilade 3y ago
This is the updated (2016) version https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-reading.pdf https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-re...
cygnion 3y ago
Thanks for the nice references. When I read documents, I look to organize and link information in a way that helps me recall its context. Another realization is that papers are often read with different 'hats' - as a reader, as a reviewer, or as a writer. To help my own process I built a document-reading that helps me curate, visualize, and recall personal knowledge as I read and annotate research papers; the app also extracts data from documents such as URLs and references - https://www.knowledgegarden.io https://www.knowledgegarden.io
exo-pla-net 3y ago
This is the 2007 version. The latest 2016 version can be read here: https://web.archive.org/web/20220615001635/http://blizzard.cs.uwaterloo.ca/keshav/home/Papers/data/07/paper-reading.pdf https://web.archive.org/web/20220615001635/http://blizzard.c... Or here: https://people.mpi-sws.org/~jcmace/teaching/cds-ss20/paper-reading.pdf https://people.mpi-sws.org/~jcmace/teaching/cds-ss20/paper-r... Or here: https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-reading.pdf https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-re... Here's the current website of Dr. Keshav, the author: https://svr-sk818-web.cl.cam.ac.uk/keshav/wiki/index.php/HTRAP https://svr-sk818-web.cl.cam.ac.uk/keshav/wiki/index.php/HTR... The links in the 2016 paper itself are rotten, but he provides updated links on his website above.

246 points by sherilm 3y ago

lubitelpospat 3y ago

Is there a DOI for this paper? Couldn’t find it

Jtsummers 3y ago
It's an ACM publication, when in doubt search their digital library: https://dl.acm.org/doi/10.1145/1273445.1273458 https://dl.acm.org/doi/10.1145/1273445.1273458 is their entry for it with the following: https://doi.org/10.1145/1273445.1273458 https://doi.org/10.1145/1273445.1273458
- lubitelpospat 3y ago
  Thank you very much!

hprotagonist 3y ago

I absolutely love this paper. Its counterpart is Kording and Mensh, “Ten Simple Rules for Structuring Papers.” doi.org/10.1101/088278 And the strategies outlined in both ought to be taught much more rigorously, and at an earlier stage in an education, than they currently are.

Jtsummers 3y ago
https://www.biorxiv.org/content/10.1101/088278v5 https://www.biorxiv.org/content/10.1101/088278v5

nittanymount 3y ago

guess this article/approach is still useful/helpful even AI could help do a good summary... thanks for sharing...

nittanymount 3y ago
someone removed 1 karma for my comment above, could you explain why? thanks. haha
- horsellama 3y ago
  If the paper is well written, the abstract is already the summary you are after. If you are then interested in the details, gpt/AI isn't helping there.
cratermoon 3y ago
IMNSHO, asking AI to summarize a paper is a flavor of what Umberto Eco terms the "alibi of photocopies", where he says, "There are many things that I do not know because I photocopied a text and then relaxed as if I had read it." Today I would write "There are many things that I do not know because I asked an LLM to summarize a text and then relaxed as if I had read it."

kvrck 3y ago

That's a long list of supporting organisations/institutions for a somewhat trivial paper.

daveguy 3y ago
I think the lesson from this is reading papers well can help your funding ability. Because it's such a general paper the author felt they should include all funding and support organizations, not necessarily those that specifically paid for this paper. Better to over-acknowledge than under-acknowledge.

r-zip 3y ago

I think the time estimates depend heavily on the field. A mathematician recently told him that it would take him 1-2 weeks (with nothing else to do in that time) to digest a paper outside his main area.

daveguy 3y ago
That makes sense. The proof-heavy part is the third pass and part the author says takes the longest (4-5 hours for a beginner). With math papers it's essentially all proof!
penguin_booze 3y ago
Makes me wonder: should it, though? The author of the paper presumably had gone through the trouble of understanding everything described in the paper, built intuition and a mental model. Instead of putting more effort in writing the latter down, it appears to me that authors are inclined to throw the proverbial baby out with the bath water: they spend more time writing out the dense proof rather than an exposition. I'm not saying proof isn't important or to exclude it. But given that more people understand plain and intuitive explanation (at the expense of accuracy, maybe), their hard work reaches broader audience that way. Isn't that what authors want, instead of "dog whistling"? Do proofs alone carry intuition? I don't think so.
- abdullahkhalids 3y ago
  It takes many dozens of hours, sometimes hundreds, to write a paper (just the writing not the technical work). People will spend a long time trying to improve the expositions. I have seen papers where co-authors have fought for weeks about the accuracy of a single sentence. I have seen papers where there were over twenty draft-revision cycles. But there are natural limits. Usually, after working for years on a problem, you become so close to it that describing to a general technical audience is very difficult. Often, after you publish the work, someone else will do the difficult work of understanding your paper, and then write a more readable exposition as part of a review paper or book.

apokryptein 3y ago

Interesting paper. Consequently, a similar approach is outlined in "How to Read a Book" by Mortimer J. Adler, which covers reading approaches for a variety of formats.

ssn 3y ago

"The first version of this document was drafted by my students: …" So why aren't they listed as co-authors? :-)

deleted 3y ago
[deleted]
disqard 3y ago
They're in the "Acknowledgements" section, which is appropriate for this particular scheme of things. Academia is nothing if not a "street cred" game, and that cred is not portioned out easily (or equitably).

jll29 3y ago

I give all my doctoral students a copy of the following great paper (and I've used a variant of the check list at the end for years - avoids errors when working on multiple papers with multiple international teams in parallel): How to Write a Paper http://www-mech.eng.cam.ac.uk/mmd/ashby-paper-V6.pdf http://www-mech.eng.cam.ac.uk/mmd/ashby-paper-V6.pdf

cratermoon 3y ago
Umberto Eco wrote a little (229p) volume titled How to Write a Thesis in 1977. In the introduction to the revised 1985 edition, Eco describes the book as one that "focuses on the spirit, mentality, and research methods required to write a good thesis, rather than on its content". It's a bit longer than Ashby's paper, and some of the internet age may look disdainfully on his advice for index cards, paper notes, and photocopies, but the intent behind them and the general method translates easily to the digital age, and thankfully we have tools like Zotero to drastically reduce some of the busywork.
beryilma 3y ago
The moment I saw Figures 3 and 4, I knew this paper would not be useful for me at all. The author of the paper is likely a visual thinker type of person, whereas I am certainly not. The boxes, colors, shading, etc. on those figures would/does easily overwhelm me. I find simply writing an outline, section titles and introductory paragraphs, and iterating over and over much more suitable to get started with writing a paper.
Beijinger 3y ago
This may get downvoted, but regarding to writing a thesis: If I open a thesis and see it was done in Latex I always think this guy can't be an idiot.
- maleldil 3y ago
  That's an interesting prior. Can you expand on it? If it's because of managing latex's complexity indicating some kind of ability, I can guarantee you that most people just copy paste and tweak stuff till it works.

svilen_dobrev 3y ago

interesting. Are there such papers - that one can check the proofs and references and etc - on IT/computing related stuff? Most papers i have stumbled upon, back in time, have been some mumbo-jumbo showing-off this or that approach or achievement, but rarely coherent or consistent (one example talks X, another talks Y, with no link inbetween). Hence i stopped reading any such papers, years ago..

cratermoon 3y ago
I recommend visiting "the morning paper: a random walk through Computer Science research, by Adrian Colyer". It ran from 2014 to February of 2021, covering a paper per week or so. You can read Colyer's analysis first to determine if reading the paper sounds worth your time. There are also various lists of key programming/CS papers. For example, "What Every Computer Scientist Should Know About Floating-Point Arithmetic", David Goldberg's 1990 paper, includes various theorem proofs essential to understanding how IEEE floating point is defined.
- cratermoon 3y ago
  Forgot the link: https://blog.acolyer.org/archives/ https://blog.acolyer.org/archives/

cratermoon 3y ago

The 3-pass method seems to roughly correspond the Adler's[1] technique for reading a book. The first pass is the systematic skimming of the inspectional level, the second pass corresponds to the analytical level, and the final pass of evaluating the arguments and ideas corresponds to his syntopical level. 1 Adler, Mortimer Jerome, and Charles Van Doren. 1972. How to Read a Book. Rev. and updated ed. New York: Simon and Schuster.

cubefox 3y ago

I don't really agree with such advice. The only important thing is to read (or skim) many papers in the relevant field. Initially they will be hard to understand as they are full of difficult jargon. But after a while, you will notice the jargon that repeats across articles, be able to look it up, and ultimately understand more and more. After a while it will be fairly easy. But there is no general "paper reading ability" one could learn. Papers outside your field of expertise will always be hard to understand as long as you are not used to the relevant field, the jargon and style.

linuxdude314 3y ago
In the case of science and math words are more than just jargon, they have precise meaning. Often jargon contains ambiguity but you can infer a general meaning. With the language of math and science you actually need to read the precise definition of a term to understand its meaning.

niam 3y ago

It's reassuring since, having not habitually read papers, I've always wondered how (in)efficiently I read compared to readers of a professional sort.

dmillar 3y ago

I used to write research at Goldman. Our chief economist would often give this book as an end-of-year gift. I found it useful in my writing: https://press.uchicago.edu/ucp/books/book/chicago/E/bo25674588.html https://press.uchicago.edu/ucp/books/book/chicago/E/bo256745...

mattmmatthews 3y ago
I'm not interested in the economy, explicitly. Would you recommend this for someone who is geared towards technical writing and just getting better at it in general? I've never thought about reading a book about how to write better, to be totally honest, but how could it hurt?
- dmillar 3y ago
  Yes, it's short and insightful for all kinds of writers--technical and otherwise. The editions subsequent to the first have decreased emphasis on economics and instead geared it toward academic-style writing.

trostaft 3y ago

My advisor gave me this paper at the beginning of my PhD; useful advice especially when google scholar dumps heaps of papers onto you every so often. Though I'd largely ignore the time estimates. That varies deeply depending on the paper and field.

mad44 3y ago

How I read a research paper https://muratbuffalo.blogspot.com/2013/07/how-i-read-research-paper.html https://muratbuffalo.blogspot.com/2013/07/how-i-read-researc...

NoToP 3y ago

But how do you read "How to Read a Paper"?

anon115 3y ago
Meow meow meow

sirpilade 3y ago

This is the updated (2016) version https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-reading.pdf https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-re...

cygnion 3y ago

Thanks for the nice references. When I read documents, I look to organize and link information in a way that helps me recall its context. Another realization is that papers are often read with different 'hats' - as a reader, as a reviewer, or as a writer. To help my own process I built a document-reading that helps me curate, visualize, and recall personal knowledge as I read and annotate research papers; the app also extracts data from documents such as URLs and references - https://www.knowledgegarden.io https://www.knowledgegarden.io

exo-pla-net 3y ago

This is the 2007 version. The latest 2016 version can be read here: https://web.archive.org/web/20220615001635/http://blizzard.cs.uwaterloo.ca/keshav/home/Papers/data/07/paper-reading.pdf https://web.archive.org/web/20220615001635/http://blizzard.c... Or here: https://people.mpi-sws.org/~jcmace/teaching/cds-ss20/paper-reading.pdf https://people.mpi-sws.org/~jcmace/teaching/cds-ss20/paper-r... Or here: https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-reading.pdf https://svr-sk818-web.cl.cam.ac.uk/keshav/papers/07/paper-re... Here's the current website of Dr. Keshav, the author: https://svr-sk818-web.cl.cam.ac.uk/keshav/wiki/index.php/HTRAP https://svr-sk818-web.cl.cam.ac.uk/keshav/wiki/index.php/HTR... The links in the 2016 paper itself are rotten, but he provides updated links on his website above.