5 ms·

I have spent a large portion of my adult life finding and reading and understanding scientific papers. The advice offered in the tweet is good, so I won't resta

by sarahj 12y ago

I have spent a large portion of my adult life finding and reading and understanding scientific papers. The advice offered in the tweet is good, so I won't restate it - but I will offer a couple of my own: * Print the paper out and make notes on it by hand (this has a small mention in one of the articles) - this technique got me through university, I tried every software package imaginable but in the end having a hard copy forced me to deal with sheer amount of reading I had to do. It also allowed me to freely make notes, add sticky notes etc. Since then I have done this with any collection of papers I want to get my head around. * Try to comprehend something on every read through - it doesn't have to be big, just something - whether it is the sample makeup or part of the methodology or the conclusions etc. You don't need to understand these in any particular order, but ensure to revise your understanding as you become more familiar with the paper. * Most papers are useless (to you at the time you are reading them) - the sad truth about research is that 90% of the stuff you devote yourself to understanding will be wrong, outdated or not useful to what you are working on. It is very hard to pick out useful papers with nothing to go on but titles, and abstract and citations. As you get more used to the field it becomes easier and familiar names, authors and institutions can guide you, but even close to a decade after reading my first paper I still probably only manage a 10% hit rate when conducting research (but hey - 10% of a lot of papers is still a lot of papers!)

a_e_k 12y ago
I'll second your first point about hard copies. It's one thing to skim a PDF on the screen to see if it might be worth my time, but if I really want to grok a paper I'm marking up a hardcopy. This has also made me appreciative of authors who put in the effort to make sure that their graphs and diagrams reduce well to grayscale. Instead of referring to the red line versus the green line, they'll use labels, marks, different cross hatch types, etc. (Don't get me wrong, good color is still very nice to have too!) It's also led me to making it a point to print off drafts of my own submissions on a B&W laser printer and make sure that my figures and captions are still understandable. I expect that this makes it easier for any colorblind readers as well, whatever type of colorblindness they may have.
mkl 12y ago
> Try to comprehend something on every read through - it doesn't have to be big, just something - whether it is the sample makeup or part of the methodology or the conclusions etc. You don't need to understand these in any particular order, but ensure to revise your understanding as you become more familiar with the paper. Lately I've been reading "How to Read a Book" by Adler and van Doren [1], which systematises this. It applies to papers just as well as books. Get the 1972 edition, as the 1940 edition is quite a bit less readable. [1] https://en.wikipedia.org/wiki/How_to_Read_a_Book https://en.wikipedia.org/wiki/How_to_Read_a_Book
jwr 12y ago
> * Most papers are useless (to you at the time you are reading them) - the sad truth about research is that 90% of the stuff you devote yourself to understanding will be wrong, outdated or not useful to what you are working on. This is very important advice. Do not assume that just because it's a published research paper it is valuable, correct, or useful. In fact, especially in applied CS, I found that authors will sometimes make what looks like an intentional effort to obfuscate the methods so that the paper is publishable (peer reviewers will not try to reproduce the results anyway), but the methods are not implementable, or at least not easily. This makes sense when you think about the competition in academia, authors doing consulting work for companies, or intending to start businesses of their own.
- seanmcdirmid 12y ago
  Usually the idea is bunk and they are trying to obscure that, more often than that the idea is great and they are trying to protect their IP for a startup. If you know something is possible, then it is not impossible to replicate results even with obscure directions about it. Except for Paxos, you really need good instructions at that point (that guy should get an award).
  - deleted 12y ago
    [deleted]
  - striking 12y ago
    Hah. I'm pretty dumb myself. That'll teach me. (I'm leaving the link, though. Great paper.) [1]: http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html#lamport-paxos http://research.microsoft.com/en-us/um/people/lamport/pubs/p...
    - tokenrove 12y ago
      Leslie Lamport is a man. You could argue something about Barbara Liskov's Viewstamped Replication, though, I guess.
      - 12y ago
seanmcdirmid 12y ago
> * Most papers are useless (to you at the time you are reading them) I would say that most papers are worthless. The publish or perish mentality in academia leads to more noise and less signal. You really have to fail fast on papers, find the seminal ones and the few under looked gems related to your topic that can hopefully be fished out via Google (though word of mouth is important). Also, if you publish, try not to be part of the problem.
- personlurking 12y ago
  > The publish or perish mentality in academia leads to more noise and less signal. Perhaps a Google-like ranking would help. The more cited a paper is (in the bibliography), the more it might not be noise. Then again, I don't like the idea that what is popular is also the best.
  - jcfrei 12y ago
    Do you know google scholar? That's basically what it's for. Might as well be added to the list of advices: searching for the important papers with google scholar or CiteSeerX.
    - personlurking 12y ago
      Touché. I have heard of it but never used it since I'm not required by my work to read scientific papers.
  - seanmcdirmid 12y ago
    One of the tricks to getting your paper in (in many, not all, fields) is to cite as many PC papers as possible. It can turn into one big citation circle jerk, or at least an echo chamber.
  - Radiant 12y ago
    That was actually inspiration for Google's PageRank algorithm: http://en.wikipedia.org/wiki/PageRank#History http://en.wikipedia.org/wiki/PageRank#History
stewbrew 12y ago
I personally would recommend against making notes on the print out. I'd rather go with keeping your notes separately in a way you can easily categorize and search your notes.
georgerobinson 12y ago
As someone who has read almost 50 academic papers in the last 6 months (MSc in Computer Science, we read a paper before class and then analyze it with the lecturer) I agree unanimously with the parent. Above making notes, I find that challenging the material in the paper is key to understanding. For example, authors may make claims without sufficient data to support their arguments or they might not explain a particular anomaly in their results. There are nuances in papers everywhere, try to pick these these out as you read them. Secondly, you will at times read papers which assume a particular level of prior background knowledge. For example, I've had to read papers on rateless codes (Spinal Codes), indoor passive radar and full duplex radio with zero background in electrical engineering or physics. 6 months ago I would look at some of the equations and think "nope, not even going to try and understand that". This is bad, don't do it. If you don't understand the equation at first glance (and I would be surprised if you did) break it down piece by piece. Annotate what each variable/Greek letter represents, how it fits together with their explaination of the equation and how it might fit a trend shown in their results. Sometimes you might find that you have to do extensive Googling to understand whole papers, don't worry, in my experience this is perfectly normal. Finally, if you don't understand something, don't continue reading. Try to understand it before progressing. Only continue reading if you absolutely cannot get it. If you find that a lack of understanding of an earlier part of the paper starts to limit your understanding of later parts, stop! Go back and re-read the earlier parts. Quite often, material later in the paper can adequately explain concepts earlier on. Best of luck!

by sarahj 12y ago

a_e_k 12y ago

I'll second your first point about hard copies. It's one thing to skim a PDF on the screen to see if it might be worth my time, but if I really want to grok a paper I'm marking up a hardcopy. This has also made me appreciative of authors who put in the effort to make sure that their graphs and diagrams reduce well to grayscale. Instead of referring to the red line versus the green line, they'll use labels, marks, different cross hatch types, etc. (Don't get me wrong, good color is still very nice to have too!) It's also led me to making it a point to print off drafts of my own submissions on a B&W laser printer and make sure that my figures and captions are still understandable. I expect that this makes it easier for any colorblind readers as well, whatever type of colorblindness they may have.

mkl 12y ago

> Try to comprehend something on every read through - it doesn't have to be big, just something - whether it is the sample makeup or part of the methodology or the conclusions etc. You don't need to understand these in any particular order, but ensure to revise your understanding as you become more familiar with the paper. Lately I've been reading "How to Read a Book" by Adler and van Doren [1], which systematises this. It applies to papers just as well as books. Get the 1972 edition, as the 1940 edition is quite a bit less readable. [1] https://en.wikipedia.org/wiki/How_to_Read_a_Book https://en.wikipedia.org/wiki/How_to_Read_a_Book

jwr 12y ago

> * Most papers are useless (to you at the time you are reading them) - the sad truth about research is that 90% of the stuff you devote yourself to understanding will be wrong, outdated or not useful to what you are working on. This is very important advice. Do not assume that just because it's a published research paper it is valuable, correct, or useful. In fact, especially in applied CS, I found that authors will sometimes make what looks like an intentional effort to obfuscate the methods so that the paper is publishable (peer reviewers will not try to reproduce the results anyway), but the methods are not implementable, or at least not easily. This makes sense when you think about the competition in academia, authors doing consulting work for companies, or intending to start businesses of their own.

seanmcdirmid 12y ago
Usually the idea is bunk and they are trying to obscure that, more often than that the idea is great and they are trying to protect their IP for a startup. If you know something is possible, then it is not impossible to replicate results even with obscure directions about it. Except for Paxos, you really need good instructions at that point (that guy should get an award).
- deleted 12y ago
  [deleted]
- striking 12y ago
  Hah. I'm pretty dumb myself. That'll teach me. (I'm leaving the link, though. Great paper.) [1]: http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html#lamport-paxos http://research.microsoft.com/en-us/um/people/lamport/pubs/p...
  - tokenrove 12y ago
    Leslie Lamport is a man. You could argue something about Barbara Liskov's Viewstamped Replication, though, I guess.
    - 12y ago

seanmcdirmid 12y ago

> * Most papers are useless (to you at the time you are reading them) I would say that most papers are worthless. The publish or perish mentality in academia leads to more noise and less signal. You really have to fail fast on papers, find the seminal ones and the few under looked gems related to your topic that can hopefully be fished out via Google (though word of mouth is important). Also, if you publish, try not to be part of the problem.

personlurking 12y ago
> The publish or perish mentality in academia leads to more noise and less signal. Perhaps a Google-like ranking would help. The more cited a paper is (in the bibliography), the more it might not be noise. Then again, I don't like the idea that what is popular is also the best.
- jcfrei 12y ago
  Do you know google scholar? That's basically what it's for. Might as well be added to the list of advices: searching for the important papers with google scholar or CiteSeerX.
  - personlurking 12y ago
    Touché. I have heard of it but never used it since I'm not required by my work to read scientific papers.
- seanmcdirmid 12y ago
  One of the tricks to getting your paper in (in many, not all, fields) is to cite as many PC papers as possible. It can turn into one big citation circle jerk, or at least an echo chamber.
- Radiant 12y ago
  That was actually inspiration for Google's PageRank algorithm: http://en.wikipedia.org/wiki/PageRank#History http://en.wikipedia.org/wiki/PageRank#History

stewbrew 12y ago

I personally would recommend against making notes on the print out. I'd rather go with keeping your notes separately in a way you can easily categorize and search your notes.

georgerobinson 12y ago

As someone who has read almost 50 academic papers in the last 6 months (MSc in Computer Science, we read a paper before class and then analyze it with the lecturer) I agree unanimously with the parent. Above making notes, I find that challenging the material in the paper is key to understanding. For example, authors may make claims without sufficient data to support their arguments or they might not explain a particular anomaly in their results. There are nuances in papers everywhere, try to pick these these out as you read them. Secondly, you will at times read papers which assume a particular level of prior background knowledge. For example, I've had to read papers on rateless codes (Spinal Codes), indoor passive radar and full duplex radio with zero background in electrical engineering or physics. 6 months ago I would look at some of the equations and think "nope, not even going to try and understand that". This is bad, don't do it. If you don't understand the equation at first glance (and I would be surprised if you did) break it down piece by piece. Annotate what each variable/Greek letter represents, how it fits together with their explaination of the equation and how it might fit a trend shown in their results. Sometimes you might find that you have to do extensive Googling to understand whole papers, don't worry, in my experience this is perfectly normal. Finally, if you don't understand something, don't continue reading. Try to understand it before progressing. Only continue reading if you absolutely cannot get it. If you find that a lack of understanding of an earlier part of the paper starts to limit your understanding of later parts, stop! Go back and re-read the earlier parts. Quite often, material later in the paper can adequately explain concepts earlier on. Best of luck!