19 ms·

How to seriously read a scientific paper

289 points by NalNezumi 6y ago

k__ 6y ago
Half-OT: What resources can you recommend to search/browse master thesis' online? I'm at my half yearly "I should write my thesis!" cycle and searching for a topic. My idea was reading existing papers and looking in the "future work" sections to find something interesting.
- kd5bjo 6y ago
  Your university or department library should have archive copies of all the theses they’ve accepted. In Iceland, they’re all collected at https://skemman.is/?locale=en https://skemman.is/?locale=en as well; similar websites likely exist for other places.
- Daub 6y ago
  Sorry to sound cynical, but in my experience future work sections in papers tend to be rather self-serving. A means to big-up their own research, or explain away discrepancies. Looking for new ideas? Start hanging around with students/staff from outside your own domain. No better way to get the ball rolling on new perspectives.
rmrfstar 6y ago
https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape...
- ohduran 6y ago
  Thanks for this! The format of the interview is clearly not aimed at clarifying how to (seriously) read scientific papers, with contradicting views among the interviewees.
- NalNezumi 6y ago
  Posted this paper long ago on HN too. It is probably the most consistent information on how to read a paper. The sciencemag article is nice read after this paper though, since there's a couple of small trips&tricks that is not in that paper (such as the emotional side of reading a paper, how to go on when you actually don't understand something/jargons, when to give up/look for better paper, etc)
  - rmrfstar 6y ago
    The part about finding a better paper is growing more important by the day. The arXiv is filling up with resume-padding garbage.
- ruph123 6y ago
  Ironic: if you can’t read a paper yet, how do you read the paper explaining how to read a paper?
  - praptak 6y ago
    You bootstrap by reading meta-scientific papers.
    - ojnabieoot 6y ago
      I was going to say "write ignorant comments on social media about the paper based solely on the title" but your approach sounds more useful.
      - ruph123 6y ago
        You must be fun at parties.
  - amelius 6y ago
    You make assumptions, just when you learned your first language. Then you check if everything makes sense. If it does, then probably your assumptions were correct.
  - belgesel 6y ago
    I think authors also taught about that and wrote the paper for newbies. Almost every new researcher is reading from top to bottom.
  - ska 6y ago
    One of the best ways into a new area is if you can find a well written PhD or MSc thesis in it. Finding that isn't always easy, but if they are well done the background introductory material will be approachable and much more complete than a paper, and the bibliography should be useful.
- sireat 6y ago
  This a variation of: https://en.wikipedia.org/wiki/How_to_Read_a_Book https://en.wikipedia.org/wiki/How_to_Read_a_Book Different ends but similar means (multiple passes)
maxioatic 6y ago
> I like to print out the paper and highlight the most relevant information, so on a quick rescan I can be reminded of the major points. I used to do this but I've tried to move to exclusively digital. I always, after some period of time, end up throwing out the papers I've printed. Plus digital allows for easier access across devices, quick searches, etc. Still though, I enjoy reading printed papers more. Curious if anyone else has a similar experience with print vs digital.
- Koliakis 6y ago
  I've been trying to make all-digital work for years. I feel like there are significant downsides with digital that make me want to go back to paper (but if I were to print out everything I read, my small apartment would be overflowing in a year and it seems so wasteful). One of the most obvious benefits to paper that comes to mind is the ability to lay multiple non-sequential pages (from the same paper and from other papers) side-by-side. That spatial aspect isn't replicable afaict and I definitely feel like it limits me in many ways.
  - maxioatic 6y ago
    Completely agree on both points. Another thing for me is just grabbing a paper and a pen and knowing that's all I can do for the next X minutes. No digital distractions possible.
  - BlueTemplar 6y ago
    It's pretty replicable - you just open the same document multiple times ! (Of course it's often easier to run out of screen space than out of desk space…)
    - Koliakis 6y ago
      It falls apart in how you interact with the pages. It's not even close to the same thing, so not replicable at all.
- bosie 6y ago
ramraj07 6y ago
By the time I was in year 4 of my PhD, reading a paper mostly involved: look at title, directly look at figure 1 and figure 8 to judge if the title actually matches the result. If the result is of genuine interest then read the abstract, and then go through the figures carefully. Otherwise for the most part the reminder of the paper is left alone. This only applies for papers where I already know the field and am only trying to gauge the incremental addition to the knowledge the paper makes. The style will be different for a paper in an unknown field, there it's best to just read it as prose from top to bottom with a marker at hand. To add more - not reading any text in the paper makes you laser focussed on trying to figure out what the data means without the authors trying to sugar coat anything with their perspective/agenda.
- twic 6y ago
  Agreed. Although the order i learned was title, figures, materials and methods. Sometimes there's useful stuff in the materials and methods, and sometimes that's where the bodies are buried.
  - ramraj07 6y ago
    At least in bio papers, the good ones make sure to give the most important method details in the legend itself so it's rare I have to refer the methods section. But yes, that does happen! Especially anything where they cure cancer in mice (the running joke is that you can also cure cancer in mice by stomping them)
- GirkovArpa 6y ago
  As a layman my impression is most papers are too verbose. At least the ones that I can understand. After reading tons of steganography papers (for example) I found they tend to begin with the same retelling of the history of steganography from the beginning of time up to now, even though it's completely irrelevant to the topic of the paper (which is not history).
  - kortilla 6y ago
    That section is helpful for researchers both to contextualize the paper and to leave pointers to other papers the current one builds on. You’re right it’s not directly about the method in the current paper but it’s an important section nonetheless, even if it’s a bit meta.
rdiddly 6y ago
My usual approach is to look for an abstract. If I see one, that means I've accidentally clicked a link to a scientific paper, and can proceed to close the tab. (Hopefully you can take that as the satire it is...)
jonnyone 6y ago
>Most often, what I am trying to get out of the papers is issues of methodology, experimental design, and statistical analysis. In lieu of consistently good science journalism, I try to take a peak at these things in the relevant papers as well. But this requires the additional knowledge of knowing the limits of certain 'methodologies, experimental designs, and statistical analyses' and what you can/can't extrapolate. Which, as somebody outside of formal research, I've only put together piecemeal through reading other criticisms. Are there any good, comprehensive resources for the limits of different study methods/experimental designs?
makach 6y ago
Sequentially, from the beginning. Take your time. Scientific articles are usually organised in a specific way that makes it easy to get a good understanding of the domain it is providing new knowledge. What does it mean to read something seriously? We consume content differently according to our own needs. Even if you read something seriously there is always a chance that you didn't read it "seriously" enough.
kriro 6y ago
For me during (CS) literature reviews it's generally: 1) Title: My research usually starts with a Google Scholar check and going through 4-5 pages and marking everything to Zotero based on title only. I only go through other databases once I exhausted the most interesting GS stuff. 2) Abstract -> Conclusions: If it fits it goes onto the to read stack with a self written 2 line summary of why I included it...I usually start sub-topic stacks at this point 3) Quick browse the methods section and figures/tables etc. 4) Complete read. If I'm in the beginning of a literature review I also read the sources and marker everything that sounds interesting to add later.
woile 6y ago
where can I find interesting papers? specially about computer science?
- kd5bjo 6y ago
  Wikipedia usually has academic references at the bottom of their CS articles, which are generally the foundational papers that introduced the idea in the first place.
  - DarkWiiPlayer 6y ago
    Dipping my toe into off-topic for a bit: Wikipedias "sources" can range from very accurate and on-topic to complete garbage. They certainly train you to filter out the good from the bad, as well as to consider whether the paper is even relevant to the statement it's being cited for.
- jarenmf 6y ago
  There's a nice compilation on this repository https://github.com/papers-we-love/papers-we-love https://github.com/papers-we-love/papers-we-love
- gsvclass 6y ago
  I'm building https://42papers.com https://42papers.com to surfaces top trending papers in Machine Learning and Computer Science. [shameless plug]
- proverbialbunny 6y ago
  Maybe 1/4th of the papers I read end up at https://arxiv.org/ https://arxiv.org/. I rarely just surf there, but when googling around looking up interesting cutting edge topics often a paper hosted on https://arxiv.org/ https://arxiv.org/ comes up on google search results. Also, there are communities that share a lot of papers. Eg if you're interested in ML (which, if you didn't know, is a CS topic), https://old.reddit.com/r/MachineLearning/ https://old.reddit.com/r/MachineLearning/ posts a lot of papers. edit: I just searched for a random CS paper for you. I haven't read it. Are you feeling lucky? https://arxiv.org/abs/1004.2772 https://arxiv.org/abs/1004.2772
sriku 6y ago
I ask myself two questions - 1. Is this paper of interest to me? 2. How will I assimilate the knowledge of this paper into my existing framework? For (1), the title+abstract helps a lot. Followed by the conclusion, which can indicate whether there is something really to be absorbed. For (2), I need some grasp of the "related work" - i.e. "how have others tried to solve this problem before this work?" So once I affirm interest, I might scan some of the references. If I'm familiar with them, I know enough to assimilate. Otherwise if the paper describes them adequately for the purpose, I settle for that with a note-to-self to dive into anything in detail. A third kind of "always on" mode for me is "can I reuse any techniques or results in this work?" and I read more carefully to gain that. For abstract and structure, my preferred structure (because I was trained like this) is .. in this specific order .. - 1. Introduction - broad sweep description of area to clue people in. 2. Related work - a survey of how people have solved some key thing. 3. Problem statement - What remains unsolved/unaddressed relative to the "related work". 4. Solution - How does this solve it. This includes methods. 5. Validation - How does the author claim that their solution actually solves the problem they set out to tackle. In some cases, this may involve user studies, in others it may be a mathematical result, etc. This is both my communication framework as well as a knowledge assimilation framework. edit: I find an abstract that's pretty much like one sentence or so for each of those items in that order easiest to assimilate. However, I admit this wasn't easy to develop initially and I struggled a lot with it before it became normal.
ACow_Adonis 6y ago
Generally, my pattern goes like this: 1. Read summary/abstract. 2. Then skim intro. 3. Then read discussion/conclusion. 4. Then methodology. 5. Then read front to back. 6. Then read front to back again: but now we're being extra skeptical: we're questioning everything, looking for common bad practices, checking doubtful references to see whether they're true (the number of references that I follow where I don't agree with the author that the reference says what they claim is too damn high). 7. on loop: read from front, hit and study topic/part you don't understand, focus on and study that part, upon understanding continue read through to see whether understanding elucidates any other parts you didn't understand. 8. repeat step 7 loop until you understand everything. Now running concurrently with most of these steps, there's two escape/break decisions to be made at almost every point: a) do I understand paper sufficiently, if yes, then stop reading. b) is paper actually worth continued reading: if no, then stop reading. overall, I'd say the vast majority of papers aren't worth reading. Some are easy and only require ~one read. A very very very very small amount are both worth reading, and take a substantial amount of time. multiple reads, several weeks/ months to grok. I don't think I'm being hyperbolic when I say the number that justify that probably are less than 10 in your entire lifetime.
omrjml 6y ago
The problem here are the writers, not the readers. Most people are just not good writers and good at communicating complex ideas clearly. But the obsession with publishing research in journal articles format means there are a lot of reluctant writers writing on unengaging platforms.
- permo-w 6y ago
  I think Asimov explored this idea a little in one of his short stories. I think there should be scientists and science writers. These writers should be employed by research institutions. Their job should be to understand the concepts and explain them in simple words. I feel this would give the science world a lot more accessibility. My problems with science papers have always been that the statistics have been written incomprehensibly (no layman knows what a p-number is) and that they repeat themselves so much that half the paper is just nothing
l1am0 6y ago
[Shameless self plug] I created a newsletter two weeks ago in which I send out my favorite scientific paper of the week on each Friday to the subscribers. Maybe a good option to try out what you learned in this article & thread :D https://simon-frey.com/weeklycspaper/ https://simon-frey.com/weeklycspaper/ As I mostly dive into topics regarding distributed systems and backend development the focus will definitely be in that area.
- brnt 6y ago
  How do you read so many papers? To properly read one I may need up to a day. If I read a few a week I wouldnt get any work done.
  - l1am0 6y ago
    I did read a lot during working on my thesis. That helped to build up a backlog of nice ones :D
- criddell 6y ago
  What papers have you sent out so far?
  - l1am0 6y ago
    You can find all previous issues in the archive: https://simon-frey.com/weeklycspaper/archive/ https://simon-frey.com/weeklycspaper/archive/
projektfu 6y ago
For me, it depends obviously on why I’m reading it. If it’s a matter of background, I read the abstract and look at the figures to see if they’re reasonable. Then I file it away. If I need to design an experiment, I read the abstract and the methodology, then study the results to see if the methodology is any good. The references in the methodology often lead to other good papers. Here you find discrepancies in what they think they’re doing, what they tell you they’re doing, and what they’re actually doing. Reagents, for example, can be very specialized, and may not actually be available from the vendor they list. It’s a great way to “meet your neighbors”, asking them if they can help you find materials. These days I mostly read critically, as I’m a practitioner and not a researcher. In this case I read the abstract and introduction. If the introduction isn’t 100% stuff I already know, I go out to the references. This makes critical reading time consuming and causes a fractal explosion of the number of papers in the collection. However, it is also how you find bullshit. Often someone in a review will assert that a reference supports a statement. Lazy or mendacious writers will copy that statement complete with the reference into their own paper. The actual reference may say nothing of the sort. Sometimes, there are multiple levels to this telephone game and each level loses nuance while adding error. When you see a lot of this, you know that there’s a bullshit problem in the domain. If the author seems prone to BS or plagiarism, the results are somewhat questionable. If the introduction is fine, I read the results and compare bits to the method. In my field, we look a lot at statistical power and effect sizes. Significance tests are often used incorrectly as a proxy and it pays to understand their limitations. It’s also important to know how close the result is to the real world (for me). A paper showing reduced mortality, faster recovery, less pain, etc., is much better than one that shows a change in a biochemical marker or cellular protein and RNA content. A study in the species of interest is usually better than one in a model. Ioannidis is right, most studies are flawed, so it’s best to avoid putting too much stock in any single one. A lot of good research has been done by questioning the accepted wisdom, which usually means one or two studies that may have flaws.
bookofjoe 6y ago
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3687192/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3687192/ https://www.elsevier.com/connect/infographic-how-to-read-a-scientific-paper?aaref=https%3A%2F%2Fwww.google.com%2F https://www.elsevier.com/connect/infographic-how-to-read-a-s... https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape... https://www.owlnet.rice.edu/~cainproj/courses/HowToReadSciArticle.pdf https://www.owlnet.rice.edu/~cainproj/courses/HowToReadSciAr... https://towardsdatascience.com/how-to-read-scientific-papers-df3afd454179 https://towardsdatascience.com/how-to-read-scientific-papers... https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008032 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
Grimm1 6y ago
I feel lucky that I had a class in college "Methods in Binary Analysis" that was centered around reading new research every week in binary analysis, presenting what you learned from the paper and then as a final project attempt to produce novel work from what had been read. Together with a professor who made a point to teach us how to effectively read a paper with many of the tips in this article being something that professor taught us.
im3w1l 6y ago
First read the abstract, if it doesn't confirm your preconceptions, try conclusions. If that doesn't confirm your preconceptions try results. If that too failed, then find a nitpick in methods. If even that fails, then move from the denial to anger stage of grief. If you are unable to eventually make it through all 5 stages, then refer to the quip about science advancing one funeral at a time.
- vagrantJin 6y ago
  > then refer to the quip about science advancing one funeral at a time. Heard another one about airline safety guidelines written in blood.
proverbialbunny 6y ago
My favorite papers are those that clearly explain their thought process, show what they did to a great level of detail, and in a clean and easy to read format. These papers can be used as a sort of make shift tutorial, which can be incredibly helpful when studying an unfamiliar topic. (I actually like them better than tutorials, because I get some perspective into their thinking and how they came to explore the topic which makes it easier to generalize the lesson.) Not just reading papers, but Wikipedia as well: When exploring a new domain of knowledge, you may not know much or any of the vocabulary being used. This turns the average person off. I see this as the most common hurdle faced by those self studying a new topic. The trick is to slow down, look at the different vocabulary you don't recognize, and then look them up. What sometimes happens is to learn this new vocabulary you'll be met with a wall of even more unfamiliar vocabulary. This turns into a dependency chain of unfamiliar topics. Looking up those words / dependencies recursively, eventually you'll get to familiar knowledge. You can start building up from that place of familiarity and before you know it you've picked up not only the basics of a new domain of knowledge but the more advanced topics build up to what you were interested in. Learning the prerequisites usually takes hours, but sometimes this process can take days. Taking days to learn a dependency chain to learn an advanced topic seems like a long time, but in comparison to taking a class on the topic or reading a text book, this style of learning is highly accelerated. Sometimes there is no alternative learning style if you need to parse a paper on a cutting edge unfamiliar topic not yet covered in text books, and imo if you pace yourself and value learning over any other goal, it can be quite a relaxing and enjoyable process. Reading papers is a lot of fun. When I'm learning new topics, I take notes. My goal is to maintain what I learn, not have it go in one ear and out the other, so why not slow down, relax, and take notes? Better yet if I come back to the same topic months later I can look over my notes. If I forgot something about the topic, I know my study style could be improved. This meta-learning feedback loop can be super helpful if you value and enjoy learning new topics (which you do or you wouldn't be on YC).
- Flex247A 6y ago
  Pretty good advice for a college undergrad. Thanks!
RocketSyntax 6y ago
1. Read the protocol in the back first if it exists. 2. Look at all of the charts, especially the subtext/ caption. 3. Highlight the actual paper for benefits of the method vs comparative methods. Try to find meta-analysis to compare multiple methods.
adrianadunn 6y ago
I work on the Semantic Scholar team at the Allen Institute for AI (allenai.org). We're working on solving the problems described in the post. We're investing considerable effort into making it easier for researchers to find and consume scientific literature. Our team is made up of engineers and researchers who have felt the pain points firsthand and are very motivated to design and build solutions that fix them. Our software is free to use, and always will be. Our search engine uses things like citation intent, citation influence, and figure and table extraction to make filtering through papers easier. We’re also currently prototyping an augmented reading experience that aims to embed contextual information directly into the reading experience so that it's easier to consume and comprehend academic literature. Give it a try: semanticscholar.org.
deleted 6y ago
[deleted]
teleforce 6y ago
The alternative three-pass technique on how to properly read research paper by Prof. Keshav is worth reading: http://ccr.sigcomm.org/online/files/p83-keshavA.pdf http://ccr.sigcomm.org/online/files/p83-keshavA.pdf

289 points by NalNezumi 6y ago

k__ 6y ago

Half-OT: What resources can you recommend to search/browse master thesis' online? I'm at my half yearly "I should write my thesis!" cycle and searching for a topic. My idea was reading existing papers and looking in the "future work" sections to find something interesting.

kd5bjo 6y ago
Your university or department library should have archive copies of all the theses they’ve accepted. In Iceland, they’re all collected at https://skemman.is/?locale=en https://skemman.is/?locale=en as well; similar websites likely exist for other places.
Daub 6y ago
Sorry to sound cynical, but in my experience future work sections in papers tend to be rather self-serving. A means to big-up their own research, or explain away discrepancies. Looking for new ideas? Start hanging around with students/staff from outside your own domain. No better way to get the ball rolling on new perspectives.

rmrfstar 6y ago

https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape...

ohduran 6y ago
Thanks for this! The format of the interview is clearly not aimed at clarifying how to (seriously) read scientific papers, with contradicting views among the interviewees.
NalNezumi 6y ago
Posted this paper long ago on HN too. It is probably the most consistent information on how to read a paper. The sciencemag article is nice read after this paper though, since there's a couple of small trips&tricks that is not in that paper (such as the emotional side of reading a paper, how to go on when you actually don't understand something/jargons, when to give up/look for better paper, etc)
- rmrfstar 6y ago
  The part about finding a better paper is growing more important by the day. The arXiv is filling up with resume-padding garbage.
ruph123 6y ago
Ironic: if you can’t read a paper yet, how do you read the paper explaining how to read a paper?
- praptak 6y ago
  You bootstrap by reading meta-scientific papers.
  - ojnabieoot 6y ago
    I was going to say "write ignorant comments on social media about the paper based solely on the title" but your approach sounds more useful.
    - ruph123 6y ago
      You must be fun at parties.
- amelius 6y ago
  You make assumptions, just when you learned your first language. Then you check if everything makes sense. If it does, then probably your assumptions were correct.
- belgesel 6y ago
  I think authors also taught about that and wrote the paper for newbies. Almost every new researcher is reading from top to bottom.
- ska 6y ago
  One of the best ways into a new area is if you can find a well written PhD or MSc thesis in it. Finding that isn't always easy, but if they are well done the background introductory material will be approachable and much more complete than a paper, and the bibliography should be useful.
sireat 6y ago
This a variation of: https://en.wikipedia.org/wiki/How_to_Read_a_Book https://en.wikipedia.org/wiki/How_to_Read_a_Book Different ends but similar means (multiple passes)

maxioatic 6y ago

> I like to print out the paper and highlight the most relevant information, so on a quick rescan I can be reminded of the major points. I used to do this but I've tried to move to exclusively digital. I always, after some period of time, end up throwing out the papers I've printed. Plus digital allows for easier access across devices, quick searches, etc. Still though, I enjoy reading printed papers more. Curious if anyone else has a similar experience with print vs digital.

Koliakis 6y ago
I've been trying to make all-digital work for years. I feel like there are significant downsides with digital that make me want to go back to paper (but if I were to print out everything I read, my small apartment would be overflowing in a year and it seems so wasteful). One of the most obvious benefits to paper that comes to mind is the ability to lay multiple non-sequential pages (from the same paper and from other papers) side-by-side. That spatial aspect isn't replicable afaict and I definitely feel like it limits me in many ways.
- maxioatic 6y ago
  Completely agree on both points. Another thing for me is just grabbing a paper and a pen and knowing that's all I can do for the next X minutes. No digital distractions possible.
- BlueTemplar 6y ago
  It's pretty replicable - you just open the same document multiple times ! (Of course it's often easier to run out of screen space than out of desk space…)
  - Koliakis 6y ago
    It falls apart in how you interact with the pages. It's not even close to the same thing, so not replicable at all.
bosie 6y ago

ramraj07 6y ago

By the time I was in year 4 of my PhD, reading a paper mostly involved: look at title, directly look at figure 1 and figure 8 to judge if the title actually matches the result. If the result is of genuine interest then read the abstract, and then go through the figures carefully. Otherwise for the most part the reminder of the paper is left alone. This only applies for papers where I already know the field and am only trying to gauge the incremental addition to the knowledge the paper makes. The style will be different for a paper in an unknown field, there it's best to just read it as prose from top to bottom with a marker at hand. To add more - not reading any text in the paper makes you laser focussed on trying to figure out what the data means without the authors trying to sugar coat anything with their perspective/agenda.

twic 6y ago
Agreed. Although the order i learned was title, figures, materials and methods. Sometimes there's useful stuff in the materials and methods, and sometimes that's where the bodies are buried.
- ramraj07 6y ago
  At least in bio papers, the good ones make sure to give the most important method details in the legend itself so it's rare I have to refer the methods section. But yes, that does happen! Especially anything where they cure cancer in mice (the running joke is that you can also cure cancer in mice by stomping them)
GirkovArpa 6y ago
As a layman my impression is most papers are too verbose. At least the ones that I can understand. After reading tons of steganography papers (for example) I found they tend to begin with the same retelling of the history of steganography from the beginning of time up to now, even though it's completely irrelevant to the topic of the paper (which is not history).
- kortilla 6y ago
  That section is helpful for researchers both to contextualize the paper and to leave pointers to other papers the current one builds on. You’re right it’s not directly about the method in the current paper but it’s an important section nonetheless, even if it’s a bit meta.

rdiddly 6y ago

My usual approach is to look for an abstract. If I see one, that means I've accidentally clicked a link to a scientific paper, and can proceed to close the tab. (Hopefully you can take that as the satire it is...)

jonnyone 6y ago

>Most often, what I am trying to get out of the papers is issues of methodology, experimental design, and statistical analysis. In lieu of consistently good science journalism, I try to take a peak at these things in the relevant papers as well. But this requires the additional knowledge of knowing the limits of certain 'methodologies, experimental designs, and statistical analyses' and what you can/can't extrapolate. Which, as somebody outside of formal research, I've only put together piecemeal through reading other criticisms. Are there any good, comprehensive resources for the limits of different study methods/experimental designs?

makach 6y ago

Sequentially, from the beginning. Take your time. Scientific articles are usually organised in a specific way that makes it easy to get a good understanding of the domain it is providing new knowledge. What does it mean to read something seriously? We consume content differently according to our own needs. Even if you read something seriously there is always a chance that you didn't read it "seriously" enough.

kriro 6y ago

For me during (CS) literature reviews it's generally: 1) Title: My research usually starts with a Google Scholar check and going through 4-5 pages and marking everything to Zotero based on title only. I only go through other databases once I exhausted the most interesting GS stuff. 2) Abstract -> Conclusions: If it fits it goes onto the to read stack with a self written 2 line summary of why I included it...I usually start sub-topic stacks at this point 3) Quick browse the methods section and figures/tables etc. 4) Complete read. If I'm in the beginning of a literature review I also read the sources and marker everything that sounds interesting to add later.

woile 6y ago

where can I find interesting papers? specially about computer science?

kd5bjo 6y ago
Wikipedia usually has academic references at the bottom of their CS articles, which are generally the foundational papers that introduced the idea in the first place.
- DarkWiiPlayer 6y ago
  Dipping my toe into off-topic for a bit: Wikipedias "sources" can range from very accurate and on-topic to complete garbage. They certainly train you to filter out the good from the bad, as well as to consider whether the paper is even relevant to the statement it's being cited for.
jarenmf 6y ago
There's a nice compilation on this repository https://github.com/papers-we-love/papers-we-love https://github.com/papers-we-love/papers-we-love
gsvclass 6y ago
I'm building https://42papers.com https://42papers.com to surfaces top trending papers in Machine Learning and Computer Science. [shameless plug]
proverbialbunny 6y ago
Maybe 1/4th of the papers I read end up at https://arxiv.org/ https://arxiv.org/. I rarely just surf there, but when googling around looking up interesting cutting edge topics often a paper hosted on https://arxiv.org/ https://arxiv.org/ comes up on google search results. Also, there are communities that share a lot of papers. Eg if you're interested in ML (which, if you didn't know, is a CS topic), https://old.reddit.com/r/MachineLearning/ https://old.reddit.com/r/MachineLearning/ posts a lot of papers. edit: I just searched for a random CS paper for you. I haven't read it. Are you feeling lucky? https://arxiv.org/abs/1004.2772 https://arxiv.org/abs/1004.2772

sriku 6y ago

I ask myself two questions - 1. Is this paper of interest to me? 2. How will I assimilate the knowledge of this paper into my existing framework? For (1), the title+abstract helps a lot. Followed by the conclusion, which can indicate whether there is something really to be absorbed. For (2), I need some grasp of the "related work" - i.e. "how have others tried to solve this problem before this work?" So once I affirm interest, I might scan some of the references. If I'm familiar with them, I know enough to assimilate. Otherwise if the paper describes them adequately for the purpose, I settle for that with a note-to-self to dive into anything in detail. A third kind of "always on" mode for me is "can I reuse any techniques or results in this work?" and I read more carefully to gain that. For abstract and structure, my preferred structure (because I was trained like this) is .. in this specific order .. - 1. Introduction - broad sweep description of area to clue people in. 2. Related work - a survey of how people have solved some key thing. 3. Problem statement - What remains unsolved/unaddressed relative to the "related work". 4. Solution - How does this solve it. This includes methods. 5. Validation - How does the author claim that their solution actually solves the problem they set out to tackle. In some cases, this may involve user studies, in others it may be a mathematical result, etc. This is both my communication framework as well as a knowledge assimilation framework. edit: I find an abstract that's pretty much like one sentence or so for each of those items in that order easiest to assimilate. However, I admit this wasn't easy to develop initially and I struggled a lot with it before it became normal.

ACow_Adonis 6y ago

Generally, my pattern goes like this: 1. Read summary/abstract. 2. Then skim intro. 3. Then read discussion/conclusion. 4. Then methodology. 5. Then read front to back. 6. Then read front to back again: but now we're being extra skeptical: we're questioning everything, looking for common bad practices, checking doubtful references to see whether they're true (the number of references that I follow where I don't agree with the author that the reference says what they claim is too damn high). 7. on loop: read from front, hit and study topic/part you don't understand, focus on and study that part, upon understanding continue read through to see whether understanding elucidates any other parts you didn't understand. 8. repeat step 7 loop until you understand everything. Now running concurrently with most of these steps, there's two escape/break decisions to be made at almost every point: a) do I understand paper sufficiently, if yes, then stop reading. b) is paper actually worth continued reading: if no, then stop reading. overall, I'd say the vast majority of papers aren't worth reading. Some are easy and only require ~one read. A very very very very small amount are both worth reading, and take a substantial amount of time. multiple reads, several weeks/ months to grok. I don't think I'm being hyperbolic when I say the number that justify that probably are less than 10 in your entire lifetime.

omrjml 6y ago

The problem here are the writers, not the readers. Most people are just not good writers and good at communicating complex ideas clearly. But the obsession with publishing research in journal articles format means there are a lot of reluctant writers writing on unengaging platforms.

permo-w 6y ago
I think Asimov explored this idea a little in one of his short stories. I think there should be scientists and science writers. These writers should be employed by research institutions. Their job should be to understand the concepts and explain them in simple words. I feel this would give the science world a lot more accessibility. My problems with science papers have always been that the statistics have been written incomprehensibly (no layman knows what a p-number is) and that they repeat themselves so much that half the paper is just nothing

l1am0 6y ago

[Shameless self plug] I created a newsletter two weeks ago in which I send out my favorite scientific paper of the week on each Friday to the subscribers. Maybe a good option to try out what you learned in this article & thread :D https://simon-frey.com/weeklycspaper/ https://simon-frey.com/weeklycspaper/ As I mostly dive into topics regarding distributed systems and backend development the focus will definitely be in that area.

brnt 6y ago
How do you read so many papers? To properly read one I may need up to a day. If I read a few a week I wouldnt get any work done.
- l1am0 6y ago
  I did read a lot during working on my thesis. That helped to build up a backlog of nice ones :D
criddell 6y ago
What papers have you sent out so far?
- l1am0 6y ago
  You can find all previous issues in the archive: https://simon-frey.com/weeklycspaper/archive/ https://simon-frey.com/weeklycspaper/archive/

projektfu 6y ago

For me, it depends obviously on why I’m reading it. If it’s a matter of background, I read the abstract and look at the figures to see if they’re reasonable. Then I file it away. If I need to design an experiment, I read the abstract and the methodology, then study the results to see if the methodology is any good. The references in the methodology often lead to other good papers. Here you find discrepancies in what they think they’re doing, what they tell you they’re doing, and what they’re actually doing. Reagents, for example, can be very specialized, and may not actually be available from the vendor they list. It’s a great way to “meet your neighbors”, asking them if they can help you find materials. These days I mostly read critically, as I’m a practitioner and not a researcher. In this case I read the abstract and introduction. If the introduction isn’t 100% stuff I already know, I go out to the references. This makes critical reading time consuming and causes a fractal explosion of the number of papers in the collection. However, it is also how you find bullshit. Often someone in a review will assert that a reference supports a statement. Lazy or mendacious writers will copy that statement complete with the reference into their own paper. The actual reference may say nothing of the sort. Sometimes, there are multiple levels to this telephone game and each level loses nuance while adding error. When you see a lot of this, you know that there’s a bullshit problem in the domain. If the author seems prone to BS or plagiarism, the results are somewhat questionable. If the introduction is fine, I read the results and compare bits to the method. In my field, we look a lot at statistical power and effect sizes. Significance tests are often used incorrectly as a proxy and it pays to understand their limitations. It’s also important to know how close the result is to the real world (for me). A paper showing reduced mortality, faster recovery, less pain, etc., is much better than one that shows a change in a biochemical marker or cellular protein and RNA content. A study in the species of interest is usually better than one in a model. Ioannidis is right, most studies are flawed, so it’s best to avoid putting too much stock in any single one. A lot of good research has been done by questioning the accepted wisdom, which usually means one or two studies that may have flaws.

bookofjoe 6y ago

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3687192/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3687192/ https://www.elsevier.com/connect/infographic-how-to-read-a-scientific-paper?aaref=https%3A%2F%2Fwww.google.com%2F https://www.elsevier.com/connect/infographic-how-to-read-a-s... https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape... https://www.owlnet.rice.edu/~cainproj/courses/HowToReadSciArticle.pdf https://www.owlnet.rice.edu/~cainproj/courses/HowToReadSciAr... https://towardsdatascience.com/how-to-read-scientific-papers-df3afd454179 https://towardsdatascience.com/how-to-read-scientific-papers... https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008032 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

Grimm1 6y ago

I feel lucky that I had a class in college "Methods in Binary Analysis" that was centered around reading new research every week in binary analysis, presenting what you learned from the paper and then as a final project attempt to produce novel work from what had been read. Together with a professor who made a point to teach us how to effectively read a paper with many of the tips in this article being something that professor taught us.

im3w1l 6y ago

First read the abstract, if it doesn't confirm your preconceptions, try conclusions. If that doesn't confirm your preconceptions try results. If that too failed, then find a nitpick in methods. If even that fails, then move from the denial to anger stage of grief. If you are unable to eventually make it through all 5 stages, then refer to the quip about science advancing one funeral at a time.

vagrantJin 6y ago
> then refer to the quip about science advancing one funeral at a time. Heard another one about airline safety guidelines written in blood.

proverbialbunny 6y ago

My favorite papers are those that clearly explain their thought process, show what they did to a great level of detail, and in a clean and easy to read format. These papers can be used as a sort of make shift tutorial, which can be incredibly helpful when studying an unfamiliar topic. (I actually like them better than tutorials, because I get some perspective into their thinking and how they came to explore the topic which makes it easier to generalize the lesson.) Not just reading papers, but Wikipedia as well: When exploring a new domain of knowledge, you may not know much or any of the vocabulary being used. This turns the average person off. I see this as the most common hurdle faced by those self studying a new topic. The trick is to slow down, look at the different vocabulary you don't recognize, and then look them up. What sometimes happens is to learn this new vocabulary you'll be met with a wall of even more unfamiliar vocabulary. This turns into a dependency chain of unfamiliar topics. Looking up those words / dependencies recursively, eventually you'll get to familiar knowledge. You can start building up from that place of familiarity and before you know it you've picked up not only the basics of a new domain of knowledge but the more advanced topics build up to what you were interested in. Learning the prerequisites usually takes hours, but sometimes this process can take days. Taking days to learn a dependency chain to learn an advanced topic seems like a long time, but in comparison to taking a class on the topic or reading a text book, this style of learning is highly accelerated. Sometimes there is no alternative learning style if you need to parse a paper on a cutting edge unfamiliar topic not yet covered in text books, and imo if you pace yourself and value learning over any other goal, it can be quite a relaxing and enjoyable process. Reading papers is a lot of fun. When I'm learning new topics, I take notes. My goal is to maintain what I learn, not have it go in one ear and out the other, so why not slow down, relax, and take notes? Better yet if I come back to the same topic months later I can look over my notes. If I forgot something about the topic, I know my study style could be improved. This meta-learning feedback loop can be super helpful if you value and enjoy learning new topics (which you do or you wouldn't be on YC).

Flex247A 6y ago
Pretty good advice for a college undergrad. Thanks!

RocketSyntax 6y ago

1. Read the protocol in the back first if it exists. 2. Look at all of the charts, especially the subtext/ caption. 3. Highlight the actual paper for benefits of the method vs comparative methods. Try to find meta-analysis to compare multiple methods.

adrianadunn 6y ago

I work on the Semantic Scholar team at the Allen Institute for AI (allenai.org). We're working on solving the problems described in the post. We're investing considerable effort into making it easier for researchers to find and consume scientific literature. Our team is made up of engineers and researchers who have felt the pain points firsthand and are very motivated to design and build solutions that fix them. Our software is free to use, and always will be. Our search engine uses things like citation intent, citation influence, and figure and table extraction to make filtering through papers easier. We’re also currently prototyping an augmented reading experience that aims to embed contextual information directly into the reading experience so that it's easier to consume and comprehend academic literature. Give it a try: semanticscholar.org.

deleted 6y ago

[deleted]

teleforce 6y ago

The alternative three-pass technique on how to properly read research paper by Prof. Keshav is worth reading: http://ccr.sigcomm.org/online/files/p83-keshavA.pdf http://ccr.sigcomm.org/online/files/p83-keshavA.pdf