4 ms·

Question: What are the selection criteria people use to determine which papers to read. I went through a period where I tried to get in the habit of reading re

by saeranv 10y ago

Question: What are the selection criteria people use to determine which papers to read. I went through a period where I tried to get in the habit of reading research papers regularly. Two issues I found that lead me to eventually dropping this: (1) At least in my industry (generative architecture & building performance modeling) it's frustrating how often papers don't include technical details, and resort simply to a high-level overviews of implementation and go straight to detailing their outcome. I don't learn much from such papers. (2) Low return of value of simply just reading the paper. I went through a period where I tried to read a paper everyday. By the end of this period I had a massive stack of highlighted, and annotated research papers I kept in a folder on my desk. And that folder has remained there untouched to this day. So I find there is low value in simply consuming research, and the key for me at least is to actually struggle through and implement the research I read. Which is why point (1) is key for me. I'm inspired by your post though, I want to start getting back into a slightly modified version of this. Identify research papers with algorithms I can play with/experiment - and then implement it, aiming for something like a paper per week or so? This process means I won't be able to do a paper per day, but it'll be a lot more valuable/interesting to me. This calls for a new git repo...

jonwachob91 10y ago
>resort simply to a high-level overviews of implementation and go straight to detailing their outcome Those don't sound like research papers but articles... What Journal(s) are you finding these papers in?
- saeranv 10y ago
  Is it only a research paper if it is published in a journal? What about conference papers? Here's an example I was reading recently which doesn't go into much detail: http://www.ibpsa.org/proceedings/asim2012/0097.pdf http://www.ibpsa.org/proceedings/asim2012/0097.pdf from the International Building Performance Simulation Association (IBPSA) conference.
  - tnecniv 10y ago
    Conferences certainly count if they are refereed (peer reviewed). I don't know anything about that field, but that paper certainly doesn't pass the first glance test that others have mentioned in this thread. Good figures, etc.
epberry 10y ago
I think it's almost impossible to regularly select only papers that will be helpful to your work or thought provoking in your field. The nature of the beast is that you're going to read a lot of irrelevant or simply bad papers so you have to develop strategies to discard those quickly. The other thing is unless you're following a specific thread of research from citations or a group of authors you're probably not going to know which papers are important/worth reading ahead of time. Some strategies I've seen conference reviewers use and that I use to quickly evaluate whether a paper is worth a deeper dive: 1. Figures. Can you understand the figure from the caption or associated text blurb? Is it neat and labeled correctly? I've found that excellent figures are a strong signal that the authors put a lot of thought and work into the paper. If you look at Yann LeCun's early works they all have really excellent diagrams with clear descriptions. Some of his figures are so classic that they regularly appear in ML presentations to this day. 2. Read the related work section. A lot of people seem to skip this but it can be dynamite if you have knowledge of the field. You'll probably be aware of other papers referenced in this section and when the authors of the current paper point out differences and similarities to other methods this will give you a great idea of whether this paper's approach is worth understanding. 3. Save the Method and Result sections for last. For me, and I would assume most people, these heavily technical sections are very time consuming to understand and ought to be read very last. Read abstract, intro, related works, and conclusion first. I would bet I discard half of the papers I pick up without ever readin the middle chunk. 4. Conference or Journal it was published in. IEEE, NIPS, CVPR are all places where the best stuff in my area gets published. Papers not from top shops should have to do more to earn your attention. Strong figures being the one I usually go back to. One thing I would not do, that I used to do, is worry about the number of citations a paper has. Awesome new research and old but potentially newly important work will not have many citations at first. This is just my list. I'd appreciate other filters people have for reading papers!
joshvm 10y ago
A fun game is following the reference trail right back to the source works, often decades ago. Pick something popular, say convolutional neural nets. Do a brief literature review and try and read the most highly cited papers as far back as you can go. You'll usually hit some really interesting (and very readable) old papers that cover really groundbreaking things. Using the convnet example, at some point you'll come across the Nobel winning paper on how cat brains interpret visual stimuli (Hubel, 1963).
matt4077 10y ago
I used to have a subscription to nature and just leaf through it, going as deep as I got into each paper. The subscription was quite cheap (may have been using student discount), and it has a sort-of "popular science" section in the front summarising the major papers for audiences from other fields. Nature publishes papers from all fields of science and it can be quite entertaining to take a deep dive into vulcanology. You'll also get to notice when different branches of science end up using the same methods. Of course that's more for entertainment value (and possibly annoying your friends). for work I stsck everything that appears interesting into an app (app-tly named "Papers") and search through it when I think there could be something in there.

by saeranv 10y ago

jonwachob91 10y ago

>resort simply to a high-level overviews of implementation and go straight to detailing their outcome Those don't sound like research papers but articles... What Journal(s) are you finding these papers in?

saeranv 10y ago
Is it only a research paper if it is published in a journal? What about conference papers? Here's an example I was reading recently which doesn't go into much detail: http://www.ibpsa.org/proceedings/asim2012/0097.pdf http://www.ibpsa.org/proceedings/asim2012/0097.pdf from the International Building Performance Simulation Association (IBPSA) conference.
- tnecniv 10y ago
  Conferences certainly count if they are refereed (peer reviewed). I don't know anything about that field, but that paper certainly doesn't pass the first glance test that others have mentioned in this thread. Good figures, etc.

epberry 10y ago

I think it's almost impossible to regularly select only papers that will be helpful to your work or thought provoking in your field. The nature of the beast is that you're going to read a lot of irrelevant or simply bad papers so you have to develop strategies to discard those quickly. The other thing is unless you're following a specific thread of research from citations or a group of authors you're probably not going to know which papers are important/worth reading ahead of time. Some strategies I've seen conference reviewers use and that I use to quickly evaluate whether a paper is worth a deeper dive: 1. Figures. Can you understand the figure from the caption or associated text blurb? Is it neat and labeled correctly? I've found that excellent figures are a strong signal that the authors put a lot of thought and work into the paper. If you look at Yann LeCun's early works they all have really excellent diagrams with clear descriptions. Some of his figures are so classic that they regularly appear in ML presentations to this day. 2. Read the related work section. A lot of people seem to skip this but it can be dynamite if you have knowledge of the field. You'll probably be aware of other papers referenced in this section and when the authors of the current paper point out differences and similarities to other methods this will give you a great idea of whether this paper's approach is worth understanding. 3. Save the Method and Result sections for last. For me, and I would assume most people, these heavily technical sections are very time consuming to understand and ought to be read very last. Read abstract, intro, related works, and conclusion first. I would bet I discard half of the papers I pick up without ever readin the middle chunk. 4. Conference or Journal it was published in. IEEE, NIPS, CVPR are all places where the best stuff in my area gets published. Papers not from top shops should have to do more to earn your attention. Strong figures being the one I usually go back to. One thing I would not do, that I used to do, is worry about the number of citations a paper has. Awesome new research and old but potentially newly important work will not have many citations at first. This is just my list. I'd appreciate other filters people have for reading papers!

joshvm 10y ago

A fun game is following the reference trail right back to the source works, often decades ago. Pick something popular, say convolutional neural nets. Do a brief literature review and try and read the most highly cited papers as far back as you can go. You'll usually hit some really interesting (and very readable) old papers that cover really groundbreaking things. Using the convnet example, at some point you'll come across the Nobel winning paper on how cat brains interpret visual stimuli (Hubel, 1963).

matt4077 10y ago

I used to have a subscription to nature and just leaf through it, going as deep as I got into each paper. The subscription was quite cheap (may have been using student discount), and it has a sort-of "popular science" section in the front summarising the major papers for audiences from other fields. Nature publishes papers from all fields of science and it can be quite entertaining to take a deep dive into vulcanology. You'll also get to notice when different branches of science end up using the same methods. Of course that's more for entertainment value (and possibly annoying your friends). for work I stsck everything that appears interesting into an app (app-tly named "Papers") and search through it when I think there could be something in there.