6 ms·
How to (seriously) read a scientific paper (2016)
- amelius 3y agoStill waiting for an open discussion forum for scientific papers. Like StackOverflow, but every post is a paper. And there are experts of various gradations that help with questions.
- user_7832 3y agoI think researchgate tried to do that, but I don't know how successful they've been.
- roh26it 3y agoIndividual subreddits have usually been my go-to place to discuss and understand the nuances of academic papers. Love LocalLLama for discussions on generative AI papers.
- deleted 3y ago[deleted]
- thsksbd 3y agoThere's too many paper
- maCDzP 3y agoLike this? https://gotit.pub/ https://gotit.pub/
- shannonnn 3y agoI’m playing around with something similar here https://thetrecs.com/ https://thetrecs.com/ . But no experts yet. Even though I do science, I have a hard time understanding abstracts from adjacent fields so I’m using llms to make the abstracts easier for me and my friends to understand
- paulpauper 3y agoIt would be quickly overwhelmed, especially for complicated math papers
- wolverine876 3y agoBetter yet (IMHO), like HN. In my field, I know where to find papers, etc. I'm interested in other fields, and not every paper; just rank them by a HN-like algorithm. I said a few days ago, in my dream HN would be mostly papers. That's where the richest, most accurate, most intellectually curious information is - far beyond most things on HN.
- pests 3y agomost of the most intellectually curious things I've ever seen on the Internet were not research papers so I don't know how you can make that claim.
- wolverine876 3y agoIt's a matter of opinion, of course. But what are you reading? I'm very interested in finding more and better! I find research papers to be an order of magnitude richer, more accurate, more thoughtful than almost anything else. Everything else is too slow and doesn't answer most of the questions I have. Research papers (and books) not only answer the questions, they take me much further than what I knew existed - what I'd hope for from experts.
- pests 3y agoI think limiting yourself to research papers alone will deprieve you of ideas which are just developing, which may not have an official body of research or even any published papers about. Ideas we as a society or culture might only beginning to tackle. I find the discussion around a thing to be sometimes more interesting than the thing itself - a research paper might spark way more interesting discussions than the paper itself. Take the "Attention Is All You Need" paper. How many have actually read it vs. how much has been written about it, or about the things it led to? There is a lot packed into those 10 pages, but I've found the thousands of discussions around transformers and GPT to be way more interesting.
- wolverine876 3y ago
- geysersam 3y agoYes please. That's the peer review of my dreams. What an extraordinarily valuable resource that would be.
- godelski 3y agohttps://openreview.net/ https://openreview.net/ ? Open Review allows public comments. It's open, you can see revisions, authors can link repositories, datasets, and websites. You can even submit papers there outside of review. But it is expected that you are posting your own work. So kinda like arXiv but a bit more. Here's an example with Mamba. https://openreview.net/forum?id=AL1fq05o7H https://openreview.net/forum?id=AL1fq05o7H
- dwayne_dibley 3y agoReddit tried.
- o_____________o 3y agoI think this is what https://pubpeer.com/ https://pubpeer.com/ wants to be
- AlotOfReading 3y agoThis misses the best skimming trick I learned from an advisor: After you read the abstract, read the last couple lines/paragraphs of the introduction. That's where you'll find the authors best summary of the paper's contributions and novelty. For instance, the first paper that popped up with some random search terms [0]: The present paper attempts to provide a structured and comprehensive overview of state-of-the-art automated driving related hardware-software practices. [...] The aim of this paper is to fill this gap in the literature with a thorough survey. The remainder of this paper is written in eight sections. Section II is an overview [...] Details [...] are given in Section III. Section IV presents [...] etc. When you have a meter high stack of papers for your lit review, comprehending long introductions and conclusions gets too expensive. You want to filter the irrelevant stuff as quickly as possible so you can spend time focused on the couple hundred papers you might actually read and cite. [0] https://doi.org/10.1109/ACCESS.2020.2983149 https://doi.org/10.1109/ACCESS.2020.2983149
- ziddoap 3y agoMaybe I'm misunderstanding what you mean, but I don't think your trick was missed at all? The second paragraph says: >I start by reading the abstract. Then, I skim the introduction [...] A few paragraphs down, another quote says: >I always start with title and abstract. [...] I then read the introduction [...] Another: >I nearly always read the abstract first [...] I generally skim the introduction, reading its last paragraph Is there something significantly different about what you are suggesting?
- AlotOfReading 3y agoWhat I'm saying is similar to the third quote, where you start after the abstract by reading the last 1-5 lines of the introduction. Only if you want to proceed further do you dedicate the time to read or skim the entire intro, which can span multiple pages. You save tens of seconds and a lot of thinking this way, plus it helps frame things by giving a concise statement you can reference as you ask "how does what I'm reading build towards the paper's goals?"
- timr 3y ago
- forrestthewoods 3y ago1. Read the abstract 2. Read the last paragraph of the intro 3. Read the first paragraph of the conclusion 4. Read any diagrams or figures 5. Inspect the citations 6. If all else fails, read the paper itself
- refurb 3y agoI pretty much do this, but I also read the "Methods" section after the conclusion. You'd be surprised how quickly you can filter out a paper based on how they actually collected the data. It's not uncommon for papers to use crappy methods at either sampling, or they use an unreliable method for measuring the key data point. Or they use the right method, but you can see they made a mistake in the process. At least in my field (biochemistry) I'd always look at the methods as a hard filter.
- dang 3y agoRelated: How to seriously read a scientific paper - https://news.ycombinator.com/item?id=24986727 https://news.ycombinator.com/item?id=24986727 - Nov 2020 (90 comments)
- bee_rider 3y agoI think it really depends on where in your career you are? Like if you are near the beginning: * read the whole intro, make sure to look up any parts of the problem description you don’t understand * at least skim the body of the work. If you are trying to implement a code, make sure to understand all the algorithm blocks and note how each one corresponds to the overall work so you don’t end up implementing a special case. * read pay extra attention to the conclusion and skim the results If you are experienced, I guess you don’t need advice, but I figure it is something like: * skim the first couple sentences (for the problem) and the conclusion of the intro * skip to the results section to see if the person is screwing you around with gamed metrics * go back if it looks good and skim their ideas
- timr 3y ago> As editor-in-chief of Science....Generally, I start with the corresponding editors' summaries, which are meant for someone like me: a science generalist who is interested in everything but dives deeply only into one field. Next, I check to see if someone wrote a News article on the paper. Third, I check to see if there is a Perspective by another scientist. This says so much about the volume of absolute crap that gets published in Science. No competent editor-in-chief should be looking to the news media to give background on a research topic.
- Celmfire 3y agoI wouldn't rule it out entirely - a good news article can add context (e.g. via interviews with authors) that is otherwise not generally part of the paper itself
- guy_named_matt 3y agobut.... why is that? if scientific papers were written well (and i'm implicating the science publishing guidelines + accepted culture of scientific writing in that, not just individual authors), shouldn't they self-contain their context to the level that they are comprehensible? note, i'm not talking about spin - that doesn't belong. but if basic comprehension of the paper requires reading secondary sources, isn't that a fundamental problem in how scientific papers are written? (i'm asking this as a person with a phd and post-doc, working as a data scientist now but i have read many many scientific papers). I'll note - older papers (1960's, 1970's, etc.), do seem to include more color and more explanation. some modern papers do too. but many are dry and kind of incomprehensible.
- sxcurry 3y agoI guess you didn’t read the article very closely. The Science editor was talking about Science News (note the capitalization) which is a well respected information resource.
- timr 3y agoOK, first...Science News is trash. Absolute trash. Maybe it's better than e.g. the current NY Times Science reporter (who is so incompetent that it's mind-blowing), but it's still just not worth reading, except maybe on the toilet. They can (and do) say nonsensical stuff all the time. (edit: I just realized that you might be suggesting that "News" in the quote refers only to "Science News". I don't agree, but even if so, it doesn't change anything about my argument. If you are editor-in-chief of a scientific journal, you shouldn't be delegating your analysis to reporters, no matter where they work. If you aren't experienced/knowledgable enough to see why a paper is important from the work in front of you, give it to someone who can. And if you don't want to do that, then reject it.)
- fuckyah 3y ago[dead]
- dekhn 3y agoI see a lot of people who basically skip to the figures, and take them at face value, and read nothing more about the paper. When I read a paper I start with the abstract and go to the methods. then I inspect the figures carefully for the common tricks scientists play- axes tricks, overfitting, etc, looking to see whether I'm c onvinced the figures actually came from running an experiment, or just from copying and pasting other gels (this is extremely common). I used to do: read the whole paper in order, recursively visiting each citation (often 2-3 levels deep); within a few hours, I was convinced the entirety of science is build on somewhat shoddy foundations.
- passwordoops 3y agoThe methods section is my major judgement call. That's where you set off they have a clue what they're doing and care enough to worry about ensuring the work can be received. Which, after all, should be the main reason a to publish a scientific paper based on experiments
- etrautmann 3y agoWhat do you mean copy pasting? Like straight up fraud and fabricating results of a blot?
- dekhn 3y agoYes, apparently copying gel images is fairly common. And the folks who do it don't give much thought to how easy it is to detect, by skilled human eye or by computer. https://www.nytimes.com/interactive/2022/10/29/opinion/science-fraud-image-manipulation-photoshop.html https://www.nytimes.com/interactive/2022/10/29/opinion/scien... https://www.nytimes.com/2024/01/22/health/dana-farber-cancer-studies-retractions.html https://www.nytimes.com/2024/01/22/health/dana-farber-cancer... https://www.nytimes.com/2023/07/19/us/stanford-president-resigns-tessier-lavigne.html https://www.nytimes.com/2023/07/19/us/stanford-president-res... I should say, when I learned about Elisabeth Bik it was a revelation because I never thought other people saw copy patterns in papers, although I always assumed it just happened accidentally (I've run gels before and the whole process is terribly manual and prone to mistakes).
- 3y ago
- VoodooJuJu 3y ago1. Read the conclusion 2. Does the conclusion agree with the beliefs and politics of both you and your benefactors? Then it's good science. Does it disagree? Then it's misinformation.
- jillesvangurp 3y agoThis is similar to how I used to do it when I was still an academic, 20 years ago. My mental process is answering questions: is this an interesting paper with something to tell? Do the authors know their field (references)? Are they able to write a coherent and well structured article? If that all checks out, you start reading the paper in more depth. The 2024 version of this would of course be "hey chat gpt, summarize this article for me and list the key points and then criticize the work". I expect, article quality should improve a lot simply because authors are also using LLMs to help them polish their articles. There's no excuse for submitting unpolished work at this point. A lot of what I did as an academic was reviewing papers for workshops, conferences and journals. That's a chore that is part of academic life and getting good at that is a core skill. It's the really bad papers that are the most work. Because after you have a suspicion that something is garbage (this is the easy part) you actually have to read the damn thing in order to deconstruct it properly and provide detailed feedback to the authors that is both fair and actionable. A well written paper is much easier to review and more fun to read. Unfortunately, well written papers are quite rare. Mostly you are dealing with awkward grammar and style (inexperienced researchers, non native speakers, etc.), people who are unaware of key references they should have cited, a lot of bullshit conclusions and shoddy methodology, poorly structured arguments, etc. You can learn a lot about writing by reading a lot of bad writing and having others criticize your work. I'm really grateful for all the people who took the time to deal with my early efforts. It wasn't great, I know that. I'm not a native speaker. I had to learn writing on the job.
- jll29 3y agoComplementary: How to Write a Paper (University of Cambridge) http://www-mech.eng.cam.ac.uk/mmd/ashby-paper-V6.pdf http://www-mech.eng.cam.ac.uk/mmd/ashby-paper-V6.pdf
- lamontcg 3y agoYou'll note that the people in this thread who claim to have higher levels of expertise reads the Methods section pretty closely. For the scientifically-interested public this is where it can all go wrong since we lack the background to properly assess the methods since we've never done it before. Once we've gone through a paper and it all seems reasonable to us, we have to really seek out expert opinion on the methods and see what critiques are out there--we're not going to be able to judge the paper's methods in a relative vacuum of actual experience.
- even_639765 3y agoThe correct way to read a paper is to start at the methods section and determine if their methodology was any good. If not, find another paper.