5 ms·

This video is great. Especially the comment that it takes a week to a month to deeply understand a research paper.

by ibash 2y ago

simonw 2y ago
The idea that it takes a week to a month to deeply understand a research paper feels to me like a massive failure in our expectations of academic writing. The reason it takes so long is that academic culture deliberately encourages creating documents that are extremely difficult for people to learn from. I am confident that in the vast majority of cases the increased understanding that somebody gets from spending 1-4 weeks of effort on a single paper was not actually worth that effort - the same result could have been had in an afternoon of direct conversation with the author of the paper. Academia really needs to break away from the culture of obfuscating these discoveries! At the very least, any paper worth its salt should be accompanied by an informal blog post that helps explain the discovery and maybe video or audio of the researcher communicating it as effectively as possible. EDIT: I just spotted that comment, it's a YouTube comment not a note in the video itself https://www.youtube.com/watch?v=nL7lAo95D-o&lc=UgxHeJrOv1g_tdQxYYh4AaABAg.A7virQ3xFm3A7vnI6iATMn https://www.youtube.com/watch?v=nL7lAo95D-o&lc=UgxHeJrOv1g_t... > If I'm trying to understand it deeply enough to mechanistically understand how the methodology work, this can take a few days to a week. > If I'm trying to reproduce the paper result to incorporate it within my own research project, I might wrestle with the thing for a full month. That comment feels very reasonable to me - for actually implementing a new deep learning strategy from a paper a full month feels fine to me. (I committed the cardinal Hacker News sin here of jumping on an opportunity to share one of my pet peeves, rather than engaging with the information directly!)
- sigmoid10 2y ago
  >academic culture deliberately encourages creating documents that are extremely difficult for people to learn from. That's because they are not meant to be used as a learning resource. They're meant to communicate results to other experts who will understand them much faster than you ever could. If you want to learn a new field, start with textbooks and then look for review papers or things like that. Understanding research papers may be your ultimate goal, but don't expect to get there just by consuming these types of papers.
  - gopher_space 2y ago
    > They're meant to communicate results to other experts who will understand them much faster than you ever could. And yet I'm still deconstructing papers entirely within my domain down to the bullet-pointed outline the author used to write it from. We do not need to write like Dickensian wordsmiths and we are not good at it. I'd consider it gatekeeping if I thought people wrote this way out of malice rather than cupidity.
    - sigmoid10 2y ago
      Meh, I consider myself pretty deep in my domains of expertise and even I regularly struggle because I haven't had that much exposition to a specific subject that is not exactly my specialty. You really can't rate these things unless you are literally among the top experts at the forefront of the topic. That's also who I'd have in mind when writing such a paper. Not some well versed enthusiast, not even a PhD student. Depending on your field, you might be able to count the people who can comfortably read and understand your paper on one hand. For everyone else, the effort will rise exponentially for every step they are removed from that specific topic.
      - gopher_space 2y ago
        I'm thinking more about a situation where the author has three bullet points to convey and needs to construct an encapsulating paragraph around them. When I'm parsing the paper I'll need to undo that work.
        sigmoid10 2y ago
        That's just what prose is. But admittedly some people are better at reading/writing it than others.
- yodsanklai 2y ago
  > The idea that it takes a week to a month to deeply understand a research paper feels to me like a massive failure in our expectations of academic writing. Isn't it just because of all the knowledge required to actually understand the paper? typically, you wouldn't expect someone without a strong background in the field to understand novel research in an afternoon. This is usually what's a PhD is for: covering the gap between master and state of the art.
- niemandhier 2y ago
  Usually a paper is the result of months of work. These months get condensed into a document of a few pages, part of this condensation process is to replace part of the needed explanations with references. Unless you are one of the few experts in exactly the field the author is working on, you will have to work through these references and make the connections to the work. After this you will have to create a mapping from the clear but h are es to grasp facts of the publication to your internal representation of the topic. All of these things take time, sometimes you get to skip the part where you have to read the references by consulting a pud thesis that the paper is based on (4 pages to ca. 120 expansion), even than you have to build your internal understanding. Talking to the author will indeed speed up this process, but that essentially means having an extremely qualified privat tutor.
- beng-nl 2y ago
  Edit: sibling comments add good perspective to this. While I agree papers should be clearer, and should be accompanied by auxiliary material making the paper easier to understand and apply, I do not agree academia deliberately encourages papers that are difficult to learn from, or of obfuscation. I have a little experience reviewing and writing papers in computer science, specifically security. As in any field, yes a lot of mentioned but also unmentioned background knowledge may be assumed; and jargon may be used; and it may be written with imperfect style and structure (researchers aren’t professional writers after all, and papers are often written on a harsh deadline). But I don’t think it’s in anyone’s interest, neither the authors nor the community at large, to make the paper inaccessible as a goal and I’ve never perceived this to be happening [1]. Source: have served on many conference PC’s and have been on the receiving end of many reviews, from which I can tell a paper being hard to follow can be grounds for rejection, and a paper being written with clarity and being easy to follow, learn from, reproduce, and build on, is greatly appreciated. [1] with one exception, come to think of it: needless formalism and mathematical notation, by one research group I remember. I got the impression the intention was to impress/intimidate the reader/reviewers and this was not appreciated.
- russfink 2y ago
  Page limits are at once both the savior and the tormenter of science.
- Simon_ORourke 2y ago
  Yes, but look at it from the other perspective, if they state things clearly they're not puffing themselves up to their fullest extent for academic peers and funding bodies.
rldjbpin 2y ago
The comment mentioned (reply to the pinned comment as of writing): > If I'm trying to understand it deeply enough to mechanistically understand how the methodology work, this can take a few days to a week. > If I'm trying to reproduce the paper result to incorporate it within my own research project, I might wrestle with the thing for a full month. > However, I don't do this process with all papers. That would be wasteful. I select which paper have high relevancy for myself and my lab by reading lots of them superficially, then I pin point a few that have disproportionate benefits. while the "deeply understand" notion might mean something else outside of the scope of the OP (assuming they are not doing this for further research), it seems that practically you spend a few days (not completely) to understand enough. personally i struggle to find good quality of technical writing in this field than not, and maybe that says about my competence more than anything else. given the empirical nature of tweaking some abstraction layer to have the paper in most cases, there must be a better way to present results.

This video is great. Especially the comment that it takes a week to a month to deeply understand a research paper.

by ibash 2y ago

This video is great. Especially the comment that it takes a week to a month to deeply understand a research paper.

The idea that it takes a week to a month to deeply understand a research paper feels to me like a massive failure in our expectations of academic writing. The reason it takes so long is that academic culture deliberately encourages creating documents that are extremely difficult for people to learn from. I am confident that in the vast majority of cases the increased understanding that somebody gets from spending 1-4 weeks of effort on a single paper was not actually worth that effort - the same result could have been had in an afternoon of direct conversation with the author of the paper. Academia really needs to break away from the culture of obfuscating these discoveries! At the very least, any paper worth its salt should be accompanied by an informal blog post that helps explain the discovery and maybe video or audio of the researcher communicating it as effectively as possible. EDIT: I just spotted that comment, it's a YouTube comment not a note in the video itself https://www.youtube.com/watch?v=nL7lAo95D-o&lc=UgxHeJrOv1g_tdQxYYh4AaABAg.A7virQ3xFm3A7vnI6iATMn https://www.youtube.com/watch?v=nL7lAo95D-o&lc=UgxHeJrOv1g_t... > If I'm trying to understand it deeply enough to mechanistically understand how the methodology work, this can take a few days to a week. > If I'm trying to reproduce the paper result to incorporate it within my own research project, I might wrestle with the thing for a full month. That comment feels very reasonable to me - for actually implementing a new deep learning strategy from a paper a full month feels fine to me. (I committed the cardinal Hacker News sin here of jumping on an opportunity to share one of my pet peeves, rather than engaging with the information directly!)

sigmoid10 2y ago
>academic culture deliberately encourages creating documents that are extremely difficult for people to learn from. That's because they are not meant to be used as a learning resource. They're meant to communicate results to other experts who will understand them much faster than you ever could. If you want to learn a new field, start with textbooks and then look for review papers or things like that. Understanding research papers may be your ultimate goal, but don't expect to get there just by consuming these types of papers.
- gopher_space 2y ago
  > They're meant to communicate results to other experts who will understand them much faster than you ever could. And yet I'm still deconstructing papers entirely within my domain down to the bullet-pointed outline the author used to write it from. We do not need to write like Dickensian wordsmiths and we are not good at it. I'd consider it gatekeeping if I thought people wrote this way out of malice rather than cupidity.
  - sigmoid10 2y ago
    Meh, I consider myself pretty deep in my domains of expertise and even I regularly struggle because I haven't had that much exposition to a specific subject that is not exactly my specialty. You really can't rate these things unless you are literally among the top experts at the forefront of the topic. That's also who I'd have in mind when writing such a paper. Not some well versed enthusiast, not even a PhD student. Depending on your field, you might be able to count the people who can comfortably read and understand your paper on one hand. For everyone else, the effort will rise exponentially for every step they are removed from that specific topic.
    - gopher_space 2y ago
      I'm thinking more about a situation where the author has three bullet points to convey and needs to construct an encapsulating paragraph around them. When I'm parsing the paper I'll need to undo that work.
      - sigmoid10 2y ago
        That's just what prose is. But admittedly some people are better at reading/writing it than others.
yodsanklai 2y ago
> The idea that it takes a week to a month to deeply understand a research paper feels to me like a massive failure in our expectations of academic writing. Isn't it just because of all the knowledge required to actually understand the paper? typically, you wouldn't expect someone without a strong background in the field to understand novel research in an afternoon. This is usually what's a PhD is for: covering the gap between master and state of the art.
niemandhier 2y ago
Usually a paper is the result of months of work. These months get condensed into a document of a few pages, part of this condensation process is to replace part of the needed explanations with references. Unless you are one of the few experts in exactly the field the author is working on, you will have to work through these references and make the connections to the work. After this you will have to create a mapping from the clear but h are es to grasp facts of the publication to your internal representation of the topic. All of these things take time, sometimes you get to skip the part where you have to read the references by consulting a pud thesis that the paper is based on (4 pages to ca. 120 expansion), even than you have to build your internal understanding. Talking to the author will indeed speed up this process, but that essentially means having an extremely qualified privat tutor.
beng-nl 2y ago
Edit: sibling comments add good perspective to this. While I agree papers should be clearer, and should be accompanied by auxiliary material making the paper easier to understand and apply, I do not agree academia deliberately encourages papers that are difficult to learn from, or of obfuscation. I have a little experience reviewing and writing papers in computer science, specifically security. As in any field, yes a lot of mentioned but also unmentioned background knowledge may be assumed; and jargon may be used; and it may be written with imperfect style and structure (researchers aren’t professional writers after all, and papers are often written on a harsh deadline). But I don’t think it’s in anyone’s interest, neither the authors nor the community at large, to make the paper inaccessible as a goal and I’ve never perceived this to be happening [1]. Source: have served on many conference PC’s and have been on the receiving end of many reviews, from which I can tell a paper being hard to follow can be grounds for rejection, and a paper being written with clarity and being easy to follow, learn from, reproduce, and build on, is greatly appreciated. [1] with one exception, come to think of it: needless formalism and mathematical notation, by one research group I remember. I got the impression the intention was to impress/intimidate the reader/reviewers and this was not appreciated.
russfink 2y ago
Page limits are at once both the savior and the tormenter of science.
Simon_ORourke 2y ago
Yes, but look at it from the other perspective, if they state things clearly they're not puffing themselves up to their fullest extent for academic peers and funding bodies.

rldjbpin 2y ago

The comment mentioned (reply to the pinned comment as of writing): > If I'm trying to understand it deeply enough to mechanistically understand how the methodology work, this can take a few days to a week. > If I'm trying to reproduce the paper result to incorporate it within my own research project, I might wrestle with the thing for a full month. > However, I don't do this process with all papers. That would be wasteful. I select which paper have high relevancy for myself and my lab by reading lots of them superficially, then I pin point a few that have disproportionate benefits. while the "deeply understand" notion might mean something else outside of the scope of the OP (assuming they are not doing this for further research), it seems that practically you spend a few days (not completely) to understand enough. personally i struggle to find good quality of technical writing in this field than not, and maybe that says about my competence more than anything else. given the empirical nature of tweaking some abstraction layer to have the paper in most cases, there must be a better way to present results.