4 ms·

It is always difficult to try to understand and implement the theory explained in papers which seem fine on the surface, but when you look more closely, you fin

It is always difficult to try to understand and implement the theory explained in papers which seem fine on the surface, but when you look more closely, you find a bunch of mistakes, there are giant holes in the details, and you end up trying to redo the whole paper. There should be journals/websites/blogs dedicated to trying to reexplain / implement papers.

bluGill 5y ago
Doing a good review is hard. I've been reviewing a few C++ papers (things proposed to C++23) lately. Many of them are over my head and all I can find are a few spelling errors. The ones I've understood took me 3 readings before I found some giant holes in the details (which I pointed out to the authors, the next revision corrected them). In one case I actually started implementing the feature myself, and only then did I realize there was something ambiguous in the paper (after talking with the author we decided to document an implementors choices as it doesn't matter for the 99% use case, and the rest could go either way depending on hardware and data details so better to allow choice) The vast majority if papers I'm far too laze to go into that level of detail on. I just assume the authors is a smart person and so I trustingly let them go. It may well be if I understood the paper I'd be horrified, ask me in 2043 when we have 20 years of hindsight... I have to believe that peer review is the same - many reviewers are just reading and looking for something obvious but not really understanding details.
- Mezzie 5y ago
  SO difficult, especially given that just because you're in the 'same' field and technically qualified to do a peer review doesn't mean you actually understand what you're reading. For example, I'm qualified to review papers on educational programs for children. I should never be asked to do that.
- mellavora 5y ago
  Someone once said that they enjoyed one of my papers, and that even though they thought the writing was very clear they still had to read it 3 times to understand it. I told them that I had to write it 100 times and spend two years before I understood it. So if they could pick it up in 3 readings over 3 days, they were doing pretty good.

bluGill 5y ago

Doing a good review is hard. I've been reviewing a few C++ papers (things proposed to C++23) lately. Many of them are over my head and all I can find are a few spelling errors. The ones I've understood took me 3 readings before I found some giant holes in the details (which I pointed out to the authors, the next revision corrected them). In one case I actually started implementing the feature myself, and only then did I realize there was something ambiguous in the paper (after talking with the author we decided to document an implementors choices as it doesn't matter for the 99% use case, and the rest could go either way depending on hardware and data details so better to allow choice) The vast majority if papers I'm far too laze to go into that level of detail on. I just assume the authors is a smart person and so I trustingly let them go. It may well be if I understood the paper I'd be horrified, ask me in 2043 when we have 20 years of hindsight... I have to believe that peer review is the same - many reviewers are just reading and looking for something obvious but not really understanding details.

Mezzie 5y ago
SO difficult, especially given that just because you're in the 'same' field and technically qualified to do a peer review doesn't mean you actually understand what you're reading. For example, I'm qualified to review papers on educational programs for children. I should never be asked to do that.
mellavora 5y ago
Someone once said that they enjoyed one of my papers, and that even though they thought the writing was very clear they still had to read it 3 times to understand it. I told them that I had to write it 100 times and spend two years before I understood it. So if they could pick it up in 3 readings over 3 days, they were doing pretty good.