4 ms·

Since this is a journal focused on cancer and not machine learning, I can understand why the editors would see this paper as being worthy for for publication. U

by plus 7y ago

Since this is a journal focused on cancer and not machine learning, I can understand why the editors would see this paper as being worthy for for publication. Unfortunately, many of the readers will read the paper uncritically. If possible, you should write a critical response to this paper, focusing on its methodological flaws, and send it to the editors. It doesn't have to be long; critical response are usually a couple pages at most. This is likely the most effective way of removing (or at the very least, heavily qualifying) bad science from research journals.

michaelhoffman 7y ago
How to Publish a Scientific Comment in 1 2 3 Easy Steps http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comment-in-123-Easy-Steps.pdf http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comm... I agree that a formal comment is best although not necessarily easy. A comment on PubPeer is easier but it will probably only be seen by those with the PubPeer extension. I do machine learning in computational biology and cancer. The issues described in the parent comment are known among experts. It’s too bad so many others don’t know or care.
- chris_va 7y ago
  Thank you for that link, it was a joy to read
deleted 7y ago
[deleted]
0xab 7y ago
This is a huge problem throughout science, not just ML. As scientists, we're rewarded for publishing cool new things that work, not for pointing out things that don't or for pointing out flaws in existing papers. If the point is to get people to not read one bad paper, it's just a waste of my time. Most papers are false and a lot of them should never have passed review. If the authors actually wanted to do good ML research, they could always have reached out to a decent ML researcher who could have told them all of this. There's no shortage of us. The journal could have reached out to an ML reviewer. Why wouldn't they? But no one did, because the results look good and so they send it off to press and it's good for both the authors and the journal to have something that is hype-worthy. It's just the sad reality of modern science.
- mehrdadn 7y ago
  > Most papers are false and a lot of them should never have passed review. Do you mean this literally or is this a metaphor to illustrate the point? If you actually mean most papers are false it'd be nice to see a link on that!
  - michaelhoffman 7y ago
    John Ioannidis claims that "most published research is false" based on some rather dubious assumptions. https://www.annualreviews.org/doi/abs/10.1146/annurev-statistics-060116-054104 https://www.annualreviews.org/doi/abs/10.1146/annurev-statis...
    - DataWorker 7y ago
      I agree with him although the accuracy of that statement is partially based on how “published research” is defined. Operational definitions and measurement are themselves much of the problem.
- PierredeFermat 7y ago
  It's amazing that a similar concern is raised/discussed here just couple hours ago: https://news.ycombinator.com/item?id=19788088 https://news.ycombinator.com/item?id=19788088 Any chance we could connect over email or something?
Florin_Andrei 7y ago
I mean, if it's an interdisciplinary study, you may want to get advisers from all sides to look at it before you publish, no?

by plus 7y ago

michaelhoffman 7y ago

How to Publish a Scientific Comment in 1 2 3 Easy Steps http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comment-in-123-Easy-Steps.pdf http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comm... I agree that a formal comment is best although not necessarily easy. A comment on PubPeer is easier but it will probably only be seen by those with the PubPeer extension. I do machine learning in computational biology and cancer. The issues described in the parent comment are known among experts. It’s too bad so many others don’t know or care.

chris_va 7y ago
Thank you for that link, it was a joy to read

deleted 7y ago

[deleted]

0xab 7y ago

This is a huge problem throughout science, not just ML. As scientists, we're rewarded for publishing cool new things that work, not for pointing out things that don't or for pointing out flaws in existing papers. If the point is to get people to not read one bad paper, it's just a waste of my time. Most papers are false and a lot of them should never have passed review. If the authors actually wanted to do good ML research, they could always have reached out to a decent ML researcher who could have told them all of this. There's no shortage of us. The journal could have reached out to an ML reviewer. Why wouldn't they? But no one did, because the results look good and so they send it off to press and it's good for both the authors and the journal to have something that is hype-worthy. It's just the sad reality of modern science.

mehrdadn 7y ago
> Most papers are false and a lot of them should never have passed review. Do you mean this literally or is this a metaphor to illustrate the point? If you actually mean most papers are false it'd be nice to see a link on that!
- michaelhoffman 7y ago
  John Ioannidis claims that "most published research is false" based on some rather dubious assumptions. https://www.annualreviews.org/doi/abs/10.1146/annurev-statistics-060116-054104 https://www.annualreviews.org/doi/abs/10.1146/annurev-statis...
  - DataWorker 7y ago
    I agree with him although the accuracy of that statement is partially based on how “published research” is defined. Operational definitions and measurement are themselves much of the problem.
PierredeFermat 7y ago
It's amazing that a similar concern is raised/discussed here just couple hours ago: https://news.ycombinator.com/item?id=19788088 https://news.ycombinator.com/item?id=19788088 Any chance we could connect over email or something?

Florin_Andrei 7y ago

I mean, if it's an interdisciplinary study, you may want to get advisers from all sides to look at it before you publish, no?