6 ms·

It's great that she did the right thing and had the paper retracted, but this is still terrible on so many levels. Maybe with a strong effect for every single

It's great that she did the right thing and had the paper retracted, but this is still terrible on so many levels. Maybe with a strong effect for every single subject a little more skepticism would have been warranted in the first place? Some manual spot checking if possible, or using a minimal independent implementation of the analysis code? Who knows if she'd gotten her grant, her assistant professorship without the publication of this incorrect finding. Who knows who didn't get any of that because they were a bit too careful in their work.

Thorrez 6y ago
> Who knows if she'd gotten her grant, her assistant professorship without the publication of this incorrect finding. Who knows who didn't get any of that because they were a bit too careful in their work. On the other hand, if she hadn't wasted her time on this useless study she might have done more useful studies and her career would be better than it is right now. She might have gotten even better grants if she hadn't made this error, and maybe fewer other people would have gotten grants. Maybe those other people being more careful helped them rather than hurt them. I don't see how speculating like this is very useful.
- darkerside 6y ago
  I think that underplays the difficulty of the work being done here. Or maybe it just underestimates the amount of luck involved. The downside is clearly greater than the upside. Why would you even imply the opposite?
  - Thorrez 6y ago
    I don't fully understand the relation between the difficulty of the work, the luck involved, and upside vs downside. I don't even know what you mean by the "downside", is that the downside of making the mistake, or the downside of not making the mistake? I'm saying the downside of making the mistake (for her) might be larger than the upside of making the mistake. I don't think that's obviously wrong. But my real main point is that speculation of this kind (in either direction) isn't very productive.
    - darkerside 6y ago
      You're right that there is an opportunity cost here, and that it may be underestimated by most people, but you just can't give people much imaginary credit for things they might have done.
      - Thorrez 6y ago
        That was sort of my point, we shouldn't be giving imaginary credit to various people like throwaway285524 was doing. Like I said, I'm not very happy speculating in either direction.
ordu 6y ago
> Maybe with a strong effect for every single subject a little more skepticism would have been warranted in the first place? It is easy to think over these lines in hindsight[1], but it is much harder to do it when there are no known mistakes. Obviously they had some plausible hypothesis, which predicted and explained results. The more strong result is, the better for hypothesis. I mean it was possible and maybe wise to suspect bug because results are too good. But it is hard from a cognitive standpoint. She describes bug as hard to find even after she found that results do not reproduce. It was even more hard to find this bug when there were less reasons to believe that there is a bug. [1] https://en.wikipedia.org/wiki/Hindsight_bias https://en.wikipedia.org/wiki/Hindsight_bias
- magicalhippo 6y ago
  A control group, using same program but with a constant radius circle, should have revealed the error quite quickly no?
  - ordu 6y ago
    I'm not sure, I didn't understand fully the bug involved. Possibly such a control group would show the dependent variable as a constant plus some random noise. It is totally expected behaviour for variable.
brabel 6y ago
> It's great that she did the right thing and had the paper retracted I'm sorry but you seem to have skipped reading the main part of the article. The paper was not retracted: "The editor and publisher were understanding and ultimately opted not to retract the paper but to instead publish a revised version of the article, linked to from the original paper, with the results section updated to reflect the true (opposite) results."

Thorrez 6y ago

> Who knows if she'd gotten her grant, her assistant professorship without the publication of this incorrect finding. Who knows who didn't get any of that because they were a bit too careful in their work. On the other hand, if she hadn't wasted her time on this useless study she might have done more useful studies and her career would be better than it is right now. She might have gotten even better grants if she hadn't made this error, and maybe fewer other people would have gotten grants. Maybe those other people being more careful helped them rather than hurt them. I don't see how speculating like this is very useful.

darkerside 6y ago
I think that underplays the difficulty of the work being done here. Or maybe it just underestimates the amount of luck involved. The downside is clearly greater than the upside. Why would you even imply the opposite?
- Thorrez 6y ago
  I don't fully understand the relation between the difficulty of the work, the luck involved, and upside vs downside. I don't even know what you mean by the "downside", is that the downside of making the mistake, or the downside of not making the mistake? I'm saying the downside of making the mistake (for her) might be larger than the upside of making the mistake. I don't think that's obviously wrong. But my real main point is that speculation of this kind (in either direction) isn't very productive.
  - darkerside 6y ago
    You're right that there is an opportunity cost here, and that it may be underestimated by most people, but you just can't give people much imaginary credit for things they might have done.
    - Thorrez 6y ago
      That was sort of my point, we shouldn't be giving imaginary credit to various people like throwaway285524 was doing. Like I said, I'm not very happy speculating in either direction.

ordu 6y ago

> Maybe with a strong effect for every single subject a little more skepticism would have been warranted in the first place? It is easy to think over these lines in hindsight[1], but it is much harder to do it when there are no known mistakes. Obviously they had some plausible hypothesis, which predicted and explained results. The more strong result is, the better for hypothesis. I mean it was possible and maybe wise to suspect bug because results are too good. But it is hard from a cognitive standpoint. She describes bug as hard to find even after she found that results do not reproduce. It was even more hard to find this bug when there were less reasons to believe that there is a bug. [1] https://en.wikipedia.org/wiki/Hindsight_bias https://en.wikipedia.org/wiki/Hindsight_bias

magicalhippo 6y ago
A control group, using same program but with a constant radius circle, should have revealed the error quite quickly no?
- ordu 6y ago
  I'm not sure, I didn't understand fully the bug involved. Possibly such a control group would show the dependent variable as a constant plus some random noise. It is totally expected behaviour for variable.

brabel 6y ago

> It's great that she did the right thing and had the paper retracted I'm sorry but you seem to have skipped reading the main part of the article. The paper was not retracted: "The editor and publisher were understanding and ultimately opted not to retract the paper but to instead publish a revised version of the article, linked to from the original paper, with the results section updated to reflect the true (opposite) results."