7 ms·

Everyone, please read the whole article, not just this comment. The second two quotes here appear very out of context in this comment - they are about anger and

Everyone, please read the whole article, not just this comment. The second two quotes here appear very out of context in this comment - they are about anger and frustration towards Gino, not towards Data Colada. The first quote here is not a defense of Gino, it's a nuanced opinion of self-proclaimed vigilantes who are also human and might not always make perfect decisions. It is followed by this: > Discussing the allegations that Gino falsified data is seen almost unanimously as fair game, especially in light of Harvard putting her on leave. People expressed more mixed feelings toward disgraced researchers such as Cuddy, who failed to replicate her findings but did not fake her data. I have no special insight here, but this at least seems like a reasonable thing to discuss. > It feels, to me, that they're making Data Colada (the data vigilante group) the bad guys here. I strongly disagree. The light questioning of Data Colada was a very small part of the article, and even that section did not give me the impression that they are "bad guys".

jonhohle 3y ago
If the findings can’t be replicated, were they worth publishing to begin with? I know there is pressure to publish, but the effect on society is significantly greater than a single researcher’s personal job security. If nothing else, papers with expensive or one-off experiments should be clearly marked as observations that should not be relied on as foundational for other work until replicated. When they’re cited peer reviewers could make sure they are referenced as such without requiring the peer reviewer to intimately know the details of the reference.
- the_snooze 3y ago
  I'm in a different field (computer security and privacy) and I want to point out that the notion of "replication" is trickier than what you might expect. Let's say I do a simple empircal measurement study, say on network traffic on some piece of internet infrastructure day-by-day in the year 2020. I find there's a huge sustained increase starting in mid-March, sustaining through the end of the year. I submit my results for publication. How can someone replicate those results, when the measurements were subject to conditions unique to that observation period? The best anyone can do is to replicate my statistics/analysis if I gave them my data set. That's different notion of replication. Would you consider that sufficient?

jonhohle 3y ago

If the findings can’t be replicated, were they worth publishing to begin with? I know there is pressure to publish, but the effect on society is significantly greater than a single researcher’s personal job security. If nothing else, papers with expensive or one-off experiments should be clearly marked as observations that should not be relied on as foundational for other work until replicated. When they’re cited peer reviewers could make sure they are referenced as such without requiring the peer reviewer to intimately know the details of the reference.

the_snooze 3y ago
I'm in a different field (computer security and privacy) and I want to point out that the notion of "replication" is trickier than what you might expect. Let's say I do a simple empircal measurement study, say on network traffic on some piece of internet infrastructure day-by-day in the year 2020. I find there's a huge sustained increase starting in mid-March, sustaining through the end of the year. I submit my results for publication. How can someone replicate those results, when the measurements were subject to conditions unique to that observation period? The best anyone can do is to replicate my statistics/analysis if I gave them my data set. That's different notion of replication. Would you consider that sufficient?