3 ms·

> Worth a policy change by university for clawbacks of salary As a matter of employment law, it's generally pretty difficult to claw back pay from an employee.

by mlyle 3y ago

> Worth a policy change by university for clawbacks of salary As a matter of employment law, it's generally pretty difficult to claw back pay from an employee. And this is almost always a good thing.

stronglikedan 3y ago
Definitely a good thing. There's already channels for this. If they can prove fraud, they can go through the legal system to get that money back. It doesn't need to be a university policy, and shouldn't be - or any enforceable employer policy for that matter.
- godelski 3y ago
  Agreed. I don't see how the current system doesn't already have channels to take action against her. If fraud can be proven, which looks like it can be, then: the school will fire her, sue her, the publisher/s can sue her, etc. Even that guy with the signed book in the comments. There is no need to update contracts to necessitate clawbacks since these essentially already exist. I'd assume that such a policy change could actually have severe unintended consequences as it would necessitate some metric to evaluate this, which it is notoriously difficult to do with respect to research. Not to mention that most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes. I'd assume the consequences would result in this territory, making mistakes far more costly. Not something we want for research, where essentially we're trying to learn how to play the rules to a game by playing the game. Often mistakes help us learn those rules.
  - mike_hearn 3y ago
    The university will almost certainly not sue. When does that ever happen? I've been reading about research fraud for years and am struggling to think of an example of that. It's more common that academics sue the people who reveal their fraud than universities sue academics. The US does have the ORI which mounts a handful of legal cases per year. No, universities are a reputation economy and do their best to avoid reacting to or calling attention to fraud. They benefit from the perception that their professors are all competent and honest. This particular incident has got some press pickup due to the irony of an ethics professor who wrote a book about breaking the rules at work being caught cheating, but academic fraud is common and people who report it say that almost invariably nothing ever happens. Even in the rare cases when there's an investigation, it all gets swept under the rug. Note that: • Harvard took over a year to request retractions, even after outsiders did the heavy lifting for them. • This prof has silently been put on leave. • Harvard won't comment. • She has many coauthors on the bad papers. Nothing has happened to them. They were all blissfully ignorant. > most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes In reality the line is blurry to the point of being meaningless. Many glaring problems in research papers get claimed to be "mistakes" even when there's no plausible way they could be so. You have to be willing to penalize mistakes too, otherwise you can never penalize fraud. As an example, check out the story at the top of this essay I wrote a couple of years ago: https://blog.plan99.net/fake-science-part-i-7e9764571422 https://blog.plan99.net/fake-science-part-i-7e9764571422 The before-surgery and after-surgery photos are clearly pixelwise identical except for the skin lesions, which were airbrushed out. Despite the sheer visual obviousness of this fraud the journal didn't notice, and when outsiders did the authors simply claimed “The photograph was taken in the same room with a similar environment; unfortunately the patient wore the same shirt” and the journal accepted this explanation! A ten year old could do better than that. BTW don't conflate the unknown nature of answers to research questions, with the known nature of how to scientifically answer questions. The rules aren't hard, especially when you start from a base as low as "don't fake your data".
    - godelski 3y ago
      I can definitely buy this argument, but mine was about the mechanisms already existing and how the OP's comment about a policy change is needed. I think your comment just adds to this, that even if the mechanism exists that it isn't being utilized. Which results in also negating the utility of such a policy. As to your other comments and blog, I'm well aware. And if I may add some constructive criticism, if you want to call something pixelwise identical, it helps to show a differencing map. It just makes the claim substantially stronger and clearer, especially since there's a lot of additional factors that can change pixels, making the likelihood of a match even lower. These are things that wouldn't be possible by their explanation, whereas human interpretation is more subjective (it'll be a bit messier than this though). Though even low differences are still pretty damning. > with the known nature of how to scientifically answer questions. I think this is a bit naive though. Generally in science we rule things out, not rule in. That's why I'd be hard pressed to say that we have a "known nature of how to scientifically answer questions" because "scientifically" is doing a lot of heavy lifting and is fuzzy. Essentially "there's wrong ways to do science, but not a clear 'right way.'" Obviously faking data isn't okay nor is just doing something and calling it an experiment. Rigor is necessary but what constitutes sufficient rigor is dependent on the strength of the claims and thus the uncertainty of those claims. A big issue we face is that we take empirical results as equivalent to analytically derived theoretical models combined with experimental results. i.e. Physics develops a theoretical mathematical model and tests experiments against that whereas {psych,medicine,ML} make a non-mathematical hypothesis and report results. The problem with the latter is that it is harder to capture uncertainty because it isn't as strictly bounded. But it is still science, and the reason this is done is because the models are likely intractable (if not provably so). But that's why Asimov wrote about the Relativity of Wrong, rather than the precise way in which we can exactly do science. In fact, if the latter was true, we'd program machines to do it. Just a bit more nuanced.
      - mike_hearn 3y ago
        There's a pixel diff linked to from the image caption. Point well taken. The a-theoretical nature of the social sciences (and many others) has come up before. "A problem in theory" discusses this: https://www.nature.com/articles/s41562-018-0522-1 https://www.nature.com/articles/s41562-018-0522-1

by mlyle 3y ago

> Worth a policy change by university for clawbacks of salary As a matter of employment law, it's generally pretty difficult to claw back pay from an employee. And this is almost always a good thing.

stronglikedan 3y ago

Definitely a good thing. There's already channels for this. If they can prove fraud, they can go through the legal system to get that money back. It doesn't need to be a university policy, and shouldn't be - or any enforceable employer policy for that matter.

godelski 3y ago
Agreed. I don't see how the current system doesn't already have channels to take action against her. If fraud can be proven, which looks like it can be, then: the school will fire her, sue her, the publisher/s can sue her, etc. Even that guy with the signed book in the comments. There is no need to update contracts to necessitate clawbacks since these essentially already exist. I'd assume that such a policy change could actually have severe unintended consequences as it would necessitate some metric to evaluate this, which it is notoriously difficult to do with respect to research. Not to mention that most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes. I'd assume the consequences would result in this territory, making mistakes far more costly. Not something we want for research, where essentially we're trying to learn how to play the rules to a game by playing the game. Often mistakes help us learn those rules.
- mike_hearn 3y ago
  The university will almost certainly not sue. When does that ever happen? I've been reading about research fraud for years and am struggling to think of an example of that. It's more common that academics sue the people who reveal their fraud than universities sue academics. The US does have the ORI which mounts a handful of legal cases per year. No, universities are a reputation economy and do their best to avoid reacting to or calling attention to fraud. They benefit from the perception that their professors are all competent and honest. This particular incident has got some press pickup due to the irony of an ethics professor who wrote a book about breaking the rules at work being caught cheating, but academic fraud is common and people who report it say that almost invariably nothing ever happens. Even in the rare cases when there's an investigation, it all gets swept under the rug. Note that: • Harvard took over a year to request retractions, even after outsiders did the heavy lifting for them. • This prof has silently been put on leave. • Harvard won't comment. • She has many coauthors on the bad papers. Nothing has happened to them. They were all blissfully ignorant. > most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes In reality the line is blurry to the point of being meaningless. Many glaring problems in research papers get claimed to be "mistakes" even when there's no plausible way they could be so. You have to be willing to penalize mistakes too, otherwise you can never penalize fraud. As an example, check out the story at the top of this essay I wrote a couple of years ago: https://blog.plan99.net/fake-science-part-i-7e9764571422 https://blog.plan99.net/fake-science-part-i-7e9764571422 The before-surgery and after-surgery photos are clearly pixelwise identical except for the skin lesions, which were airbrushed out. Despite the sheer visual obviousness of this fraud the journal didn't notice, and when outsiders did the authors simply claimed “The photograph was taken in the same room with a similar environment; unfortunately the patient wore the same shirt” and the journal accepted this explanation! A ten year old could do better than that. BTW don't conflate the unknown nature of answers to research questions, with the known nature of how to scientifically answer questions. The rules aren't hard, especially when you start from a base as low as "don't fake your data".
  - godelski 3y ago
    I can definitely buy this argument, but mine was about the mechanisms already existing and how the OP's comment about a policy change is needed. I think your comment just adds to this, that even if the mechanism exists that it isn't being utilized. Which results in also negating the utility of such a policy. As to your other comments and blog, I'm well aware. And if I may add some constructive criticism, if you want to call something pixelwise identical, it helps to show a differencing map. It just makes the claim substantially stronger and clearer, especially since there's a lot of additional factors that can change pixels, making the likelihood of a match even lower. These are things that wouldn't be possible by their explanation, whereas human interpretation is more subjective (it'll be a bit messier than this though). Though even low differences are still pretty damning. > with the known nature of how to scientifically answer questions. I think this is a bit naive though. Generally in science we rule things out, not rule in. That's why I'd be hard pressed to say that we have a "known nature of how to scientifically answer questions" because "scientifically" is doing a lot of heavy lifting and is fuzzy. Essentially "there's wrong ways to do science, but not a clear 'right way.'" Obviously faking data isn't okay nor is just doing something and calling it an experiment. Rigor is necessary but what constitutes sufficient rigor is dependent on the strength of the claims and thus the uncertainty of those claims. A big issue we face is that we take empirical results as equivalent to analytically derived theoretical models combined with experimental results. i.e. Physics develops a theoretical mathematical model and tests experiments against that whereas {psych,medicine,ML} make a non-mathematical hypothesis and report results. The problem with the latter is that it is harder to capture uncertainty because it isn't as strictly bounded. But it is still science, and the reason this is done is because the models are likely intractable (if not provably so). But that's why Asimov wrote about the Relativity of Wrong, rather than the precise way in which we can exactly do science. In fact, if the latter was true, we'd program machines to do it. Just a bit more nuanced.
    - mike_hearn 3y ago
      There's a pixel diff linked to from the image caption. Point well taken. The a-theoretical nature of the social sciences (and many others) has come up before. "A problem in theory" discusses this: https://www.nature.com/articles/s41562-018-0522-1 https://www.nature.com/articles/s41562-018-0522-1