4 ms·

In case anyone hasn’t read it, the Datacolada article demonstrating Ariely committed fraud is a great read and extremely convincing. https://datacolada.org/98

In case anyone hasn’t read it, the Datacolada article demonstrating Ariely committed fraud is a great read and extremely convincing. https://datacolada.org/98 https://datacolada.org/98

nkurz 3y ago
Great article. The "Author Feedback" section at the bottom is interesting. All 4 that are there read exactly like you would hope: dedicated researches agreeing with the Datacolada analysis and expressing disappointment that they did not catch this error in time. One wonders where the truth is.
- ColoursofOSINT 3y ago
  My favorite thing is that Gino is now suing DataColada [1], and basically is claiming that 'they were right in every other case and I supported them, but with me they are lying'. 1: https://datacolada.org/113 https://datacolada.org/113
tqi 3y ago
(Not an excuse in any way) This data was so shoddily faked that I have a hard time believing someone did this with an intention to deceive. Uniform distribution with a hard cutoff at 50K??
- bombcar 3y ago
  If you fake and get away with it you’re already admitting to a type of laziness, so it expands y til finally you are faking it so lazily that you get caught. Faking data realistically is almost as hard as getting it honestly. I know, I “expanded the data pool” for my eight grade science project.
  - bee_rider 3y ago
    That somehow loops around to becoming an interesting 8’th grade science project. Do an easy experiment once correctly, then again with falsified data. Present the results side-by-side. Ask people if they can tell you which is which. Present have a bit on what sort of statistics could catch your faked data set.
    - bombcar 3y ago
      Yeah, I did the simplistic thing of decide a question and an answer that sounded good; ran five or so tests, and then extrapolated additional sets based on that. In my defense I would have run more tests had I started when I should have :)
  - bonoboTP 3y ago
    Except the "it" in "getting it" is a significant or even bombastic and sexy finding. There's no reliable way to generate those, it's not just about hardness. The more correct and meticulous you are with your data, the more it will generally tell you that the bombastic claim is false. The more meticulous and effortful you are about faking, the likelier it will slip by. Your suggestion only works in the simulated science, like school projects where you are retracing the footsteps of past successful scientists, to verify an already known result. There putting in more work will reveal that effect more and cleaner, because your teacher already knew how the thing works in the first place. This is totally unlike real science where we confront the frontier of the unknown.
  - tqi 3y ago
    Yeah that is fair, but it's mind bogglingly lazy/brazen. How much more effort is it have taken to just take an average of a few RAND calls to get a normal distribution instead of a uniform one? Did they not know the central limit theorem??
  - User23 3y ago
    > I “expanded the data pool” for my eight grade science project. My teachers warned against this and told us they would catch us if we tried.
- 77pt77 3y ago
  Amazing that whoever faked this data was ever able to get to harvard let alone get tenure. It's just so laughably faked it's not even funny.
- a_bonobo 3y ago
  In science we only catch the sloppy fakers, we don't catch the 'good' fakers. There was a similar case a while ago in spider biology: https://www.nature.com/articles/d41586-022-02156-2 https://www.nature.com/articles/d41586-022-02156-2 Pruitt had several influential spider biology papers out and when others failed to replicate and dug into the Supplementary Data, they also found lazy patterns. >When Laskowski dug into data sets that Pruitt had provided for the study, she was shocked to find stretches of data that seemed to have been duplicated, to represent findings for multiple spiders. This questionable data helped to bolster a long-unproven theory that repeated social interactions in a group of spiders cause individuals to behave predictably. So yeah, who knows how much scientific fraud there actually is; I haven't seen a case of fraud where the data was convincingly faked, which means that these cases are hard to detect or hard to prove.
chaostheory 3y ago
FYI datacoloda has a GoFundMe to defend against a lawsuit filed by Francesca Gino. https://www.gofundme.com/f/uhbka-support-data-coladas-legal-defense https://www.gofundme.com/f/uhbka-support-data-coladas-legal-... In early August 2023, Professor Gino filed a lawsuit for defamation against Harvard University, and against Leif, Joe, and Uri personally, claiming 25 million dollars in damages. Defending oneself in court is time-consuming and expensive regardless of the merits of the lawsuit – as First Amendment lawyer Ken White put it to Vox , “The process is the punishment.” Targets of scientific criticism can thus use the legal system to silence their critics. At present, Leif, Joe, and Uri do not have pro bono representation. The lawyers they’ve spoken to currently estimate that their defense could cost anywhere between $50,000 and $600,000 (depending on how far the lawsuit progresses). Their employers have so far only agreed to pay part of the legal fees. Defending science requires defending legitimate scientific criticism against legal bullying. Edit: I initially wrote that they met their GoFundMe goal of $350,000, which is true. However, I’m not sure why they set their goal to only $350k when they mention that legal costs could skyrocket to $600,000 which they have not met
yard2010 3y ago
Great article. nice job. In the response letter Arielly says that he got the data from the insurance company (with which he collaborated on this experiment) and didn't suspect a thing. What is the level of responsibility which is expected from a researcher? What should be the consequences for the researcher on such cases?
- JumpCrisscross 3y ago
  > Arielly says that he got the data from the insurance company (with which he collaborated on this experiment) and didn't suspect a thing The insurance company confirmed the data Arielly represents he got is not the data they sent. Arielly is a fraud.
  - andrewflnr 3y ago
    I'll grant that the motive is more obscure on the insurers side, but I wouldn't be so quick to take their word for it either.
    - JumpCrisscross 3y ago
      > I wouldn't be so quick to take their word for it Dan Arielly is a curious figure to give this sort of benefit of doubt.
      - andrewflnr 3y ago
        I'd say the same for a random insurance company.
        User23 3y ago
        Insurance adjusters are straight-laced to the point of being anal. They go to prison if they're not. I trust them here.
dang 3y ago
Discussed at the time: Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments) (Lots more related links at https://news.ycombinator.com/item?id=37719476 https://news.ycombinator.com/item?id=37719476)

In case anyone hasn’t read it, the Datacolada article demonstrating Ariely committed fraud is a great read and extremely convincing. https://datacolada.org/98 https://datacolada.org/98

nkurz 3y ago

Great article. The "Author Feedback" section at the bottom is interesting. All 4 that are there read exactly like you would hope: dedicated researches agreeing with the Datacolada analysis and expressing disappointment that they did not catch this error in time. One wonders where the truth is.

ColoursofOSINT 3y ago
My favorite thing is that Gino is now suing DataColada [1], and basically is claiming that 'they were right in every other case and I supported them, but with me they are lying'. 1: https://datacolada.org/113 https://datacolada.org/113

tqi 3y ago

(Not an excuse in any way) This data was so shoddily faked that I have a hard time believing someone did this with an intention to deceive. Uniform distribution with a hard cutoff at 50K??

bombcar 3y ago
If you fake and get away with it you’re already admitting to a type of laziness, so it expands y til finally you are faking it so lazily that you get caught. Faking data realistically is almost as hard as getting it honestly. I know, I “expanded the data pool” for my eight grade science project.
- bee_rider 3y ago
  That somehow loops around to becoming an interesting 8’th grade science project. Do an easy experiment once correctly, then again with falsified data. Present the results side-by-side. Ask people if they can tell you which is which. Present have a bit on what sort of statistics could catch your faked data set.
  - bombcar 3y ago
    Yeah, I did the simplistic thing of decide a question and an answer that sounded good; ran five or so tests, and then extrapolated additional sets based on that. In my defense I would have run more tests had I started when I should have :)
- bonoboTP 3y ago
  Except the "it" in "getting it" is a significant or even bombastic and sexy finding. There's no reliable way to generate those, it's not just about hardness. The more correct and meticulous you are with your data, the more it will generally tell you that the bombastic claim is false. The more meticulous and effortful you are about faking, the likelier it will slip by. Your suggestion only works in the simulated science, like school projects where you are retracing the footsteps of past successful scientists, to verify an already known result. There putting in more work will reveal that effect more and cleaner, because your teacher already knew how the thing works in the first place. This is totally unlike real science where we confront the frontier of the unknown.
- tqi 3y ago
  Yeah that is fair, but it's mind bogglingly lazy/brazen. How much more effort is it have taken to just take an average of a few RAND calls to get a normal distribution instead of a uniform one? Did they not know the central limit theorem??
- User23 3y ago
  > I “expanded the data pool” for my eight grade science project. My teachers warned against this and told us they would catch us if we tried.
77pt77 3y ago
Amazing that whoever faked this data was ever able to get to harvard let alone get tenure. It's just so laughably faked it's not even funny.
a_bonobo 3y ago
In science we only catch the sloppy fakers, we don't catch the 'good' fakers. There was a similar case a while ago in spider biology: https://www.nature.com/articles/d41586-022-02156-2 https://www.nature.com/articles/d41586-022-02156-2 Pruitt had several influential spider biology papers out and when others failed to replicate and dug into the Supplementary Data, they also found lazy patterns. >When Laskowski dug into data sets that Pruitt had provided for the study, she was shocked to find stretches of data that seemed to have been duplicated, to represent findings for multiple spiders. This questionable data helped to bolster a long-unproven theory that repeated social interactions in a group of spiders cause individuals to behave predictably. So yeah, who knows how much scientific fraud there actually is; I haven't seen a case of fraud where the data was convincingly faked, which means that these cases are hard to detect or hard to prove.

chaostheory 3y ago

FYI datacoloda has a GoFundMe to defend against a lawsuit filed by Francesca Gino. https://www.gofundme.com/f/uhbka-support-data-coladas-legal-defense https://www.gofundme.com/f/uhbka-support-data-coladas-legal-... In early August 2023, Professor Gino filed a lawsuit for defamation against Harvard University, and against Leif, Joe, and Uri personally, claiming 25 million dollars in damages. Defending oneself in court is time-consuming and expensive regardless of the merits of the lawsuit – as First Amendment lawyer Ken White put it to Vox , “The process is the punishment.” Targets of scientific criticism can thus use the legal system to silence their critics. At present, Leif, Joe, and Uri do not have pro bono representation. The lawyers they’ve spoken to currently estimate that their defense could cost anywhere between $50,000 and $600,000 (depending on how far the lawsuit progresses). Their employers have so far only agreed to pay part of the legal fees. Defending science requires defending legitimate scientific criticism against legal bullying. Edit: I initially wrote that they met their GoFundMe goal of $350,000, which is true. However, I’m not sure why they set their goal to only $350k when they mention that legal costs could skyrocket to $600,000 which they have not met

yard2010 3y ago

Great article. nice job. In the response letter Arielly says that he got the data from the insurance company (with which he collaborated on this experiment) and didn't suspect a thing. What is the level of responsibility which is expected from a researcher? What should be the consequences for the researcher on such cases?

JumpCrisscross 3y ago
> Arielly says that he got the data from the insurance company (with which he collaborated on this experiment) and didn't suspect a thing The insurance company confirmed the data Arielly represents he got is not the data they sent. Arielly is a fraud.
- andrewflnr 3y ago
  I'll grant that the motive is more obscure on the insurers side, but I wouldn't be so quick to take their word for it either.
  - JumpCrisscross 3y ago
    > I wouldn't be so quick to take their word for it Dan Arielly is a curious figure to give this sort of benefit of doubt.
    - andrewflnr 3y ago
      I'd say the same for a random insurance company.
      - User23 3y ago
        Insurance adjusters are straight-laced to the point of being anal. They go to prison if they're not. I trust them here.

dang 3y ago

Discussed at the time: Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments) (Lots more related links at https://news.ycombinator.com/item?id=37719476 https://news.ycombinator.com/item?id=37719476)