5 ms·

This is Dan Ariely's response to the case: http://datacolada.org/storage_strong/DanBlogComment_Aug_16_2021_final.pdf http://datacolada.org/storage_strong/DanBl

by chirau 5y ago

This is Dan Ariely's response to the case: http://datacolada.org/storage_strong/DanBlogComment_Aug_16_2021_final.pdf http://datacolada.org/storage_strong/DanBlogComment_Aug_16_2...

deleted 5y ago
[deleted]
warent 5y ago
This is very helpful. Assuming Dan is being honest, it seems pretty obvious the insurance company is at fault here. It's so hard to know though. He could be covering for the whole group, or a teammate, or even himself. Frankly I'm fine believing it was the insurance company though. Auto insurance has already screwed me before and now I have an irrational and emotionally charged inclination to view them as villain
- drno123 5y ago
  However, car insurance company has nothing to gain from faking the numbers. Researches who publish an influential study that gets a lot of citations, and because of it they can get tenure, big contracts or lucrative job offers, have everything to gain from faking numbers.
  - asdfasgasdgasdg 5y ago
    That's a good point, but it could be that the person tasked with gathering the numbers had something to gain (their own time, if they felt they could fudge and still get paid), and nobody checked the work. Still seems like a long shot. Typically we ask, "Cui bono?"
    - vlovich123 5y ago
      The guy who’s now a senior executive at a very large public company making quite good money and was very well regarded in the field for a long time for making a breakthrough discovery?
  - tpoacher 5y ago
    This is only true of researchers whose career depends on fraud. In this case, if anything, an established researcher has everything to lose from such data, whereas the insurance company could give out flaky data for whatever reason and not care about it.
- vlovich123 5y ago
  Yeah… in cases like this it always seems to be the PI doing something shady. Notwithstanding your personal hatred for the insurance industry, I would bias my prior towards malfeasance on the PIs part, especially since his defense is basically the equivalent “I totally have a girlfriend, she just goes to a different school… in Canada. You don’t know her”. This is a smart CYA response and nothing more. When you hear the totality of the evidence (primarily that he kept promoting this study after he 100% knew it didn’t replicate, he was investigating other similar kinds of effects that would have to be dead ends), it paints a clearer picture of him as a researcher.
- DantesKite 5y ago
  The thing is Dan has a history of lying and exaggerating results, specifically around this idea of nudging peoples behaviors. https://www.buzzfeednews.com/article/stephaniemlee/dan-ariely-honesty-study-retraction https://www.buzzfeednews.com/article/stephaniemlee/dan-ariel...
wdevanny 5y ago
While it does seem like the insurance company is most likely "at fault" for the fraudulent data, how did none of the researchers catch this? My understanding is that even looking at a histogram of the mileages should have raised serious doubts about the data.
- emblaegh 5y ago
  > how did none of the researchers catch this That's so, so easy to happen, specially 10 years ago. If they're not already primed to think a dataset might be fraudulent, a researcher might likely just think they've won the statistical signal lottery.
deleted 5y ago
[deleted]
sjakek 5y ago
Having had to come up with simulated datasets before, whoever faked this data clearly had no understanding of how the random number generator in excel/elsewhere would work. It is EXTREMELY easy to generate a normally distributed dataset, with a zero lower bound. Literally one line of code/formulas in excel e.g., =max(NORMINV(RAND(),10000,5000),0). We learned it in the first three weeks of my (graduate) intro stats class. You could then apply that formula to however many data points you needed, and in this case pass in a well known mean for # of miles driven per year. It's been 6 years since I took that class and I still remember the formula. I suppose it's possible that Ariely, a professor with decades of years of experience as a researcher, who had to have taken more stats classes than me (literally one) to get his PhD, would be dumb enough to use a random # generator. But that is an awfully, awfully stats illiterate mistake to make. Certainly, his team would have something to gain by a fascinating finding, though at this point Ariely already had two massively successful books prior to this finding. But the analyst at XYZ insurance that didnt want to be bothered to pull an actual dataset from SQL or oracle also couldve just been lazy...

by chirau 5y ago

This is Dan Ariely's response to the case: http://datacolada.org/storage_strong/DanBlogComment_Aug_16_2021_final.pdf http://datacolada.org/storage_strong/DanBlogComment_Aug_16_2...

deleted 5y ago

[deleted]

warent 5y ago

This is very helpful. Assuming Dan is being honest, it seems pretty obvious the insurance company is at fault here. It's so hard to know though. He could be covering for the whole group, or a teammate, or even himself. Frankly I'm fine believing it was the insurance company though. Auto insurance has already screwed me before and now I have an irrational and emotionally charged inclination to view them as villain

drno123 5y ago
However, car insurance company has nothing to gain from faking the numbers. Researches who publish an influential study that gets a lot of citations, and because of it they can get tenure, big contracts or lucrative job offers, have everything to gain from faking numbers.
- asdfasgasdgasdg 5y ago
  That's a good point, but it could be that the person tasked with gathering the numbers had something to gain (their own time, if they felt they could fudge and still get paid), and nobody checked the work. Still seems like a long shot. Typically we ask, "Cui bono?"
  - vlovich123 5y ago
    The guy who’s now a senior executive at a very large public company making quite good money and was very well regarded in the field for a long time for making a breakthrough discovery?
- tpoacher 5y ago
  This is only true of researchers whose career depends on fraud. In this case, if anything, an established researcher has everything to lose from such data, whereas the insurance company could give out flaky data for whatever reason and not care about it.
vlovich123 5y ago
Yeah… in cases like this it always seems to be the PI doing something shady. Notwithstanding your personal hatred for the insurance industry, I would bias my prior towards malfeasance on the PIs part, especially since his defense is basically the equivalent “I totally have a girlfriend, she just goes to a different school… in Canada. You don’t know her”. This is a smart CYA response and nothing more. When you hear the totality of the evidence (primarily that he kept promoting this study after he 100% knew it didn’t replicate, he was investigating other similar kinds of effects that would have to be dead ends), it paints a clearer picture of him as a researcher.
DantesKite 5y ago
The thing is Dan has a history of lying and exaggerating results, specifically around this idea of nudging peoples behaviors. https://www.buzzfeednews.com/article/stephaniemlee/dan-ariely-honesty-study-retraction https://www.buzzfeednews.com/article/stephaniemlee/dan-ariel...

wdevanny 5y ago

While it does seem like the insurance company is most likely "at fault" for the fraudulent data, how did none of the researchers catch this? My understanding is that even looking at a histogram of the mileages should have raised serious doubts about the data.

emblaegh 5y ago
> how did none of the researchers catch this That's so, so easy to happen, specially 10 years ago. If they're not already primed to think a dataset might be fraudulent, a researcher might likely just think they've won the statistical signal lottery.

deleted 5y ago

[deleted]

sjakek 5y ago

Having had to come up with simulated datasets before, whoever faked this data clearly had no understanding of how the random number generator in excel/elsewhere would work. It is EXTREMELY easy to generate a normally distributed dataset, with a zero lower bound. Literally one line of code/formulas in excel e.g., =max(NORMINV(RAND(),10000,5000),0). We learned it in the first three weeks of my (graduate) intro stats class. You could then apply that formula to however many data points you needed, and in this case pass in a well known mean for # of miles driven per year. It's been 6 years since I took that class and I still remember the formula. I suppose it's possible that Ariely, a professor with decades of years of experience as a researcher, who had to have taken more stats classes than me (literally one) to get his PhD, would be dumb enough to use a random # generator. But that is an awfully, awfully stats illiterate mistake to make. Certainly, his team would have something to gain by a fascinating finding, though at this point Ariely already had two massively successful books prior to this finding. But the analyst at XYZ insurance that didnt want to be bothered to pull an actual dataset from SQL or oracle also couldve just been lazy...