8 ms·

Evidence of Fraud in an Influential Study About Procrastination

365 points by Anon84 1mo ago

herewulf 1mo ago
I came here for a procrastination joke comment and didn't find it, but I suppose someone will get around to it eventually.
- peterbonney 1mo ago
  I’ll do it later
- toast0 1mo ago
  Here's one. I always leave my procrastination for later.
- lelanthran 1mo ago
  > I came here for a procrastination joke comment and didn't find it, but I suppose someone will get around to it eventually. Too soon; they'll all do it later.
fabiensanglard 1mo ago
Oh the irony...
keyle 1mo ago
I'm glad the authors seem to have acknowledged the problem and not doing much about it :-)
- fn-mote 1mo ago
  Per the post, the original authors have requested that the original paper be retracted. That’s well above the bar of “not doing much”.
  - dash2 1mo ago
    Actually the post is ambiguous: did the authors want the original article retracted, or the replication by Hyndman and Bisin? I hope the former! "Finally, it should be noted that when we shared these posts with Ariely and Wertenbroch a few weeks ago, they reached out to Psychological Science to request that the article be retracted. As of this writing, that process is ongoing."
    - fn-mote 1mo ago
      The post is unambiguous if you make it to the very end, where there are particular statements from each of the people involved. Look for: > Klaus Wertenbroch […] indicated that he has asked the editor to retract the paper.
      - oldgradstudent 1mo ago
        That's pathetic. How did he agree to be a co-author when "he never had access to the data for any of the studies"?
hnacct2001 1mo ago
Having skimmed this article and the replication study itself (which references some studies that complicate our common understanding of the positive effects of deadlines) I’m left wondering what the constructive alternative is to deadlines and/or how deadlines should be maintained or determined.
- bitwize 1mo ago
  There is no alternative to deadlines because time is money, and those downstream of you can ill afford to wait on your deliverables. Part of management's job is to determine how much time it should take so a reasonable timetable that stakeholders can agree to can be set, and then hold you accountable for delivering within that time frame. I wish we could revive Tim Bryce's mind with AI, because I'd love to hear what he'd have to say in a podcast about this topic: "Fraud in a procrastination study? You don't say! I don't understand why all this taxpayer money is going to study procrastination in the first place. Procrastination is a well-known phenomenon with a well-understood cause: laziness. Procrastinators didn't have a boot placed to their behinds growing up, so they reached adulthood slacking and coasting on their trust-fund money or couch-surfing at their friends' places, mooching off others. You see this a lot in modern software organizations, especially among the programmers. And now those programmers are all crying because they're about to get replaced by Claude Code. Boo hoo! The military knew how to deal with procrastinators, too: send them to scrub latrines with a toothbrush, and drum them out if they didn't shape up after that! We don't need millions of dollars in government grants going to academic research to study this phenomenon. It's as old as humanity itself—we called it the deadly sin of sloth. Friends, keep the faith!"
  - bluefirebrand 1mo ago
    > I wish we could revive Tim Bryce's mind with AI, because I'd love to hear what he'd have to say in a podcast about this topic: If he's anything like how the rest of your post read, he sounds like a pretty awful person.
    - bitwize 1mo ago
ProjectArcturis 1mo ago
Dan Ariely remaining at Duke, after it has become clear that much if not most of his work is fraudulent, is at least as big a black mark for Duke as Jason Arday was for Cambridge. Not to mention the Epstein ties.
- FiddlyPack 1mo ago
  Dan Ariely was one of the biggest disappointments and betrayals of my intellectual life. I found the "research" he did to be mind-blowing, groundbreaking stuff at the time, and I told many people his work, sharing his papers and podcasts as a true enthusiast. I am astonished that he holds an academic-adjacent position anywhere, no matter what his regrets might be today. Private redemption shouldn't be out of reach, but it shouldn't be possible for him to hold a position at any institution concerned with uncovering Truth. He should have resigned out of respect, if his institution was too gutless to do so. It's a tragedy.
- dash2 1mo ago
  It's arguably worse. Jason Arday was doing qualitative work in education. Nobody pays any attention to anything UK educationalists write. Dan Ariely was a genuine superstar and writer of popular bestsellers (with academic content).
  - suddenlybananas 1mo ago
    I agree Ariely was more influential and his fraud more flagrant (obviously Arday's lies were more absurd, but they were less academically relevant), but it's not true that Arday had no attention given to him. He did provide advice to government groups on policy [1,2]. Furthermore, the high publicity around his story about being non-verbal until he was 11 probably gave a lot of false hope to people with non-verbal children. [1] https://committees.parliament.uk/oralevidence/2482/html/ https://committees.parliament.uk/oralevidence/2482/html/ [2] https://faculty.educ.cam.ac.uk/230223-jason-arday https://faculty.educ.cam.ac.uk/230223-jason-arday
Aurornis 1mo ago
Dan Ariely has a long history of controversies and studies with falsified data. He wrote 3 popular books and gained a lot of fame before others started uncovering the problems in his studies https://en.wikipedia.org/wiki/Dan_Ariely https://en.wikipedia.org/wiki/Dan_Ariely It’s amazing that problems with his studies are still being uncovered. Equally amazing that Duke still maintains a relationship with him after so many problems have been identified with his research.
- Drupon 1mo ago
  [flagged]
- p-e-w 1mo ago
  > In 2006, when he was a professor at the MIT Media Lab, Ariely conducted experiments including administering electric shocks with a research assistant who had no human–subject training. […] Ariely confirmed this and said that he wasn't aware that the research assistant did not have the needed one-hour online human–subject training. A one-hour online training is all that’s needed to be considered ready to conduct experiments involving things like electric shocks on human subjects? Forget about this guy, the entire field needs to be burned to the ground and rebuilt from scratch.
  - paytonjjones 1mo ago
    I don't know the details here but I am a psychologist. The "electric shock" normally used in psych studies is tiny, it's like the equivalent of one of those little toy prank buzzers. You couldn't possibly hurt someone with it, this is really a non-issue. Related side note: Electroconvulsive therapy uses much larger electric shocks but, administered safely by a clinic, is extremely safe, has relatively minor side effects compared to e.g. SSRIs, and is very effective for severe depression. It's a huge shame that it's undeserved reputation prevents it from being more popular. Blame "One Flew Over the Cuckoo's Nest" for its inaccurate portrayal.
    - Waterluvian 1mo ago
bawolff 1mo ago
It does seem like its way too easy to do this kind of fraud. The study seems extremely easy to replicate (no fancy equipment needed, just ask some people to proofread), and yet nobody did until now. Is there some way to change incentives to get people to try and replicate? Assign undergrads to replicate random research papers?
- p-e-w 1mo ago
  > Is there some way to change incentives to get people to try and replicate? Sure. Pay them to do it.
  - tempestn 1mo ago
    If the culture shifted such that people were skeptical of original results but excited about replication, replication would be the big news and people would want to do it. How to get there, I don't know.
    - ChrisMarshallNY 1mo ago
      I seem to remember a lot of hype about room-temperature superconductors, that croaked upon [lack of] replication. That wasn't too long ago. Before that, cold fusion. We still have the ion drive that doesn't require fuel. I don't think it has ever been successfully replicated, but I think that it also hasn't been successfully repudiated, either. This kind of thing has been going on for a long time. I suspect that not all of it is deliberate fraud, but rather, people wanting their preconceived bias to be satisfied.
      - unloader6118 1mo ago
        Replicating superconductor experiment gives you the conductor and the skill. Both of them are useful as is. Replicating a social experiment, when successful, don't give you anything new.
    - qsera 1mo ago
      Then it "replication" becomes a target and thus would be gamed instead. This is a constant tug of war..
reader9274 1mo ago
[flagged]
CurbStomper 1mo ago
[dead]
freediddy 1mo ago
No paper should ever be referenced until it has been replicated by an independent group. The replication crisis in science is real, and it won't take long for no one to believe anything anymore.
- Polizeiposaune 1mo ago
  That rule would make it difficult to report on a failure to replicate a published paper.
  - freediddy 1mo ago
    Every paper needs to include an independent replication of the study.
    - monocasa 1mo ago
      If only we had that level of science funding.
    - sethammons 1mo ago
      So build a second particle collider to validate what comes out of CERN?
  - hirvi74 1mo ago
    Does some sort of batched style of research already exist? By batched, I mean a study where n number of teams all agree on a single methodology, and then conduct the same experiment simultaneously. Each team could then publish their results independently and the teams could publish one aggregate. I know meta studies attempt to do this in a way, but they can sometimes have their own problems too. I cannot recall coming across this style of research in much of what I have read as a hobby, but maybe it is more common in some domains than others? I am not smart, and there is nothing new under the Sun, so surely something like this exists, no?
    - Quekid5 1mo ago
ghm2199 1mo ago
The reason why people(or ai agents) have not been able to systematically destroy bad research in fields where fraud is so rampant is because they data are not published or easily accessible without emailing the author and asking them. I sincerely hope that any self respectful journal that publishes papers should have a minimum bar: submit the data on a public link with no paywall or blockers and feature the link prominently on your paper.
- alansaber 1mo ago
  There is a lot of bad ML research for the same reason.
- mike_hearn 1mo ago
  In this case even emailing the author didn't work. The data was obtained via some roundabout method. But it's not the real reason IMHO. You could use LLMs to write takedowns like this one. I gave ChatGPT the paper PDF and some neutral prompts. Although it didn't go straight to 'this is clearly fraud' - it's too nice for that - it did compute and point out the absurdly huge effect sizes, and give a relatively low confidence the paper's central claim would replicate. The real reason there aren't bots systematically destroying papers is because universities defend fraud, they don't punish it. You'd just be burning tokens to reveal lots of fraud with no outcome.
grebc 1mo ago
The name of the place should be enough of a giveaway for any thinking person Center for Advanced Hindsight. SMH.
csnover 1mo ago
Short of blindly outsourcing the work to an LLM and checking to see if its hallucinated problems are actually problems[0], how can more laymen (without the ability to run replication studies, and without degrees in statistics) learn to more easily identify questionable/potentially fraudulent research like this? This is a question that has been really bothering me for a while. I have a short list of things I’ve learnt to check, but it’s all stuff I’ve picked up in a very ad hoc way (reading and listening to these kinds of critical analyses, mostly). For example, until now, I wouldn’t think about implausibly large Cohen’s d, despite this seeming like an easily generalisable rule that anyone could eyeball. But without being told, and without the reference points given in the article, I wouldn’t have a clue. And I would like to have more of a clue. And I would like other people to have more of a clue. How? [0] https://news.ycombinator.com/item?id=49484925 https://news.ycombinator.com/item?id=49484925
mulanroo 1mo ago
One thing I didn't know before reading this: an unusually large effect size can itself be a red flag. You don't have to understand all the statistics to at least notice when a result looks almost too good to be true.
- recursivecaveat 1mo ago
  It is interesting how all these famous non-replicating psych paper results feel reasonably intuitive to me. Like I can buy that having a more structured and detailed deadline helps you procrastinate less. I can even buy that doing a goofy little "power pose" exercise kind of hypes yourself up to give some self confidence and do better on an interview or whatever. You would almost expect the opposite and that all the famous studies would be the ones that run counter to intuition. Maybe those get more replication attempts, or people feel less confident faking them. The direction is intuitive, but the effect sizes are, as you say, unusually large. I'm no statistician, but if you have an effect size that's several multiples of the standard deviation, that would mean that it dominates the natural variation in results right? So you're kind of making an implicit claim that other factors in play (for eg in this proofreading task, how busy someone is or their reading level) are peanuts in comparison. Further, if there are these relatively simple factors like how a deadline is expressed that make you several times better, you'd expect anybody who stumbles into several of them and stacks them together to be absolutely off the charts. Somebody who listens to Mozart, gets the right priming, doesn't deplete their ego, and receives short deadlines is suddenly like 50X more productive, which doesn't seem to match reality.
  - yayamo 1mo ago
    Your exact intuition about the implausibility of many large effect sizes is called the “Piranha problem” by my favourite statistician Andrew Gelman first here in a blog post: https://statmodeling.stat.columbia.edu/2017/12/15/eats/ https://statmodeling.stat.columbia.edu/2017/12/15/eats/ And then as a published paper: https://sites.stat.columbia.edu/gelman/research/published/piranha_ams_notices.pdf https://sites.stat.columbia.edu/gelman/research/published/pi... > how many explanatory variables can have large eﬀects on an outcome?
- 1mo ago
kazinator 1mo ago
They probably procrastinated all their deadlines and then fibbed things.
blfr 1mo ago
And yet it will live forever, just like Dunning-Kruger despite its statistical issues. I fully expect to see it cited during some workshop or other pointless training exercise.
sabot90260 1mo ago
Figures, even the studies about procrastination are fraught with issues. Guess I'll put off reading the details until tomorrow.
MarceliusK 1mo ago
[dead]
jarenmf 1mo ago
Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematics. I was excited to read it, but it didn't take long for me to find out that it's complete BS-derivations that don't follow, made-up equation terms, and conflation of basic terms in the field. I was really surprised that this passed the editor and at least two peer reviewers. I dug deeper and found that the same author has published 11 research articles in top journals in 2026 alone — mostly physics and chemistry, and adjacent fields— and I'm talking about really top journals like Scientific Reports and Physics Letters B. Most of these have serious mathematical flaws that are unfixable. I wrote to a couple of editors, but only one took it seriously enough to initiate an investigation. I made a promise to myself that if no real action is taken, I will seriously consider changing careers.
- Yokolos 1mo ago
  Even as imperfect as it is, it's infinitely better than the alternatives like belief in religion/religious texts/pastors or anybody else who is just making shit up based on whatever they think sounds good. None of that is falsifiable or makes any attempt to iteratively arrive at anything true or real. There is literally nothing better that we have other than the scientific method. Yes, science is self-correcting, even if it is slow or whatever else. Nothing else that we have can approach even that bar.
- dilawar 1mo ago
  > Science is self-correcting, but sometimes As long as people mislead or make mistake unknowingly, it's fine to rely to self-correcting mechanism of science. The institutional rot is much deeper. It is not good to leave all this fraud to self-correcting mechanisms of science. We also have "trust" to worry about.
  - anthonyrstevens 1mo ago
    Maybe instead of throwing around ideas of "institutional rot" we can realize that there are hundreds of thousands of people doing "science" in the right way, are not out for material gain, do not engage in fraud, and are interested in truth and scientific advancement. The fact that there are a vanishingly small # of cases where bad things happen in the process are the exceptions that prove the rule, IMHO. Certain segments of society like to hype these up (or even lie about their frequency) in order to discredit the scientific method. "Institutional rot" is a meme that needs to go away. Identify the problems at the edges, fix them, and move on to letting the bulk of the work continue as it always has - quietly building upon previous valid results to the betterment of society.
nobrains 1mo ago
this image https://datacolada.org/wp-content/uploads/scatter_like_vs_interesting_partial_original-1024x910.png https://datacolada.org/wp-content/uploads/scatter_like_vs_in... itself visually shows fake data. Too perfectly clustered.
crimsoneer 1mo ago
At the risk of bringing up a controversial topic, in the midst of the Jason Arday furore last month when everyone was like "this wouldn't happen to a white professor", the person who kept coming to mind for me was Ariely. I became a data scientist in no small part due to his early work, and seeing academia totally incapable of punishing him is deeply disappointing.
zhivota 1mo ago
I hope that with LLMs we'll see a lot faster identification of these anomalies. Imagine that this guy taught at MIT and has his own lab at Duke, basically more than 20 years of taking up some of the most prestigious positions in his field, all based on fraud. The butterfly effect this has on science as a practice is huge. Some marginal researcher who actually deserved it was bumped out of those places, and somewhere down the line someone was bumped all the way out of academia because of this guy's lies. And this is hardly the only person in this situation. What's scary is that the next fraudster will read this take down and see what they have to do to avoid detection. It wouldn't be that hard to form data that looks a lot more plausible. Replication may be the only solution, unfortunately it's so expensive and time consuming, without any of the actual benefits of doing the original study. Maybe it should be government funded, the US replication lab, its only mission to replicate the top X papers from some named fields each year.
- qmmmur 1mo ago
  They will be generated faster than they can be detected.
kodoman 1mo ago
People for a long time have had far too much faith in peer review as and had a dim view of anything other then experimental results and work, despite how much fraud and just bad experiments their are. Not to mention how many variables their might actually be to account for in something like psychology or any of the social science's. Deriving things from axioms and postulates alone is problematic but some one who can do both begins to actually do useful work in my opinion and some one who can accept the difficult their actually is in knowing anything about human psychology simply from studying it and underneath all experiments is assumptions and the people in the experiment who have so many variables. This is not like medicine human psychology is very flexible and for me though not a social scientist I feel of all theories of human psychology they are only applicable to people now and not even 300 years ago, though of course one must concede that the micro psychological principles might be applicable to humans more generally.
2b3a51 1mo ago
The last point in the OA about rounding the estimate of the time spent on the assigned tasks. Just wondering if the response would depend on the wording of the instructions to participants and the nature of the incentivisation. Me being a bit of a boy scout / Belbin 'company worker', I would tend to use a timer (I'm a pomodoro method person for things like marking or checking references &c). The rest, oh yes. I stopped paying much attention to studies based on dozens of undergraduates doing weird concocted tasks years ago.
dramm 1mo ago
Dan Ariely is well past the point that he needs to not be a faculty member at Duke University. Data Coda do a fantastic job ferreting out academic fraud. With these fraudulent behavioral science experiments like this relying on printed papers/markups, and coauthors not present and who did not even see the raw data, I wonder if the experiment was ever actually conducted. I would love to see that question answered where fraud is suspected, something I'd hope IRBs would step up to do.
black_13 1mo ago
[dead]
LeChuck 1mo ago
It's shocking to me how badly the data was falsified. You'd think a researcher who is A. well versed in statistics and B. has seen a lot of actual research data would be able to produce more believable data. Does this mean we're only seeing the tip of the iceberg? how many papers are based on hard to detect falsifications?
- neurotroph 1mo ago
  Why bother? Low effort data falsification is enough to land a high-ranking paper. Sure, really simulating data with some noise would be much more convincing but its just not necessary. Raw data is very rarely inspected during peer review, much less so after publication, and most reviewers do not address statistics — if they do, the usually refer to their pet model and/or use the same incorrect tools as most psychologists.
neilellis 1mo ago
I'll get around to reading this at some point.
Unified-Mentor 1mo ago
[dead]
largbae 1mo ago
So this paper appears to be fabricated, but was the conclusion actually wrong? I do appear to perform better on tasks with an external deadline than on self-set ones.
- qmmmur 1mo ago
  That’s not how science works
- jldugger 1mo ago
  The conclusion is faulty given that it was derived from faulty data. The 2002 paper had two pilot studies and 2 bigger studies. The replication of study 2 failed[1], and the original data has substantial concerning features the rest of the datacolada article lays out. This has two unfortunate implications. First, if study 2 was not necessary to support the conclusions, it seems likely it would not have been performed or included in this paper. So given it seems necessary, the conclusion is invalid. In past examples (the Reinhart-Rogoff paper comes to mind) when this happens, the authors claim it wasn't necessary and the conclusion is still valid and the professional embarrassment of a retraction is not called for. But in this case the replication failure might stand as a strike _against_ the theory. But second, this is not the first questionable data coming from Ariely's lab, and it seems unlikely this was a data entry mistake. If study 2 is not trustworthy, we should update our priors about the trustworthiness of study 1. Note it's not guaranteed to be doctored in some way, just worthy of additional scrutiny. And if that one also fails to replicate, the paper and its conclusion seems unsalvageable. Presumably his coauthor is now panicking about not keeping data from 25 years ago to exhonerate and distance himself. [1]: https://journals.sagepub.com/doi/full/10.1177/09567976261460772 https://journals.sagepub.com/doi/full/10.1177/09567976261460...
lol8675309 1mo ago
Sure. The guy put it off until the last minute, and then had to fake his way through it. It writes itself.
deleted 1mo ago
[deleted]
Bluestein 1mo ago
Procrastination: They couldn't be bothered.-
jsw97 1mo ago
There is such a smell around this whole literature. I wonder what it's like to be a grad student in social psychology right now. When the norm is surprising results, any responsible, competent researcher is going to realize that they can't do it honestly. So this leaves behind people who are willing to bend the rules, or people who don't see the problem with, for example, the garden of forking paths.
- neurotroph 1mo ago
  In good universities replication crisis and the underlying structures are taught. (Un?)surprisingly, not much has changed though. Except for some unis, some journals, and a bubble of rigorous research continuing to work on meta-science and replications. Have to admire their stamina. Source: did my PhD on exactly this topic
GstaEkman 1mo ago
[dead]
subarctic 1mo ago
There's a bit of irony that probably 50% of us are procrastinating while reading this
nkmnz 1mo ago
What should I do now with the 4+ books authored by Ariely that I have sitting on my shelf?
- Rendello 1mo ago
  Books on procrastination? Now you don't have to get around to reading them! (I jest)
silexia 1mo ago
"Trust the science" is one of the most anti-science phrases ever invented. The whole point of science is to challenge everything.

365 points by Anon84 1mo ago

I came here for a procrastination joke comment and didn't find it, but I suppose someone will get around to it eventually.

peterbonney 1mo ago
I’ll do it later
toast0 1mo ago
Here's one. I always leave my procrastination for later.
lelanthran 1mo ago
> I came here for a procrastination joke comment and didn't find it, but I suppose someone will get around to it eventually. Too soon; they'll all do it later.

fabiensanglard 1mo ago

Oh the irony...

keyle 1mo ago

I'm glad the authors seem to have acknowledged the problem and not doing much about it :-)

fn-mote 1mo ago
Per the post, the original authors have requested that the original paper be retracted. That’s well above the bar of “not doing much”.
- dash2 1mo ago
  Actually the post is ambiguous: did the authors want the original article retracted, or the replication by Hyndman and Bisin? I hope the former! "Finally, it should be noted that when we shared these posts with Ariely and Wertenbroch a few weeks ago, they reached out to Psychological Science to request that the article be retracted. As of this writing, that process is ongoing."
  - fn-mote 1mo ago
    The post is unambiguous if you make it to the very end, where there are particular statements from each of the people involved. Look for: > Klaus Wertenbroch […] indicated that he has asked the editor to retract the paper.
    - oldgradstudent 1mo ago
      That's pathetic. How did he agree to be a co-author when "he never had access to the data for any of the studies"?

hnacct2001 1mo ago

Having skimmed this article and the replication study itself (which references some studies that complicate our common understanding of the positive effects of deadlines) I’m left wondering what the constructive alternative is to deadlines and/or how deadlines should be maintained or determined.

bitwize 1mo ago
There is no alternative to deadlines because time is money, and those downstream of you can ill afford to wait on your deliverables. Part of management's job is to determine how much time it should take so a reasonable timetable that stakeholders can agree to can be set, and then hold you accountable for delivering within that time frame. I wish we could revive Tim Bryce's mind with AI, because I'd love to hear what he'd have to say in a podcast about this topic: "Fraud in a procrastination study? You don't say! I don't understand why all this taxpayer money is going to study procrastination in the first place. Procrastination is a well-known phenomenon with a well-understood cause: laziness. Procrastinators didn't have a boot placed to their behinds growing up, so they reached adulthood slacking and coasting on their trust-fund money or couch-surfing at their friends' places, mooching off others. You see this a lot in modern software organizations, especially among the programmers. And now those programmers are all crying because they're about to get replaced by Claude Code. Boo hoo! The military knew how to deal with procrastinators, too: send them to scrub latrines with a toothbrush, and drum them out if they didn't shape up after that! We don't need millions of dollars in government grants going to academic research to study this phenomenon. It's as old as humanity itself—we called it the deadly sin of sloth. Friends, keep the faith!"
- bluefirebrand 1mo ago
  > I wish we could revive Tim Bryce's mind with AI, because I'd love to hear what he'd have to say in a podcast about this topic: If he's anything like how the rest of your post read, he sounds like a pretty awful person.
  - bitwize 1mo ago

ProjectArcturis 1mo ago

Dan Ariely remaining at Duke, after it has become clear that much if not most of his work is fraudulent, is at least as big a black mark for Duke as Jason Arday was for Cambridge. Not to mention the Epstein ties.

FiddlyPack 1mo ago
Dan Ariely was one of the biggest disappointments and betrayals of my intellectual life. I found the "research" he did to be mind-blowing, groundbreaking stuff at the time, and I told many people his work, sharing his papers and podcasts as a true enthusiast. I am astonished that he holds an academic-adjacent position anywhere, no matter what his regrets might be today. Private redemption shouldn't be out of reach, but it shouldn't be possible for him to hold a position at any institution concerned with uncovering Truth. He should have resigned out of respect, if his institution was too gutless to do so. It's a tragedy.
dash2 1mo ago
It's arguably worse. Jason Arday was doing qualitative work in education. Nobody pays any attention to anything UK educationalists write. Dan Ariely was a genuine superstar and writer of popular bestsellers (with academic content).
- suddenlybananas 1mo ago
  I agree Ariely was more influential and his fraud more flagrant (obviously Arday's lies were more absurd, but they were less academically relevant), but it's not true that Arday had no attention given to him. He did provide advice to government groups on policy [1,2]. Furthermore, the high publicity around his story about being non-verbal until he was 11 probably gave a lot of false hope to people with non-verbal children. [1] https://committees.parliament.uk/oralevidence/2482/html/ https://committees.parliament.uk/oralevidence/2482/html/ [2] https://faculty.educ.cam.ac.uk/230223-jason-arday https://faculty.educ.cam.ac.uk/230223-jason-arday

Aurornis 1mo ago

Dan Ariely has a long history of controversies and studies with falsified data. He wrote 3 popular books and gained a lot of fame before others started uncovering the problems in his studies https://en.wikipedia.org/wiki/Dan_Ariely https://en.wikipedia.org/wiki/Dan_Ariely It’s amazing that problems with his studies are still being uncovered. Equally amazing that Duke still maintains a relationship with him after so many problems have been identified with his research.

Drupon 1mo ago
[flagged]
p-e-w 1mo ago
> In 2006, when he was a professor at the MIT Media Lab, Ariely conducted experiments including administering electric shocks with a research assistant who had no human–subject training. […] Ariely confirmed this and said that he wasn't aware that the research assistant did not have the needed one-hour online human–subject training. A one-hour online training is all that’s needed to be considered ready to conduct experiments involving things like electric shocks on human subjects? Forget about this guy, the entire field needs to be burned to the ground and rebuilt from scratch.
- paytonjjones 1mo ago
  I don't know the details here but I am a psychologist. The "electric shock" normally used in psych studies is tiny, it's like the equivalent of one of those little toy prank buzzers. You couldn't possibly hurt someone with it, this is really a non-issue. Related side note: Electroconvulsive therapy uses much larger electric shocks but, administered safely by a clinic, is extremely safe, has relatively minor side effects compared to e.g. SSRIs, and is very effective for severe depression. It's a huge shame that it's undeserved reputation prevents it from being more popular. Blame "One Flew Over the Cuckoo's Nest" for its inaccurate portrayal.
  - Waterluvian 1mo ago

bawolff 1mo ago

It does seem like its way too easy to do this kind of fraud. The study seems extremely easy to replicate (no fancy equipment needed, just ask some people to proofread), and yet nobody did until now. Is there some way to change incentives to get people to try and replicate? Assign undergrads to replicate random research papers?

p-e-w 1mo ago
> Is there some way to change incentives to get people to try and replicate? Sure. Pay them to do it.
- tempestn 1mo ago
  If the culture shifted such that people were skeptical of original results but excited about replication, replication would be the big news and people would want to do it. How to get there, I don't know.
  - ChrisMarshallNY 1mo ago
    I seem to remember a lot of hype about room-temperature superconductors, that croaked upon [lack of] replication. That wasn't too long ago. Before that, cold fusion. We still have the ion drive that doesn't require fuel. I don't think it has ever been successfully replicated, but I think that it also hasn't been successfully repudiated, either. This kind of thing has been going on for a long time. I suspect that not all of it is deliberate fraud, but rather, people wanting their preconceived bias to be satisfied.
    - unloader6118 1mo ago
      Replicating superconductor experiment gives you the conductor and the skill. Both of them are useful as is. Replicating a social experiment, when successful, don't give you anything new.
  - qsera 1mo ago
    Then it "replication" becomes a target and thus would be gamed instead. This is a constant tug of war..

reader9274 1mo ago

[flagged]

CurbStomper 1mo ago

[dead]

freediddy 1mo ago

No paper should ever be referenced until it has been replicated by an independent group. The replication crisis in science is real, and it won't take long for no one to believe anything anymore.

Polizeiposaune 1mo ago
That rule would make it difficult to report on a failure to replicate a published paper.
- freediddy 1mo ago
  Every paper needs to include an independent replication of the study.
  - monocasa 1mo ago
    If only we had that level of science funding.
  - sethammons 1mo ago
    So build a second particle collider to validate what comes out of CERN?
- hirvi74 1mo ago
  Does some sort of batched style of research already exist? By batched, I mean a study where n number of teams all agree on a single methodology, and then conduct the same experiment simultaneously. Each team could then publish their results independently and the teams could publish one aggregate. I know meta studies attempt to do this in a way, but they can sometimes have their own problems too. I cannot recall coming across this style of research in much of what I have read as a hobby, but maybe it is more common in some domains than others? I am not smart, and there is nothing new under the Sun, so surely something like this exists, no?
  - Quekid5 1mo ago

ghm2199 1mo ago

The reason why people(or ai agents) have not been able to systematically destroy bad research in fields where fraud is so rampant is because they data are not published or easily accessible without emailing the author and asking them. I sincerely hope that any self respectful journal that publishes papers should have a minimum bar: submit the data on a public link with no paywall or blockers and feature the link prominently on your paper.

alansaber 1mo ago
There is a lot of bad ML research for the same reason.
mike_hearn 1mo ago
In this case even emailing the author didn't work. The data was obtained via some roundabout method. But it's not the real reason IMHO. You could use LLMs to write takedowns like this one. I gave ChatGPT the paper PDF and some neutral prompts. Although it didn't go straight to 'this is clearly fraud' - it's too nice for that - it did compute and point out the absurdly huge effect sizes, and give a relatively low confidence the paper's central claim would replicate. The real reason there aren't bots systematically destroying papers is because universities defend fraud, they don't punish it. You'd just be burning tokens to reveal lots of fraud with no outcome.

grebc 1mo ago

The name of the place should be enough of a giveaway for any thinking person Center for Advanced Hindsight. SMH.

csnover 1mo ago

Short of blindly outsourcing the work to an LLM and checking to see if its hallucinated problems are actually problems[0], how can more laymen (without the ability to run replication studies, and without degrees in statistics) learn to more easily identify questionable/potentially fraudulent research like this? This is a question that has been really bothering me for a while. I have a short list of things I’ve learnt to check, but it’s all stuff I’ve picked up in a very ad hoc way (reading and listening to these kinds of critical analyses, mostly). For example, until now, I wouldn’t think about implausibly large Cohen’s d, despite this seeming like an easily generalisable rule that anyone could eyeball. But without being told, and without the reference points given in the article, I wouldn’t have a clue. And I would like to have more of a clue. And I would like other people to have more of a clue. How? [0] https://news.ycombinator.com/item?id=49484925 https://news.ycombinator.com/item?id=49484925

mulanroo 1mo ago

One thing I didn't know before reading this: an unusually large effect size can itself be a red flag. You don't have to understand all the statistics to at least notice when a result looks almost too good to be true.

recursivecaveat 1mo ago
It is interesting how all these famous non-replicating psych paper results feel reasonably intuitive to me. Like I can buy that having a more structured and detailed deadline helps you procrastinate less. I can even buy that doing a goofy little "power pose" exercise kind of hypes yourself up to give some self confidence and do better on an interview or whatever. You would almost expect the opposite and that all the famous studies would be the ones that run counter to intuition. Maybe those get more replication attempts, or people feel less confident faking them. The direction is intuitive, but the effect sizes are, as you say, unusually large. I'm no statistician, but if you have an effect size that's several multiples of the standard deviation, that would mean that it dominates the natural variation in results right? So you're kind of making an implicit claim that other factors in play (for eg in this proofreading task, how busy someone is or their reading level) are peanuts in comparison. Further, if there are these relatively simple factors like how a deadline is expressed that make you several times better, you'd expect anybody who stumbles into several of them and stacks them together to be absolutely off the charts. Somebody who listens to Mozart, gets the right priming, doesn't deplete their ego, and receives short deadlines is suddenly like 50X more productive, which doesn't seem to match reality.
- yayamo 1mo ago
  Your exact intuition about the implausibility of many large effect sizes is called the “Piranha problem” by my favourite statistician Andrew Gelman first here in a blog post: https://statmodeling.stat.columbia.edu/2017/12/15/eats/ https://statmodeling.stat.columbia.edu/2017/12/15/eats/ And then as a published paper: https://sites.stat.columbia.edu/gelman/research/published/piranha_ams_notices.pdf https://sites.stat.columbia.edu/gelman/research/published/pi... > how many explanatory variables can have large eﬀects on an outcome?
1mo ago

kazinator 1mo ago

They probably procrastinated all their deadlines and then fibbed things.

blfr 1mo ago

And yet it will live forever, just like Dunning-Kruger despite its statistical issues. I fully expect to see it cited during some workshop or other pointless training exercise.

sabot90260 1mo ago

Figures, even the studies about procrastination are fraught with issues. Guess I'll put off reading the details until tomorrow.

MarceliusK 1mo ago

[dead]

jarenmf 1mo ago

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematics. I was excited to read it, but it didn't take long for me to find out that it's complete BS-derivations that don't follow, made-up equation terms, and conflation of basic terms in the field. I was really surprised that this passed the editor and at least two peer reviewers. I dug deeper and found that the same author has published 11 research articles in top journals in 2026 alone — mostly physics and chemistry, and adjacent fields— and I'm talking about really top journals like Scientific Reports and Physics Letters B. Most of these have serious mathematical flaws that are unfixable. I wrote to a couple of editors, but only one took it seriously enough to initiate an investigation. I made a promise to myself that if no real action is taken, I will seriously consider changing careers.

Yokolos 1mo ago
Even as imperfect as it is, it's infinitely better than the alternatives like belief in religion/religious texts/pastors or anybody else who is just making shit up based on whatever they think sounds good. None of that is falsifiable or makes any attempt to iteratively arrive at anything true or real. There is literally nothing better that we have other than the scientific method. Yes, science is self-correcting, even if it is slow or whatever else. Nothing else that we have can approach even that bar.
dilawar 1mo ago
> Science is self-correcting, but sometimes As long as people mislead or make mistake unknowingly, it's fine to rely to self-correcting mechanism of science. The institutional rot is much deeper. It is not good to leave all this fraud to self-correcting mechanisms of science. We also have "trust" to worry about.
- anthonyrstevens 1mo ago
  Maybe instead of throwing around ideas of "institutional rot" we can realize that there are hundreds of thousands of people doing "science" in the right way, are not out for material gain, do not engage in fraud, and are interested in truth and scientific advancement. The fact that there are a vanishingly small # of cases where bad things happen in the process are the exceptions that prove the rule, IMHO. Certain segments of society like to hype these up (or even lie about their frequency) in order to discredit the scientific method. "Institutional rot" is a meme that needs to go away. Identify the problems at the edges, fix them, and move on to letting the bulk of the work continue as it always has - quietly building upon previous valid results to the betterment of society.

nobrains 1mo ago

this image https://datacolada.org/wp-content/uploads/scatter_like_vs_interesting_partial_original-1024x910.png https://datacolada.org/wp-content/uploads/scatter_like_vs_in... itself visually shows fake data. Too perfectly clustered.

crimsoneer 1mo ago

At the risk of bringing up a controversial topic, in the midst of the Jason Arday furore last month when everyone was like "this wouldn't happen to a white professor", the person who kept coming to mind for me was Ariely. I became a data scientist in no small part due to his early work, and seeing academia totally incapable of punishing him is deeply disappointing.

zhivota 1mo ago

I hope that with LLMs we'll see a lot faster identification of these anomalies. Imagine that this guy taught at MIT and has his own lab at Duke, basically more than 20 years of taking up some of the most prestigious positions in his field, all based on fraud. The butterfly effect this has on science as a practice is huge. Some marginal researcher who actually deserved it was bumped out of those places, and somewhere down the line someone was bumped all the way out of academia because of this guy's lies. And this is hardly the only person in this situation. What's scary is that the next fraudster will read this take down and see what they have to do to avoid detection. It wouldn't be that hard to form data that looks a lot more plausible. Replication may be the only solution, unfortunately it's so expensive and time consuming, without any of the actual benefits of doing the original study. Maybe it should be government funded, the US replication lab, its only mission to replicate the top X papers from some named fields each year.

qmmmur 1mo ago
They will be generated faster than they can be detected.

kodoman 1mo ago

People for a long time have had far too much faith in peer review as and had a dim view of anything other then experimental results and work, despite how much fraud and just bad experiments their are. Not to mention how many variables their might actually be to account for in something like psychology or any of the social science's. Deriving things from axioms and postulates alone is problematic but some one who can do both begins to actually do useful work in my opinion and some one who can accept the difficult their actually is in knowing anything about human psychology simply from studying it and underneath all experiments is assumptions and the people in the experiment who have so many variables. This is not like medicine human psychology is very flexible and for me though not a social scientist I feel of all theories of human psychology they are only applicable to people now and not even 300 years ago, though of course one must concede that the micro psychological principles might be applicable to humans more generally.

2b3a51 1mo ago

The last point in the OA about rounding the estimate of the time spent on the assigned tasks. Just wondering if the response would depend on the wording of the instructions to participants and the nature of the incentivisation. Me being a bit of a boy scout / Belbin 'company worker', I would tend to use a timer (I'm a pomodoro method person for things like marking or checking references &c). The rest, oh yes. I stopped paying much attention to studies based on dozens of undergraduates doing weird concocted tasks years ago.

dramm 1mo ago

Dan Ariely is well past the point that he needs to not be a faculty member at Duke University. Data Coda do a fantastic job ferreting out academic fraud. With these fraudulent behavioral science experiments like this relying on printed papers/markups, and coauthors not present and who did not even see the raw data, I wonder if the experiment was ever actually conducted. I would love to see that question answered where fraud is suspected, something I'd hope IRBs would step up to do.

black_13 1mo ago

[dead]

LeChuck 1mo ago

It's shocking to me how badly the data was falsified. You'd think a researcher who is A. well versed in statistics and B. has seen a lot of actual research data would be able to produce more believable data. Does this mean we're only seeing the tip of the iceberg? how many papers are based on hard to detect falsifications?

neurotroph 1mo ago
Why bother? Low effort data falsification is enough to land a high-ranking paper. Sure, really simulating data with some noise would be much more convincing but its just not necessary. Raw data is very rarely inspected during peer review, much less so after publication, and most reviewers do not address statistics — if they do, the usually refer to their pet model and/or use the same incorrect tools as most psychologists.

neilellis 1mo ago

I'll get around to reading this at some point.

Unified-Mentor 1mo ago

[dead]

largbae 1mo ago

So this paper appears to be fabricated, but was the conclusion actually wrong? I do appear to perform better on tasks with an external deadline than on self-set ones.

qmmmur 1mo ago
That’s not how science works
jldugger 1mo ago
The conclusion is faulty given that it was derived from faulty data. The 2002 paper had two pilot studies and 2 bigger studies. The replication of study 2 failed[1], and the original data has substantial concerning features the rest of the datacolada article lays out. This has two unfortunate implications. First, if study 2 was not necessary to support the conclusions, it seems likely it would not have been performed or included in this paper. So given it seems necessary, the conclusion is invalid. In past examples (the Reinhart-Rogoff paper comes to mind) when this happens, the authors claim it wasn't necessary and the conclusion is still valid and the professional embarrassment of a retraction is not called for. But in this case the replication failure might stand as a strike _against_ the theory. But second, this is not the first questionable data coming from Ariely's lab, and it seems unlikely this was a data entry mistake. If study 2 is not trustworthy, we should update our priors about the trustworthiness of study 1. Note it's not guaranteed to be doctored in some way, just worthy of additional scrutiny. And if that one also fails to replicate, the paper and its conclusion seems unsalvageable. Presumably his coauthor is now panicking about not keeping data from 25 years ago to exhonerate and distance himself. [1]: https://journals.sagepub.com/doi/full/10.1177/09567976261460772 https://journals.sagepub.com/doi/full/10.1177/09567976261460...

lol8675309 1mo ago

Sure. The guy put it off until the last minute, and then had to fake his way through it. It writes itself.

deleted 1mo ago

[deleted]

Bluestein 1mo ago

Procrastination: They couldn't be bothered.-

jsw97 1mo ago

There is such a smell around this whole literature. I wonder what it's like to be a grad student in social psychology right now. When the norm is surprising results, any responsible, competent researcher is going to realize that they can't do it honestly. So this leaves behind people who are willing to bend the rules, or people who don't see the problem with, for example, the garden of forking paths.

neurotroph 1mo ago
In good universities replication crisis and the underlying structures are taught. (Un?)surprisingly, not much has changed though. Except for some unis, some journals, and a bubble of rigorous research continuing to work on meta-science and replications. Have to admire their stamina. Source: did my PhD on exactly this topic

GstaEkman 1mo ago

[dead]

subarctic 1mo ago

There's a bit of irony that probably 50% of us are procrastinating while reading this

nkmnz 1mo ago

What should I do now with the 4+ books authored by Ariely that I have sitting on my shelf?

Rendello 1mo ago
Books on procrastination? Now you don't have to get around to reading them! (I jest)

silexia 1mo ago

"Trust the science" is one of the most anti-science phrases ever invented. The whole point of science is to challenge everything.