6 ms·

Isn't this just one specific analysis using a very narrow definition of "nudge", one that doesn't even begin to encompass the work being done at those "300 "res

Isn't this just one specific analysis using a very narrow definition of "nudge", one that doesn't even begin to encompass the work being done at those "300 "research" departments"? Further, isn't this using the same data from the original meta analysis that did find "of small to medium size" effect[0]? Why would this, alone, undo decades of research and clear, bright-line conclusions such as the ones cited in my sibling comments? In other words, why is this letter the final word on the topic of "nudge", to you, and not the original meta-analysis? Sounds like you think everyone should pack it up and go home, all because of one letter using an alternative set of definitions and analysis. Just seems like an overreaction on your part, especially given how vocal and... you-sounding (for lack of a better term) the "anti-nudge" crowd often is. To take a wider view, a comment like yours is a more malicious form of nerd-sniping[1], especially on HN. Claim to have relevant credentials, voice a contrarian-but-popular-here opinion, and make a wild conclusion to give those reading it a feeling of "inside baseball." [0] https://www.pnas.org/doi/full/10.1073/pnas.2107346118#sec-3 https://www.pnas.org/doi/full/10.1073/pnas.2107346118#sec-3 [1] https://xkcd.com/356/ https://xkcd.com/356/

pizzathyme 4y ago
I agree, I have worked at 5 separate tech companies and have conducted hundreds of statistically significant experiments, changing what people select by default. These methods have been effective at helping hundreds of millions of people improve choices. It’s common practice, think: pricing on Amazon, default tips on Uber, default purchase price in a video game, etc. I don’t see how this would make me reinterpret all those successful results. Maybe I don’t understand what this is saying.
- ad8e 4y ago
  One interpretation is that it is very hard to extract value from the nudge literature. When reading research articles, one must estimate and adjust for biases. This adjustment shifts the p-values by unclear amounts. So a positive result may just be a fluke. As for your own AB tests, you have seen the processes that go into them and do not need to adjust for unknown biases. So when they demonstrate a nudge effect, you can believe it.
- efitz 4y ago
  How much of this is “nudging” vs. “clearly explaining trade-offs”? I’ve not done any rigorous research, but I’ve participated in projects that resulted in dramatic shifts towards customers choosing what the dev team thought was the “best” outcome, just by altering wording, or making “dangerous” choices harder (such as by requiring more clicks to enable).
- salawat 4y ago
  All you've proven is that controlling the default is a viable means of making a decision for someone. I assure you, Opt-outs are much less likely to happen when combined with not explaining there is a choice to be made in the first place.
sacrosancty 4y ago
I think social scientists have lost the right to the benefit of the doubt. They don't preregister their trials. They don't publish their negative results. They're notoriously bad at statistics, notoriously let their political beliefs distort their conclusions, notoriously scatter their work over thousands of arbitrary, hard-to-compare small-sample-size studies instead of concentrating their resources. I don't think it matters if this criticism is right or not. The fact that it's even possible to make such a criticism is already a condemnation of the scientific incompetence of social scientists.
- TameAntelope 4y ago
  > I don't think it matters if this criticism is right or not. Okay, you and I care about very different things.
  - harry8 4y ago
    >> I don't think it matters if this criticism is right or not. > Okay, you and I care about very different things. Clearly it is being suggested that it doesn't matter with respect to the social studies departments being in dire need of drastic reform. If you don't care about that either direction why are you commenting on this thread? You do actually care one way and you're "point scoring" to further the argument? Something else? I'm misreading something that I think I'm reading clearly?
    - TameAntelope 4y ago
      Hm? I just think it matters if the letter (the original submission) is correct or not.
      - harry8 4y ago
        So are social studies departments and funding in dire need of reform in your opinion? How does the "correctness" in your view of the original submission affect that need or non-need?

pizzathyme 4y ago

I agree, I have worked at 5 separate tech companies and have conducted hundreds of statistically significant experiments, changing what people select by default. These methods have been effective at helping hundreds of millions of people improve choices. It’s common practice, think: pricing on Amazon, default tips on Uber, default purchase price in a video game, etc. I don’t see how this would make me reinterpret all those successful results. Maybe I don’t understand what this is saying.

ad8e 4y ago
One interpretation is that it is very hard to extract value from the nudge literature. When reading research articles, one must estimate and adjust for biases. This adjustment shifts the p-values by unclear amounts. So a positive result may just be a fluke. As for your own AB tests, you have seen the processes that go into them and do not need to adjust for unknown biases. So when they demonstrate a nudge effect, you can believe it.
efitz 4y ago
How much of this is “nudging” vs. “clearly explaining trade-offs”? I’ve not done any rigorous research, but I’ve participated in projects that resulted in dramatic shifts towards customers choosing what the dev team thought was the “best” outcome, just by altering wording, or making “dangerous” choices harder (such as by requiring more clicks to enable).
salawat 4y ago
All you've proven is that controlling the default is a viable means of making a decision for someone. I assure you, Opt-outs are much less likely to happen when combined with not explaining there is a choice to be made in the first place.

sacrosancty 4y ago

I think social scientists have lost the right to the benefit of the doubt. They don't preregister their trials. They don't publish their negative results. They're notoriously bad at statistics, notoriously let their political beliefs distort their conclusions, notoriously scatter their work over thousands of arbitrary, hard-to-compare small-sample-size studies instead of concentrating their resources. I don't think it matters if this criticism is right or not. The fact that it's even possible to make such a criticism is already a condemnation of the scientific incompetence of social scientists.

TameAntelope 4y ago
> I don't think it matters if this criticism is right or not. Okay, you and I care about very different things.
- harry8 4y ago
  >> I don't think it matters if this criticism is right or not. > Okay, you and I care about very different things. Clearly it is being suggested that it doesn't matter with respect to the social studies departments being in dire need of drastic reform. If you don't care about that either direction why are you commenting on this thread? You do actually care one way and you're "point scoring" to further the argument? Something else? I'm misreading something that I think I'm reading clearly?
  - TameAntelope 4y ago
    Hm? I just think it matters if the letter (the original submission) is correct or not.
    - harry8 4y ago
      So are social studies departments and funding in dire need of reform in your opinion? How does the "correctness" in your view of the original submission affect that need or non-need?