4 ms·

It's funny. When you first do work, you want the experiment to satisfy your hypothesis. When you are building on work, you also want the replication to succeed.

by jmugan 2y ago

It's funny. When you first do work, you want the experiment to satisfy your hypothesis. When you are building on work, you also want the replication to succeed. But when it is a famous result like this, you actually want it to fail so people talk about your result. There are uncountable ways that these experiments can be unconsciously and subtly affected by the desire of the experimenter. As an aside, I believe one interesting confounder in the marshmallow test is that it tests more (or at least as much) the subject's trust that the eventual reward will actually be given as it does the subject's ability to wait for the reward. So if you live in an unpredictable environment, it's better to just eat it.

readthenotes1 2y ago
I recall that your confounder was used as an explanation to wave away the air reproducibility of the marshmallow test, but I do not recall anyone actually ever testing that. This recent article seems to indicate that it's all just horse feathers and so you can make up any confounder you want to explain it away...
cryptoegorophy 2y ago
As someone who lived in “unpredictable environment “ in the 90s in Russia the correct statement would be: So if you live in an unpredictable environment, it's better to just move to a predictable environment. I saw a lot more people saving for the future rather than spending it all, which I surprisingly found the other way around in Canada, which is a predictable environment.
- zanellato19 2y ago
  > As someone who lived in “unpredictable environment “ in the 90s in Russia the correct statement would be: So if you live in an unpredictable environment, it's better to just move to a predictable environment. I think _everyone_ would take that opportunity if it was presented. Or at least most people.
- jkolio 2y ago
  Your environment was predictable, in that savings up enough would likely give you the option to move out. There are places where saving up enough money to leave your unpredictable environment unpredictably ends with you drowned in the Mediterranean, or in limbo short of your intended destination, or shipped back to where you spent so long trying to leave, in addition to the "success" case.
  - cryptoegorophy 2y ago
    I guess it depends on the meaning of predictable? We couldn’t predict 1 usd = 5rub would turn into 1usd =25rub almost overnight. My parents had friends in business killed (sadly not predictably) so it was unpredictable when it would be their turn.
- __jonas 2y ago
  Kind of interesting, because what happened in Russia in the 90s specifically with privatisation of UDSSR assets through vouchers is almost comparable to the marshmallow test on a superficial level.. > The vouchers, each corresponding to a share in the national wealth, were distributed equally among the population, including minors. They could be exchanged for shares in the enterprises to be privatized. Because most people were not well-informed about the nature of the program or were very poor, they were quick to sell their vouchers for money, unprepared or unwilling to invest. [1] [1] https://en.m.wikipedia.org/wiki/Privatization_in_Russia https://en.m.wikipedia.org/wiki/Privatization_in_Russia
kromem 2y ago
Definitely happens from time to time. When I took a look at a frequently cited paper 'disproving' Dunning-Kreuger, I was surprised by just how god awful the methodology actually was: https://www.lesswrong.com/posts/6Tqm8Jet9mzo6buj9/the-dunning-kruger-of-disproving-dunning-kruger https://www.lesswrong.com/posts/6Tqm8Jet9mzo6buj9/the-dunnin...
matthewdgreen 2y ago
>But when it is a famous result like this, you actually want it to fail so people talk about your result. There are finite resources for replication and so those resources must be allocated. High-profile results tend to attract good and skeptical replication attempts. This has always seemed like a pretty good approach to me. But replication takes time, and some people think it's a catastrophe that "bad" results don't immediately get corrected.

by jmugan 2y ago

readthenotes1 2y ago

I recall that your confounder was used as an explanation to wave away the air reproducibility of the marshmallow test, but I do not recall anyone actually ever testing that. This recent article seems to indicate that it's all just horse feathers and so you can make up any confounder you want to explain it away...

cryptoegorophy 2y ago

As someone who lived in “unpredictable environment “ in the 90s in Russia the correct statement would be: So if you live in an unpredictable environment, it's better to just move to a predictable environment. I saw a lot more people saving for the future rather than spending it all, which I surprisingly found the other way around in Canada, which is a predictable environment.

zanellato19 2y ago
> As someone who lived in “unpredictable environment “ in the 90s in Russia the correct statement would be: So if you live in an unpredictable environment, it's better to just move to a predictable environment. I think _everyone_ would take that opportunity if it was presented. Or at least most people.
jkolio 2y ago
Your environment was predictable, in that savings up enough would likely give you the option to move out. There are places where saving up enough money to leave your unpredictable environment unpredictably ends with you drowned in the Mediterranean, or in limbo short of your intended destination, or shipped back to where you spent so long trying to leave, in addition to the "success" case.
- cryptoegorophy 2y ago
  I guess it depends on the meaning of predictable? We couldn’t predict 1 usd = 5rub would turn into 1usd =25rub almost overnight. My parents had friends in business killed (sadly not predictably) so it was unpredictable when it would be their turn.
__jonas 2y ago
Kind of interesting, because what happened in Russia in the 90s specifically with privatisation of UDSSR assets through vouchers is almost comparable to the marshmallow test on a superficial level.. > The vouchers, each corresponding to a share in the national wealth, were distributed equally among the population, including minors. They could be exchanged for shares in the enterprises to be privatized. Because most people were not well-informed about the nature of the program or were very poor, they were quick to sell their vouchers for money, unprepared or unwilling to invest. [1] [1] https://en.m.wikipedia.org/wiki/Privatization_in_Russia https://en.m.wikipedia.org/wiki/Privatization_in_Russia

kromem 2y ago

Definitely happens from time to time. When I took a look at a frequently cited paper 'disproving' Dunning-Kreuger, I was surprised by just how god awful the methodology actually was: https://www.lesswrong.com/posts/6Tqm8Jet9mzo6buj9/the-dunning-kruger-of-disproving-dunning-kruger https://www.lesswrong.com/posts/6Tqm8Jet9mzo6buj9/the-dunnin...

matthewdgreen 2y ago

>But when it is a famous result like this, you actually want it to fail so people talk about your result. There are finite resources for replication and so those resources must be allocated. High-profile results tend to attract good and skeptical replication attempts. This has always seemed like a pretty good approach to me. But replication takes time, and some people think it's a catastrophe that "bad" results don't immediately get corrected.