3 ms·

Nothing Scales

41 points by raviparikh 5y ago

Retric 5y ago
Several different examples of why details matter. People respond differently to disease testing vs vaccination, it’s not just one big bucket of medical stuff. “However, we rarely actually run the original intervention at larger scale. Instead, the tendency is to water it down, which can make things significantly less effective. Case in point: replicating an effective education intervention in Uganda in more schools yielded virtually-identical results, whereas a modified program that tried to simulate how policymakers would reduce costs was substantially worse. That’s the theory that Evidence Action favors for why No Lean Season didn’t work at scale—they think the implementation changed in important ways.“
flerchin 5y ago
Along the same lines as the Individuals Matter conversation from yesterday. https://news.ycombinator.com/item?id=29235257 https://news.ycombinator.com/item?id=29235257
tuatoru 5y ago
One intervention scales: giving money, i.e. the power to choose. A HIV test may not be as important right now as getting that rotten tooth pulled, or getting clothes for a child so she can attend school, or getting food, or fixing the water pump or replacing the mosquito net. Every person is in their own situation and doing anything other than empowering them is paternalistic.
mikewarot 5y ago
It seems to me it's not scaling that is the problem, its getting a replication to happen in the first place. The original program is tailored to the local society, there have to be adjustments made any time you move somewhere else.
- selestify 5y ago
  Isn't that part of scaling? Getting the same thing to replicate through time and space?
- Out_of_Characte 5y ago
  But if you make adjustments then its no longer replication that you're doing.
- pbronez 5y ago
  The article addresses that directly: "Note that this is a very different challenge from the “replication crisis” that has most famously plagued social psychology. The average treatment effect of appointments in our study matches the one in the other study I mentioned above, and the original study that motivated No Lean Season literally contains a second RCT that, in part, replicates the main result."

41 points by raviparikh 5y ago

Several different examples of why details matter. People respond differently to disease testing vs vaccination, it’s not just one big bucket of medical stuff. “However, we rarely actually run the original intervention at larger scale. Instead, the tendency is to water it down, which can make things significantly less effective. Case in point: replicating an effective education intervention in Uganda in more schools yielded virtually-identical results, whereas a modified program that tried to simulate how policymakers would reduce costs was substantially worse. That’s the theory that Evidence Action favors for why No Lean Season didn’t work at scale—they think the implementation changed in important ways.“

flerchin 5y ago

Along the same lines as the Individuals Matter conversation from yesterday. https://news.ycombinator.com/item?id=29235257 https://news.ycombinator.com/item?id=29235257

tuatoru 5y ago

One intervention scales: giving money, i.e. the power to choose. A HIV test may not be as important right now as getting that rotten tooth pulled, or getting clothes for a child so she can attend school, or getting food, or fixing the water pump or replacing the mosquito net. Every person is in their own situation and doing anything other than empowering them is paternalistic.

mikewarot 5y ago

It seems to me it's not scaling that is the problem, its getting a replication to happen in the first place. The original program is tailored to the local society, there have to be adjustments made any time you move somewhere else.

selestify 5y ago
Isn't that part of scaling? Getting the same thing to replicate through time and space?
Out_of_Characte 5y ago
But if you make adjustments then its no longer replication that you're doing.
pbronez 5y ago
The article addresses that directly: "Note that this is a very different challenge from the “replication crisis” that has most famously plagued social psychology. The average treatment effect of appointments in our study matches the one in the other study I mentioned above, and the original study that motivated No Lean Season literally contains a second RCT that, in part, replicates the main result."