3 ms·

The massive increase seen in the control group seems to indicate that either ~5% of the money delivers >50% of the benefit, or there were some other factors tha

The massive increase seen in the control group seems to indicate that either ~5% of the money delivers >50% of the benefit, or there were some other factors that were more significant than the money.

munificent 3y ago
It seems to me like demand effects[1] could explain a large fraction of the observed behavior. People like being given money. If you tell them you're conducting a study to determine if giving them leads to them doing XYZ, it should come as no surprise when they do XYZ, even if the money didn't literally enable it: they are incentivized to XYZ all they can in order to keep the money pump flowing. [1]: https://en.wikipedia.org/wiki/Demand_characteristics https://en.wikipedia.org/wiki/Demand_characteristics
mik1998 3y ago
It looks like regression to the mean.
mmmpetrichor 3y ago
Based on what I can tell, their data is solely based on whether they "reported living in a home". This sounds like extremely bad science. I can easily see someone reporting whatever they think you want to hear if you're giving them money. That would definitely explain why the lowest group had such a huge increase. I'm hugely skeptical.
notahacker 3y ago
Yeah, this. It looks like any interpretation of the data is complicated slightly by the three groups all having slightly different initial situations, but the approx 34% of people finding stable housing within six months after receiving a monthly stipend that looks sufficient to cover budget room rent vs the 31% who found housing after receiving virtually nothing doesn't look like a promising result. There are always going to be secular trends of surveyed people regardless of what they're given. First of all a lot of homelessness isn't permanent, so if you monitor a group of homeless people some of them will cease to be homeless over the course of time regardless of whether they're getting anything from you or even aware they're being monitored. And secondly people are aware they're being studied, and often react. That may be as simple and positive as doing more to help themselves because the nice person showed concern about where they'd be in six months time, but it may also be people acting or claiming to act differently because they believe the payments are linked to what they report (no matter how much surveyors insist it's unconditional and there's no judgement and no possibility of the experiment being stopped or continued...)

The massive increase seen in the control group seems to indicate that either ~5% of the money delivers >50% of the benefit, or there were some other factors that were more significant than the money.

munificent 3y ago

It seems to me like demand effects[1] could explain a large fraction of the observed behavior. People like being given money. If you tell them you're conducting a study to determine if giving them leads to them doing XYZ, it should come as no surprise when they do XYZ, even if the money didn't literally enable it: they are incentivized to XYZ all they can in order to keep the money pump flowing. [1]: https://en.wikipedia.org/wiki/Demand_characteristics https://en.wikipedia.org/wiki/Demand_characteristics

mik1998 3y ago

It looks like regression to the mean.

mmmpetrichor 3y ago

Based on what I can tell, their data is solely based on whether they "reported living in a home". This sounds like extremely bad science. I can easily see someone reporting whatever they think you want to hear if you're giving them money. That would definitely explain why the lowest group had such a huge increase. I'm hugely skeptical.

notahacker 3y ago

Yeah, this. It looks like any interpretation of the data is complicated slightly by the three groups all having slightly different initial situations, but the approx 34% of people finding stable housing within six months after receiving a monthly stipend that looks sufficient to cover budget room rent vs the 31% who found housing after receiving virtually nothing doesn't look like a promising result. There are always going to be secular trends of surveyed people regardless of what they're given. First of all a lot of homelessness isn't permanent, so if you monitor a group of homeless people some of them will cease to be homeless over the course of time regardless of whether they're getting anything from you or even aware they're being monitored. And secondly people are aware they're being studied, and often react. That may be as simple and positive as doing more to help themselves because the nice person showed concern about where they'd be in six months time, but it may also be people acting or claiming to act differently because they believe the payments are linked to what they report (no matter how much surveyors insist it's unconditional and there's no judgement and no possibility of the experiment being stopped or continued...)