4 ms·

I wonder if selecting your population via people who respond to a Facebook ad about studying quitting social media introduces any bias. (I can't see the article

by ja3k 7y ago

I wonder if selecting your population via people who respond to a Facebook ad about studying quitting social media introduces any bias. (I can't see the article so I don't know what their ad actually was) I worry that the correct takeaway is people who are interested in the idea of taking a break from social media probably should. Maybe the study was sufficiently well compensated that people who feel positively about social media still felt participation was worthwhile?

hogFeast 7y ago
It is amazing to me that yours appears to be the only question that asks this. But yes, there are problems with this study imo...none of which are really addressed in the paper. Giving someone $100 to quit is kind of an "economist" thing to do but is that really enough of an incentive to do something you wouldn't otherwise do? No. They do say that their ad didn't mention quitting Facebook. And they did some kind of survey first (which is how they got the $100 figure if I understood that correctly) but it still doesn't look great to me. In particular, they claim that they did an RCT because they randomly assigned the people who clicked on their ad...that isn't random. I also think compliance is far harder to observe than the author's believe. The subjects deactivate, and if they log back in it reactivates but is that foolproof? Not really. Tbf though, the number of subjects who fail to complete is low given upside of $100...so maybe. The key parts of the survey really rely on self-reporting (will people have a bias to report improved wellbeing given all the bad publicity about FB? probably). But yeah, this isn't really an RCT. It would be nice to see one but I think the issue is that most people who use Facebook a lot probably aren't interested in stopping.

4 ms·

I wonder if selecting your population via people who respond to a Facebook ad about studying quitting social media introduces any bias. (I can't see the article

by ja3k 7y ago

hogFeast 7y ago
It is amazing to me that yours appears to be the only question that asks this. But yes, there are problems with this study imo...none of which are really addressed in the paper. Giving someone $100 to quit is kind of an "economist" thing to do but is that really enough of an incentive to do something you wouldn't otherwise do? No. They do say that their ad didn't mention quitting Facebook. And they did some kind of survey first (which is how they got the $100 figure if I understood that correctly) but it still doesn't look great to me. In particular, they claim that they did an RCT because they randomly assigned the people who clicked on their ad...that isn't random. I also think compliance is far harder to observe than the author's believe. The subjects deactivate, and if they log back in it reactivates but is that foolproof? Not really. Tbf though, the number of subjects who fail to complete is low given upside of $100...so maybe. The key parts of the survey really rely on self-reporting (will people have a bias to report improved wellbeing given all the bad publicity about FB? probably). But yeah, this isn't really an RCT. It would be nice to see one but I think the issue is that most people who use Facebook a lot probably aren't interested in stopping.