6 ms·

How do you get people into a trial like this without the bias of either "people kind enough to spend time helping research" or "people who need money enough tha

by corin_ 6y ago

How do you get people into a trial like this without the bias of either "people kind enough to spend time helping research" or "people who need money enough that being a paid test subject is worth their time"? Might not matter for something like a vaccine trial, but for this?

rpedela 6y ago
Sometimes participation is required to pass psychology classes.
- koolba 6y ago
  Indeed though even there you’ll have significant selection bias for psych v.s. other majors.
austincheney 6y ago
Generally it’s college students who, despite having no income, are not living in poverty and have plenty of time.
- throwaway0a5e 6y ago
  ...and who on average have very limited life experience. A random subset of psych majors is going to have very different characteristics than a random subset of dining hall student employees is going to have very different characteristics than a subset of frat members.
  - mercer 6y ago
    I can't vouch for this particular study, but my experience in college has been that possible differences between 'subset of psych majors' and the rest of the population are among the earliest considerations to, well, consider.
- deleted 6y ago
  [deleted]
- deleted 6y ago
  [deleted]
tixocloud 6y ago
You've just highlighted the key issue with studies, focus groups and marketing research. The population will always be biased - even the responses from the participants are likely to be biased because they are more likely to give a biased response. The insight that you get is marginally useful but nonetheless, it's better than no insight at all.
- skruban 6y ago
  why would a biased insight be better than no insight? Depends on the amount of biasness, doesn't it?
  - diegoperini 6y ago
    Because having multiple researches about the same subject but with different biases is the way to eliminate bias with meta analysis studies.
playpause 6y ago
Selection bias (and mitigation of it) is a big topic, and worth googling, if you're interested. I recommend reading 'Bad Science' by Ben Goldacre, which is a fun read and goes into fascinating detail about various forms of bias in scientific studies and their reporting in the media, and how such problems can be managed.
bonoboTP 6y ago
My harsh and honest opinion is that this kind of research is useless and is mostly a waste of resources and most likely serves other motives than we may think (narratives, signaling etc). All experiments with virtual money, with artificial lab setups, with people knowing they are in an experiment are very very biased. Even if being mean would gain you some money in an artificial experiment, there are other social factors that can counteract this, including pleasing the experiment designers, thinking of yourself as a good person. People probably introspect more in these observed scenarios etc. You know all those people writing their bachelor and master theses based on questionnaires filled out by random recruits from social media for the chance of winning a gift card? I see tons of those. Are they rigorous and valuable research? Unlikely, but at least they don't get into the press. We keep falling for this stuff even after the replication crisis, because science is used by the masses as a replacement of the priesthood. People in labcoats must say to us with jargon that compassion is good. We are too afraid of and are still recovering from the shock of the early and crude interpretation of the Darwinian theory. So we crave the "people are good after all" message and when there is demand there will be supply.
- crawfordcomeaux 6y ago
  Upvoted for the skepticism and the reality of how controlled experiments can produce desirable results that don't replicate. I still believe people are good after all because I've been conducting a real-world study for nearly 3 years, off-and-on. I started out with a car, no money, and the assumption that I'd gotten good at identifying my needs, asking for them to be meet, and being seen/heard. I now suspect people who are actively working to unlearn their social conditioning are more willing to help others because they're willing to see some aspect of their own humanity in me. That seems to be the trend, at least. When people first see me, they often respond either by showing fear (referring to me as Charles Manson was something a stranger did a couple weeks ago), intrigue/curiosity (asking why I'm dressed the way I am, possibly asking to take a photo), smile and walk away, candidly catch me on film, compliment me for what I'm wearing, or thank me for what I'm wearing. It could also be that people who are unlearning toxic/limiting social norms value seeing others doing the same, which I definitely do with my fashion. My experiments have led me through many adventures and now I'm writing this while next to my sleeping partner & 19-month old, so I'm not actively testing in the field as much. We're planning on expanding our community during the pandemic to find land to live on and people to live with. That's the next big experiment. We're hoping someone will simply give us land to conduct experiments in community building on. My experiences could also have a bit to do with the fact that I'm white and perceived as a man, since I have a big beard. Ultimately, I'd need way more people doing what I'm doing to see what really happens outside the cloak of white privilege.

by corin_ 6y ago

rpedela 6y ago

Sometimes participation is required to pass psychology classes.

koolba 6y ago
Indeed though even there you’ll have significant selection bias for psych v.s. other majors.

austincheney 6y ago

Generally it’s college students who, despite having no income, are not living in poverty and have plenty of time.

throwaway0a5e 6y ago
...and who on average have very limited life experience. A random subset of psych majors is going to have very different characteristics than a random subset of dining hall student employees is going to have very different characteristics than a subset of frat members.
- mercer 6y ago
  I can't vouch for this particular study, but my experience in college has been that possible differences between 'subset of psych majors' and the rest of the population are among the earliest considerations to, well, consider.
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]

tixocloud 6y ago

You've just highlighted the key issue with studies, focus groups and marketing research. The population will always be biased - even the responses from the participants are likely to be biased because they are more likely to give a biased response. The insight that you get is marginally useful but nonetheless, it's better than no insight at all.

skruban 6y ago
why would a biased insight be better than no insight? Depends on the amount of biasness, doesn't it?
- diegoperini 6y ago
  Because having multiple researches about the same subject but with different biases is the way to eliminate bias with meta analysis studies.

playpause 6y ago

Selection bias (and mitigation of it) is a big topic, and worth googling, if you're interested. I recommend reading 'Bad Science' by Ben Goldacre, which is a fun read and goes into fascinating detail about various forms of bias in scientific studies and their reporting in the media, and how such problems can be managed.

bonoboTP 6y ago

My harsh and honest opinion is that this kind of research is useless and is mostly a waste of resources and most likely serves other motives than we may think (narratives, signaling etc). All experiments with virtual money, with artificial lab setups, with people knowing they are in an experiment are very very biased. Even if being mean would gain you some money in an artificial experiment, there are other social factors that can counteract this, including pleasing the experiment designers, thinking of yourself as a good person. People probably introspect more in these observed scenarios etc. You know all those people writing their bachelor and master theses based on questionnaires filled out by random recruits from social media for the chance of winning a gift card? I see tons of those. Are they rigorous and valuable research? Unlikely, but at least they don't get into the press. We keep falling for this stuff even after the replication crisis, because science is used by the masses as a replacement of the priesthood. People in labcoats must say to us with jargon that compassion is good. We are too afraid of and are still recovering from the shock of the early and crude interpretation of the Darwinian theory. So we crave the "people are good after all" message and when there is demand there will be supply.

crawfordcomeaux 6y ago
Upvoted for the skepticism and the reality of how controlled experiments can produce desirable results that don't replicate. I still believe people are good after all because I've been conducting a real-world study for nearly 3 years, off-and-on. I started out with a car, no money, and the assumption that I'd gotten good at identifying my needs, asking for them to be meet, and being seen/heard. I now suspect people who are actively working to unlearn their social conditioning are more willing to help others because they're willing to see some aspect of their own humanity in me. That seems to be the trend, at least. When people first see me, they often respond either by showing fear (referring to me as Charles Manson was something a stranger did a couple weeks ago), intrigue/curiosity (asking why I'm dressed the way I am, possibly asking to take a photo), smile and walk away, candidly catch me on film, compliment me for what I'm wearing, or thank me for what I'm wearing. It could also be that people who are unlearning toxic/limiting social norms value seeing others doing the same, which I definitely do with my fashion. My experiments have led me through many adventures and now I'm writing this while next to my sleeping partner & 19-month old, so I'm not actively testing in the field as much. We're planning on expanding our community during the pandemic to find land to live on and people to live with. That's the next big experiment. We're hoping someone will simply give us land to conduct experiments in community building on. My experiences could also have a bit to do with the fact that I'm white and perceived as a man, since I have a big beard. Ultimately, I'd need way more people doing what I'm doing to see what really happens outside the cloak of white privilege.