3 ms·

When your survey on kinks is primarily shared by kinky people and on places like fetlife (the kinky social media), I would assume that the sample is somewhat bi

When your survey on kinks is primarily shared by kinky people and on places like fetlife (the kinky social media), I would assume that the sample is somewhat biased.

d0mine 3y ago
The article addresses your point in some detail.
- derbOac 3y ago
  I do survey research, and there's ways of addressing the concerns the author raises with survey techniques. It's still interesting and they're right that these types of things (rare, taboo) are sometimes actually best studied with biased samples sometimes in some ways, with lots of anonymity. However, there are ways of addressing these concerns in "normal" samples (list randomization is an example; https://www.svri.org/blog/measuring-taboo-topics-list-randomization-gender-based-violence https://www.svri.org/blog/measuring-taboo-topics-list-random...), especially if they're large enough, and with trying to weight things. There are anonymous surveys that are carefully sampled; I'm not sure people would trust those less than something like the linked survey. If you used the right questions, you could also probably try to reweight the sample for other behaviors. This comes across as a criticism and maybe it is but I tend to think the more information like this the better, so I still think of it as a net good thing. It's just difficult to make much of things like percentiles in the absence of a very strong rationale for what you're comparing against, either in terms of a comparison population or in terms of what is being compared. Is an overall kink score meaningful? What about an overall sadomasochism score? Relative to who? You never really know your population, but sometimes you can get enough information to get a good enough sense of where your sample is in context.
  - d0mine 3y ago
    Is list randomization applicable to a survey with ~100 sensitive questions? From your link "In basic terms, list randomization aggregates a response to a sensitive question alongside responses to non-sensitive questions, thereby masking the respondent’s specific answer to the sensitive question. By randomizing lists with and without the sensitive question, researchers can identify prevalence or incidence of the sensitive item within the population or differences between groups (for example treatment and control), but not attribute the sensitive response on an individual basis. If respondents believe that their sensitive answer is not disclosed to the interviewer, they may be more likely to report private behaviors, such as experience of violence."
    - derbOac 3y ago
      100 questions would be a lot, yes, unless you had very large samples, which isn't unheard of in today's age. I suspect it would work better to combine the items somehow. Alternatively, you might be able to somehow use a subset of the 100 as a kind of anchor, to see how distorted a differently designed sample was, and possibly reweight things. For example, if you were really serious, you might be able to identify, say, 5-10 items with a lot of variance in the population, and ask about them using list randomization, and also maybe directly to determine how people endorse the items differently if asked indirectly versus directly. Depending on what you did with that, you could then use those items as weighting variables for the other 90-95 items in another sample. In the very least you'd be able to compare the samples to see how different the distributions are on the anchor items.
Aella 3y ago
I think it's prob slightly biased towards kinky people, but it wasn't primarily shared across fetlife. Most of the responses came from viral videos of people sharing their results on Tiktok.
- gremlinsinc 3y ago
  have you tried contacting onlyfans and other sites to integrate a pop-up ad that just randomly loads asking people to just answer one question and then you could maybe gather information that way? similar to those quick YouTube surveys that pop up instead of ads sometimes? You could maybe work out a discount deal or even see if maybe your data could be useful for their business and you could reach out to other sites like cam or video sites. Maybe there could be a sexual census where all porn sites for one week a year do a major push to get people to take the full survey, maybe giving people free stuff for the effort.
- pclmulqdq 3y ago
  Very interesting, thanks! I had heard that “kinktok” was pretty big, but I assume the streams cross on TikTok a lot more than in dedicated kink spaces. There’s still a bias towards people who are open about sex, but I am not sure that group is actually any less vanilla than people who keep it private. I saw a bunch of your older surveys and was impressed at the quality of the questions, but the places I saw them were always kink-adjacent, and I am glad you are getting a lot more reach.
- raffraffraff 3y ago
  ...Which makes me doubt the honesty of their answers. TikTok is a platform designed to encourage show-offs and attention seekers. "Oh me? I'd fuck anything! I've done the Karma Sutra with my house, but it's, like, sooo boring and vanilla"

When your survey on kinks is primarily shared by kinky people and on places like fetlife (the kinky social media), I would assume that the sample is somewhat biased.

d0mine 3y ago

The article addresses your point in some detail.

derbOac 3y ago
I do survey research, and there's ways of addressing the concerns the author raises with survey techniques. It's still interesting and they're right that these types of things (rare, taboo) are sometimes actually best studied with biased samples sometimes in some ways, with lots of anonymity. However, there are ways of addressing these concerns in "normal" samples (list randomization is an example; https://www.svri.org/blog/measuring-taboo-topics-list-randomization-gender-based-violence https://www.svri.org/blog/measuring-taboo-topics-list-random...), especially if they're large enough, and with trying to weight things. There are anonymous surveys that are carefully sampled; I'm not sure people would trust those less than something like the linked survey. If you used the right questions, you could also probably try to reweight the sample for other behaviors. This comes across as a criticism and maybe it is but I tend to think the more information like this the better, so I still think of it as a net good thing. It's just difficult to make much of things like percentiles in the absence of a very strong rationale for what you're comparing against, either in terms of a comparison population or in terms of what is being compared. Is an overall kink score meaningful? What about an overall sadomasochism score? Relative to who? You never really know your population, but sometimes you can get enough information to get a good enough sense of where your sample is in context.
- d0mine 3y ago
  Is list randomization applicable to a survey with ~100 sensitive questions? From your link "In basic terms, list randomization aggregates a response to a sensitive question alongside responses to non-sensitive questions, thereby masking the respondent’s specific answer to the sensitive question. By randomizing lists with and without the sensitive question, researchers can identify prevalence or incidence of the sensitive item within the population or differences between groups (for example treatment and control), but not attribute the sensitive response on an individual basis. If respondents believe that their sensitive answer is not disclosed to the interviewer, they may be more likely to report private behaviors, such as experience of violence."
  - derbOac 3y ago
    100 questions would be a lot, yes, unless you had very large samples, which isn't unheard of in today's age. I suspect it would work better to combine the items somehow. Alternatively, you might be able to somehow use a subset of the 100 as a kind of anchor, to see how distorted a differently designed sample was, and possibly reweight things. For example, if you were really serious, you might be able to identify, say, 5-10 items with a lot of variance in the population, and ask about them using list randomization, and also maybe directly to determine how people endorse the items differently if asked indirectly versus directly. Depending on what you did with that, you could then use those items as weighting variables for the other 90-95 items in another sample. In the very least you'd be able to compare the samples to see how different the distributions are on the anchor items.

Aella 3y ago

I think it's prob slightly biased towards kinky people, but it wasn't primarily shared across fetlife. Most of the responses came from viral videos of people sharing their results on Tiktok.

gremlinsinc 3y ago
have you tried contacting onlyfans and other sites to integrate a pop-up ad that just randomly loads asking people to just answer one question and then you could maybe gather information that way? similar to those quick YouTube surveys that pop up instead of ads sometimes? You could maybe work out a discount deal or even see if maybe your data could be useful for their business and you could reach out to other sites like cam or video sites. Maybe there could be a sexual census where all porn sites for one week a year do a major push to get people to take the full survey, maybe giving people free stuff for the effort.
pclmulqdq 3y ago
Very interesting, thanks! I had heard that “kinktok” was pretty big, but I assume the streams cross on TikTok a lot more than in dedicated kink spaces. There’s still a bias towards people who are open about sex, but I am not sure that group is actually any less vanilla than people who keep it private. I saw a bunch of your older surveys and was impressed at the quality of the questions, but the places I saw them were always kink-adjacent, and I am glad you are getting a lot more reach.
raffraffraff 3y ago
...Which makes me doubt the honesty of their answers. TikTok is a platform designed to encourage show-offs and attention seekers. "Oh me? I'd fuck anything! I've done the Karma Sutra with my house, but it's, like, sooo boring and vanilla"