4 ms·

If I'm reading this correctly, this is based off a Facebook survey in which you self select your education level. The survey responses had enough blatantly tran

If I'm reading this correctly, this is based off a Facebook survey in which you self select your education level. The survey responses had enough blatantly transphobic entries to warrant mentioning in the limitations, and the % of responders with PhD's is a fair amount higher than the % of Americans with PhD's. I'd bet many people were dishonest about their education.

forgotpwd16 5y ago
Seems you're right. From page 4 of the paper: 56 This analysis used the COVID Trends and Impact Survey (CTIS), created by the Delphi Group at 57 Carnegie Mellon University (CMU) and conducted in collaboration with Facebook Data for 58 Good. Survey sampling is described in the eMethods. Survey weights accounting for the 59 sampling design and non-response are post-stratified to match the US general population by 60 age, gender, and state. The study design ensures CMU researchers do not see usernames or 61 profile information and Facebook does not see survey microdata. A not sane approach giving off potentially erroneous data.
d_tr 5y ago
Another question I have is whether varying Facebook usage patterns and tendency to actually take the survey, among different groups and people with varying opinions, is accounted for and whether it can be accounted for at all.
rsynnott 5y ago
A _Facebook survey_? Oh, come on. That's a ridiculous way to do pretty much anything.
bobbobson 5y ago
The survey was online, yes, which is what allowed over 5 million people taking part in it. This is not uncommon in scientific articles, and I don't see why mentioning Facebook in your comment is relevant. Also 2.1% of the responders had a PhD and 2.0% of the US population has a PhD, that does not seem like "a fair amount higher". They also mention that "our sample is more educated with higher vaccine uptake than general population, indicating that vaccine hesitancy is underestimated in this sample." and that "Survey weights adjust for non response and coverage bias". With regards to the "small percentage" of transphobic comments, they say that "A sensitivity analysis eliminating respondents with self-described gender produced very similar results". So basically everything you said to try to discredit this particular paper that goes against your preconceived beliefs is irrelevant/wrong.
- boomboomsubban 5y ago
  >The survey was online, yes, which is what allowed over 5 million people taking part in it. I didn't get that part. Five million was mentioned, but there were only five hundred thousand responses. I'd guess five million were offered the survey? >Also 2.1% of the responders had a PhD and 2.0% of the US population has a PhD, that does not seem like "a fair amount higher" The percent with a PhD I could find ranged from 1.2% to 1.6%, and you're ignoring that some amount of the 13% of "no response" would have PhD's as well. >They also mention that "our sample is more educated with higher vaccine uptake than general population, indicating that vaccine hesitancy is underestimated in this sample. Or that indicates people are lying about their education, supporting my previous point. They mention that one of the limitations is that the answers just need to be taken in good faith. So basically I will remain skeptical of Facebook surveys.
- jollybean 5y ago
  The means by which the poll was conducted is fundamental to the nature of the quality of the results.

forgotpwd16 5y ago

Seems you're right. From page 4 of the paper: 56 This analysis used the COVID Trends and Impact Survey (CTIS), created by the Delphi Group at 57 Carnegie Mellon University (CMU) and conducted in collaboration with Facebook Data for 58 Good. Survey sampling is described in the eMethods. Survey weights accounting for the 59 sampling design and non-response are post-stratified to match the US general population by 60 age, gender, and state. The study design ensures CMU researchers do not see usernames or 61 profile information and Facebook does not see survey microdata. A not sane approach giving off potentially erroneous data.

d_tr 5y ago

Another question I have is whether varying Facebook usage patterns and tendency to actually take the survey, among different groups and people with varying opinions, is accounted for and whether it can be accounted for at all.

rsynnott 5y ago

A _Facebook survey_? Oh, come on. That's a ridiculous way to do pretty much anything.

bobbobson 5y ago

The survey was online, yes, which is what allowed over 5 million people taking part in it. This is not uncommon in scientific articles, and I don't see why mentioning Facebook in your comment is relevant. Also 2.1% of the responders had a PhD and 2.0% of the US population has a PhD, that does not seem like "a fair amount higher". They also mention that "our sample is more educated with higher vaccine uptake than general population, indicating that vaccine hesitancy is underestimated in this sample." and that "Survey weights adjust for non response and coverage bias". With regards to the "small percentage" of transphobic comments, they say that "A sensitivity analysis eliminating respondents with self-described gender produced very similar results". So basically everything you said to try to discredit this particular paper that goes against your preconceived beliefs is irrelevant/wrong.

boomboomsubban 5y ago
>The survey was online, yes, which is what allowed over 5 million people taking part in it. I didn't get that part. Five million was mentioned, but there were only five hundred thousand responses. I'd guess five million were offered the survey? >Also 2.1% of the responders had a PhD and 2.0% of the US population has a PhD, that does not seem like "a fair amount higher" The percent with a PhD I could find ranged from 1.2% to 1.6%, and you're ignoring that some amount of the 13% of "no response" would have PhD's as well. >They also mention that "our sample is more educated with higher vaccine uptake than general population, indicating that vaccine hesitancy is underestimated in this sample. Or that indicates people are lying about their education, supporting my previous point. They mention that one of the limitations is that the answers just need to be taken in good faith. So basically I will remain skeptical of Facebook surveys.
jollybean 5y ago
The means by which the poll was conducted is fundamental to the nature of the quality of the results.