4 ms·

The response rate is so low as to be entirely meaningless. If you're content with sampling only 0.000001% of the population, you can get any results you want by

The response rate is so low as to be entirely meaningless. If you're content with sampling only 0.000001% of the population, you can get any results you want by simply selecting for the people who care the most by taking a bunch of time to lecture the individual if they seem like they'd vote the other way (oh no, you have to leave? but we didn't even have time to submit your vote yet... :( ), and wording the question in such a way to make the answer you want be the only reasonable response.

orhmeh09 2y ago
What do you think is a good sample size? How would you arrive at one?
- aunty_helen 2y ago
  There isn’t one. Some things are not measurable by this standard and the stats relatively meaningless. 3000 people from a whole continent? Cmon
  - kragen 2y ago
    please see the introduction to statistical methods linked above at https://news.ycombinator.com/item?id=40590633 https://news.ycombinator.com/item?id=40590633
- dumbo-octopus 2y ago
  If your question is asked entirely free of bias from a truly randomly selected population, you can use standard statistical methods to come up with a number based on your desired confidence interval. On the other hand, if you're an organization with your funding tied to the perception that people support your mission, and you're surveying whether people support your mission, I'm going to need damn near everyone in the population to respond to your survey before I'll buy it.
  - deleted 2y ago
    [deleted]
pembrook 2y ago
Yep, asking people vague questions about the boogieman du jour is a questionable excercise at best. If a competing research team had gone out and done a survey with the question rephrased as, "Do you want a global ban on medical syringes and bottled water?" You might find out, amazingly, 92% of people strongly support single-use plastics!
- lolinder 2y ago
  The actual question was [0]: > The United Nations agreed in 2022 to develop a global treaty to end plastic pollution. How important or unimportant do you believe it is to have global rules to… > Ban unnecessary single-use plastic products, e.g., shopping bags, cutlery, cups & plates? As survey questions go, this is actually a pretty precise question. [0] https://www.ipsos.com/sites/default/files/ct/news/documents/2024-05/Ipsos%20report%20-%20Single-use%20plastics.pdf https://www.ipsos.com/sites/default/files/ct/news/documents/...
  - dumbo-octopus 2y ago
    Providing the context about the UN treaty makes it unnecessarily loaded. People are biased towards being agreeable, if you start out saying "everyone in the world agrees X, do you also agree X?", people are more likely to say yes than if you asked the question in isolation. The word "unnecessary" also adds bias, and skews the results: "Of course we shouldn't unnecessarily use plastic. Oh, but my personal water bottle and the bag I get my food in and the cutlery I use to eat it and the... well not those, those are very necessary: I need them to eat and drink!"
  - TylerE 2y ago
    That's a horrible question. What exactly is "unnecessary"?
    - lolinder 2y ago
      > e.g., shopping bags, cutlery, cups & plates?
GeoAtreides 2y ago
> The response rate is so low as to be entirely meaningless. This calculator says you need a 385 sample size for a billion (or infinite) population: https://www.calculator.net/sample-size-calculator.html?type=1&cl=95&ci=5&pp=50&ps=1000000000&x=Calculate https://www.calculator.net/sample-size-calculator.html?type=... Please check the theory presented below the calculator, it explains why such a low number is enough for a confidence level of 95% that the real value is within ±5% of the measured/surveyed value.
- Slyfox33 2y ago
  That's extremely dependent on who the 385 people are.
  - orhmeh09 2y ago
    TFA goes into this a little > The samples in Argentina, Australia, Belgium, Canada, France, Germany, Great Britain, Hungary, Italy, Japan, the Netherlands, Poland, the Republic of Korea, Spain, Sweden, and the United States can be taken as representative of these countries' general adult population under the age of 75. > Due to the fact that the vast majority of the data was collected via Ipsos' online panels (India being the only exception, in which 1,800 were interviewed face-to-face and 400 were interviewed online), participation tends to consist of those who have access to the necessary technology. The samples in Brazil, Chile, Colombia, Indonesia, Ireland, Malaysia, Mexico, Morocco, Nigeria, Peru, Singapore, South Africa, Thailand, Turkey and Uganda are more urban, more educated and/or more affluent than the general population. While not nationally representative, the survey results for these countries provide a useful and unique indication of the direction of public opinion. > > Weighting has been employed to balance demographics and ensure that the sample's composition reflects that of the adult population according to the most recent census data. > > The precision of Ipsos online polls are calculated using a credibility interval with a poll of 1,000 accurate to +/- 3•5 percentage points and of 500 accurate to +/- 5.0 percentage points. For more information on the Ipsos use of credibility intervals, please visit the Ipsos website. Ipsos is fairly reputable in the field, FWIW.
- dumbo-octopus 2y ago

orhmeh09 2y ago

What do you think is a good sample size? How would you arrive at one?

aunty_helen 2y ago
There isn’t one. Some things are not measurable by this standard and the stats relatively meaningless. 3000 people from a whole continent? Cmon
- kragen 2y ago
  please see the introduction to statistical methods linked above at https://news.ycombinator.com/item?id=40590633 https://news.ycombinator.com/item?id=40590633
dumbo-octopus 2y ago
If your question is asked entirely free of bias from a truly randomly selected population, you can use standard statistical methods to come up with a number based on your desired confidence interval. On the other hand, if you're an organization with your funding tied to the perception that people support your mission, and you're surveying whether people support your mission, I'm going to need damn near everyone in the population to respond to your survey before I'll buy it.
- deleted 2y ago
  [deleted]

pembrook 2y ago

Yep, asking people vague questions about the boogieman du jour is a questionable excercise at best. If a competing research team had gone out and done a survey with the question rephrased as, "Do you want a global ban on medical syringes and bottled water?" You might find out, amazingly, 92% of people strongly support single-use plastics!

lolinder 2y ago
The actual question was [0]: > The United Nations agreed in 2022 to develop a global treaty to end plastic pollution. How important or unimportant do you believe it is to have global rules to… > Ban unnecessary single-use plastic products, e.g., shopping bags, cutlery, cups & plates? As survey questions go, this is actually a pretty precise question. [0] https://www.ipsos.com/sites/default/files/ct/news/documents/2024-05/Ipsos%20report%20-%20Single-use%20plastics.pdf https://www.ipsos.com/sites/default/files/ct/news/documents/...
- dumbo-octopus 2y ago
  Providing the context about the UN treaty makes it unnecessarily loaded. People are biased towards being agreeable, if you start out saying "everyone in the world agrees X, do you also agree X?", people are more likely to say yes than if you asked the question in isolation. The word "unnecessary" also adds bias, and skews the results: "Of course we shouldn't unnecessarily use plastic. Oh, but my personal water bottle and the bag I get my food in and the cutlery I use to eat it and the... well not those, those are very necessary: I need them to eat and drink!"
- TylerE 2y ago
  That's a horrible question. What exactly is "unnecessary"?
  - lolinder 2y ago
    > e.g., shopping bags, cutlery, cups & plates?

GeoAtreides 2y ago

> The response rate is so low as to be entirely meaningless. This calculator says you need a 385 sample size for a billion (or infinite) population: https://www.calculator.net/sample-size-calculator.html?type=1&cl=95&ci=5&pp=50&ps=1000000000&x=Calculate https://www.calculator.net/sample-size-calculator.html?type=... Please check the theory presented below the calculator, it explains why such a low number is enough for a confidence level of 95% that the real value is within ±5% of the measured/surveyed value.

Slyfox33 2y ago
That's extremely dependent on who the 385 people are.
- orhmeh09 2y ago
  TFA goes into this a little > The samples in Argentina, Australia, Belgium, Canada, France, Germany, Great Britain, Hungary, Italy, Japan, the Netherlands, Poland, the Republic of Korea, Spain, Sweden, and the United States can be taken as representative of these countries' general adult population under the age of 75. > Due to the fact that the vast majority of the data was collected via Ipsos' online panels (India being the only exception, in which 1,800 were interviewed face-to-face and 400 were interviewed online), participation tends to consist of those who have access to the necessary technology. The samples in Brazil, Chile, Colombia, Indonesia, Ireland, Malaysia, Mexico, Morocco, Nigeria, Peru, Singapore, South Africa, Thailand, Turkey and Uganda are more urban, more educated and/or more affluent than the general population. While not nationally representative, the survey results for these countries provide a useful and unique indication of the direction of public opinion. > > Weighting has been employed to balance demographics and ensure that the sample's composition reflects that of the adult population according to the most recent census data. > > The precision of Ipsos online polls are calculated using a credibility interval with a poll of 1,000 accurate to +/- 3•5 percentage points and of 500 accurate to +/- 5.0 percentage points. For more information on the Ipsos use of credibility intervals, please visit the Ipsos website. Ipsos is fairly reputable in the field, FWIW.
dumbo-octopus 2y ago