4 ms·

> We carried out an online survey, administered via Survey Monkey This type of thing, where they do some statistics on survey data, seems to be fairly typical

> We carried out an online survey, administered via Survey Monkey This type of thing, where they do some statistics on survey data, seems to be fairly typical in psychology research. But I find it hard to believe that you can actually get good data from self-reporting in surveys. There must be selection effects: "The survey was advertised on social media and through the researchers’ own networks". And the questions may not be interpreted by the respondents as imagined. What does it mean if you select "None of the above" in their core question? That you think that objects have no attributes whatsoever? "Do you ever view objects as having: Gender / Human-like attributes / Feelings / Other / None of the above" See also "replication crisis". Psychology is at the center of it.

Sohcahtoa82 1y ago
> I find it hard to believe that you can get actually get good data from self-reporting in surveys. Especially when you consider the Lizardman's Constant of 4% https://slatestarcodex.com/2013/04/12/noisy-poll-results-and-reptilian-muslim-climatologists-from-mars/ https://slatestarcodex.com/2013/04/12/noisy-poll-results-and... This article was written in 2013. The Internet has become a significantly more toxic place since then, and I would imagine the constant is closer to 15-20% these days.
- Tijdreiziger 1y ago
  > […] I would imagine the constant is closer to 15-20% these days. On what basis?
  - vntok 1y ago
    An online survey administered via Survey Monkey, of course.
  - wafflemaker 1y ago
    On basis of spending a lot of my teenage time browsing 4chan, I'd say it's not 15%, but closer to 69% or even whole 420%. You just loose faith in humanity from spending too much time on the internet.
    - AlecSchueler 1y ago
      So we're writing off the methodologies of the entire field of psychology on the basis of numbers pulled out of your--on the basis of your gut feeling?
      - kbelder 1y ago
        In fairness, Wafflemaker's gut feeling probably has a better track record.
    - Tijdreiziger 1y ago
      I doubt 4chan is representative for the entire Internet.
  - AlecSchueler 1y ago
    Don't you know this is HN? A hand wave here is worth more than any methodical research.
  - Sohcahtoa82 1y ago
    Based on what I learned from this video: https://youtu.be/r7l0Rq9E8MY?t=2 https://youtu.be/r7l0Rq9E8MY?t=2
Tijdreiziger 1y ago
Forget the quality of the data, you can give the same data to different researchers and get wildly different conclusions. [1, from 2, from 3] (Disclaimer: not my area of expertise.) [1] N. Breznau et al., “Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty,” PNAS, October 2022, URL: https://www.pnas.org/doi/10.1073/pnas.2203150119 https://www.pnas.org/doi/10.1073/pnas.2203150119 [2] B. Klaas, “The Crisis of Zombie Social Science.” The Garden of Forking Paths. https://www.forkingpaths.co/p/the-crisis-of-zombie-social-science https://www.forkingpaths.co/p/the-crisis-of-zombie-social-sc... [3] The crisis of zombie social science (20 points, 2 days ago) – https://news.ycombinator.com/item?id=44272057 https://news.ycombinator.com/item?id=44272057
LorenPechtel 1y ago
Yup. There's so much pressure to publish something that we consistently see attempts to extract data from noise.
deleted 1y ago
[deleted]

Sohcahtoa82 1y ago

> I find it hard to believe that you can get actually get good data from self-reporting in surveys. Especially when you consider the Lizardman's Constant of 4% https://slatestarcodex.com/2013/04/12/noisy-poll-results-and-reptilian-muslim-climatologists-from-mars/ https://slatestarcodex.com/2013/04/12/noisy-poll-results-and... This article was written in 2013. The Internet has become a significantly more toxic place since then, and I would imagine the constant is closer to 15-20% these days.

Tijdreiziger 1y ago
> […] I would imagine the constant is closer to 15-20% these days. On what basis?
- vntok 1y ago
  An online survey administered via Survey Monkey, of course.
- wafflemaker 1y ago
  On basis of spending a lot of my teenage time browsing 4chan, I'd say it's not 15%, but closer to 69% or even whole 420%. You just loose faith in humanity from spending too much time on the internet.
  - AlecSchueler 1y ago
    So we're writing off the methodologies of the entire field of psychology on the basis of numbers pulled out of your--on the basis of your gut feeling?
    - kbelder 1y ago
      In fairness, Wafflemaker's gut feeling probably has a better track record.
  - Tijdreiziger 1y ago
    I doubt 4chan is representative for the entire Internet.
- AlecSchueler 1y ago
  Don't you know this is HN? A hand wave here is worth more than any methodical research.
- Sohcahtoa82 1y ago
  Based on what I learned from this video: https://youtu.be/r7l0Rq9E8MY?t=2 https://youtu.be/r7l0Rq9E8MY?t=2

Tijdreiziger 1y ago

Forget the quality of the data, you can give the same data to different researchers and get wildly different conclusions. [1, from 2, from 3] (Disclaimer: not my area of expertise.) [1] N. Breznau et al., “Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty,” PNAS, October 2022, URL: https://www.pnas.org/doi/10.1073/pnas.2203150119 https://www.pnas.org/doi/10.1073/pnas.2203150119 [2] B. Klaas, “The Crisis of Zombie Social Science.” The Garden of Forking Paths. https://www.forkingpaths.co/p/the-crisis-of-zombie-social-science https://www.forkingpaths.co/p/the-crisis-of-zombie-social-sc... [3] The crisis of zombie social science (20 points, 2 days ago) – https://news.ycombinator.com/item?id=44272057 https://news.ycombinator.com/item?id=44272057

LorenPechtel 1y ago

Yup. There's so much pressure to publish something that we consistently see attempts to extract data from noise.

deleted 1y ago

[deleted]