5 ms·

Show HN: We built an open-source social-media simulator for research

49 points by sothatsit 4y ago

Psychology research is currently facing a crisis: A significant amount of published studies cannot be reliably reproduced. Yet these studies will still be used in news headlines, and treated as fact. Research into social media has not been immune to this either, with the problem exacerbated by study designs that don’t match the real world. We built the Misinformation Game to aid in this problem. Our tool allows researchers to design social-media studies that look and feel like social media, while also making sure that each participant has a similar experience. This is really important for the ecological validity of studies (i.e., is it similar to reality), and the reproducibility of studies (chance plays less of a role). The tool’s reproducibility is further helped because the configuration of studies can be downloaded and shared alongside published research, minimising the work required for replication. I have really enjoyed working on this project, and I hope you also find it interesting! I think there are huge opportunities to aid in research in non-tech fields using software, and I hope this project is a step in that direction. There are example studies for you to play through that are available through the website. I’m excited to hear your thoughts on it! All the best, ~ Paddy

ChicagoBoy11 4y ago
A non trivial part of social media is the social component, where folks build their own networks according to all sorts of graphs and in many instances using real world identities to do that. Do you have a sense as to the impact that the absence of that has on using something like this? Sure, we can fairly accurately replicate the mechanics of a social network, but the decision making in it surely is influenced by the difference in the social graph/tie to real identity/etc.
- sothatsit 4y ago
  Thanks for the comment! You are right that social graphs are important. Our tool does not support them explicitly, as it is designed for studies that can take place over one session (e.g., minutes to hours), not studies over longer periods. However, we hypothesise that participants will still form associations with the identities that are included within studies. These identities can all be selected by the researchers, and they could choose to use real identities. Our tool is better for research such as: * Do followers influence perceived credibility? * Does the number of likes influence whether someone will like a post? * Does the emotion in posts influence participant’s belief in misinformation? (This is a bit more complicated to test, but can be done) More complex research regarding social networks would likely require a specialised system. This is aimed for a more general audience, who would otherwise use questionnaires for many of these questions. I hope that helps :)
  - ChicagoBoy11 4y ago
    Thank you for the reply, it does! I do quite a bit of work with young people around social media and behavior, so I'm just noodling ways where I think this may be useful in setting up scenarios for them! Thanks so much for all the work!!!
    - sothatsit 4y ago
      I didn’t even think about using it for education! If you end up making use of it, I’d love to hear what you did and how it went.
anigbrowl 4y ago
A significant amount of published studies cannot be reliably reproduced. A problem likely to accelerate with the supposedly imminent (Thursday) unavailability of the free Twitter API.
pphysch 4y ago
How does the ad industry do this sort of testing? Do they just get backdoor access to real social networks?
- sothatsit 4y ago
  I’m not sure, but I’d assume they’d have a lot of data from everywhere their ads are placed, and purchasing the data would be much more viable for them.
philipov 4y ago
Please tell me you called your simulator the Terror Nexus.
- pfraze 4y ago
  Youre thinking of the Torment Nexus. The Terror Nexus was a sequel by the author’s son that most people don’t consider canon.
- sothatsit 4y ago
  I’m sorry to say that I have no idea what you’re referring to… Are there some books I should read?
genpfault 4y ago
> social-media simulator Google People[1]? [1]: https://qntm.org/person https://qntm.org/person
- deleted 4y ago
  [deleted]
- sothatsit 4y ago
  I can’t believe that something like this would exist…
disadvantage 4y ago
What's your thoughts on things like Kinzen[0] who fight misinfo? They recently got acquired by Spotify[1] [0] https://www.kinzen.com/ https://www.kinzen.com/ [1] https://techcrunch.com/2022/10/05/spotify-acquires-content-moderation-tech-company-kinzen-to-address-platform-safety-issues/ https://techcrunch.com/2022/10/05/spotify-acquires-content-m...
- sothatsit 4y ago
  Fighting misinformation can be really difficult, so it’s always good to see people working on the problem! However, while a moderation tool is great, I’m not sure if it would help with more mundane misinformation (e.g., controversial figures saying outlandish statements). Personally, it seems like misinformation education would be a longer-term solution. However, I’m not an expert on this, I am just hopeful that with a better understanding of the Psychology behind misinformation, it can be fought more effectively. If you’re interested, I work with people from the Ecker Memory & Cognition Lab, which has released handbooks to tackling misinformation: https://www.emc-lab.org/ https://www.emc-lab.org/.
luxuryballs 4y ago
I found a post that said “5G is spreading COVID” and, believe it or not, I have a source for that topic https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8580522/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8580522/
- sothatsit 4y ago
  Wow, that is startling… I tried to write example posts that were too ridiculous to be real, but here we are.

49 points by sothatsit 4y ago

ChicagoBoy11 4y ago

A non trivial part of social media is the social component, where folks build their own networks according to all sorts of graphs and in many instances using real world identities to do that. Do you have a sense as to the impact that the absence of that has on using something like this? Sure, we can fairly accurately replicate the mechanics of a social network, but the decision making in it surely is influenced by the difference in the social graph/tie to real identity/etc.

sothatsit 4y ago
Thanks for the comment! You are right that social graphs are important. Our tool does not support them explicitly, as it is designed for studies that can take place over one session (e.g., minutes to hours), not studies over longer periods. However, we hypothesise that participants will still form associations with the identities that are included within studies. These identities can all be selected by the researchers, and they could choose to use real identities. Our tool is better for research such as: * Do followers influence perceived credibility? * Does the number of likes influence whether someone will like a post? * Does the emotion in posts influence participant’s belief in misinformation? (This is a bit more complicated to test, but can be done) More complex research regarding social networks would likely require a specialised system. This is aimed for a more general audience, who would otherwise use questionnaires for many of these questions. I hope that helps :)
- ChicagoBoy11 4y ago
  Thank you for the reply, it does! I do quite a bit of work with young people around social media and behavior, so I'm just noodling ways where I think this may be useful in setting up scenarios for them! Thanks so much for all the work!!!
  - sothatsit 4y ago
    I didn’t even think about using it for education! If you end up making use of it, I’d love to hear what you did and how it went.

anigbrowl 4y ago

A significant amount of published studies cannot be reliably reproduced. A problem likely to accelerate with the supposedly imminent (Thursday) unavailability of the free Twitter API.

pphysch 4y ago

How does the ad industry do this sort of testing? Do they just get backdoor access to real social networks?

sothatsit 4y ago
I’m not sure, but I’d assume they’d have a lot of data from everywhere their ads are placed, and purchasing the data would be much more viable for them.

philipov 4y ago

Please tell me you called your simulator the Terror Nexus.

pfraze 4y ago
Youre thinking of the Torment Nexus. The Terror Nexus was a sequel by the author’s son that most people don’t consider canon.
sothatsit 4y ago
I’m sorry to say that I have no idea what you’re referring to… Are there some books I should read?

genpfault 4y ago

> social-media simulator Google People[1]? [1]: https://qntm.org/person https://qntm.org/person

deleted 4y ago
[deleted]
sothatsit 4y ago
I can’t believe that something like this would exist…

disadvantage 4y ago

What's your thoughts on things like Kinzen[0] who fight misinfo? They recently got acquired by Spotify[1] [0] https://www.kinzen.com/ https://www.kinzen.com/ [1] https://techcrunch.com/2022/10/05/spotify-acquires-content-moderation-tech-company-kinzen-to-address-platform-safety-issues/ https://techcrunch.com/2022/10/05/spotify-acquires-content-m...

sothatsit 4y ago
Fighting misinformation can be really difficult, so it’s always good to see people working on the problem! However, while a moderation tool is great, I’m not sure if it would help with more mundane misinformation (e.g., controversial figures saying outlandish statements). Personally, it seems like misinformation education would be a longer-term solution. However, I’m not an expert on this, I am just hopeful that with a better understanding of the Psychology behind misinformation, it can be fought more effectively. If you’re interested, I work with people from the Ecker Memory & Cognition Lab, which has released handbooks to tackling misinformation: https://www.emc-lab.org/ https://www.emc-lab.org/.

luxuryballs 4y ago

I found a post that said “5G is spreading COVID” and, believe it or not, I have a source for that topic https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8580522/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8580522/

sothatsit 4y ago
Wow, that is startling… I tried to write example posts that were too ridiculous to be real, but here we are.