3 ms·
Pollution of online social spaces caused by rampaging d/misinformation is a growing societal concern. However, recent decisions to reduce access to social media
by 7d7n 2y ago
Pollution of online social spaces caused by rampaging d/misinformation is a growing societal concern. However, recent decisions to reduce access to social media APIs are causing a shortage of publicly available, recent, social media data, thus hindering the advancement of computational social science as a whole. To address this pressing issue, we present a large, high-coverage dataset of social interactions and user-generated content from Bluesky Social.
The dataset contains the complete post history of over 4M users (81% of all registered accounts), totaling 235M posts. We also make available social data covering follow, comment, repost, and quote interactions.
Since Bluesky allows users to create and bookmark feed generators (i.e., content recommendation algorithms), we also release the full output of several popular algorithms available on the platform, along with their timestamped “like” interactions and time of bookmarking.
This dataset allows unprecedented analysis of online behavior and human-machine engagement patterns. Notably, it provides ground-truth data for studying the effects of content exposure and self-selection, and performing content virality and diffusion analysis.
- yawnxyz 2y agoI wonder how much time it takes to run this / what the script is / how resource intensive it is? Bsky is public right, so do you get rate limited? Do you scrape or use an official API? So many questions Also, I feel like only recently there's been an influx of people who have actually interesting things to say so I'd love to see nextyear's dataset
- viccis 2y agoNot sure about bulk export but you can set up a full stream of all activity without even registering an account.
- unshavedyak 2y agoBlows my mind that they can send that much for free.
- ks2048 2y agoI was checking out the Python API today (the "firehouse" via "atproto" package) and got 5000 posts in 7.5 seconds.
- verdverm 2y agoI believe they are enabling(ed?) filters so you can control how much and what you actually get from the firehose
- srik 2y ago[deleted]
- paxys 2y ago"Personal data" that was voluntarily published on a public microblogging platform with the explicit intention to share it with the world?
- srik 2y ago[deleted]
- ronsor 2y agoDon't make it public then.
- srik 2y ago[deleted]
- ronsor 2y agoIs it more entitled to observe public data than to willingly put data in the public and then expect to control the actions of others?
- paxys 2y agoIs you reading my comment on HN also entitlement? I certainly didn't give you permission to do it. It may have some personal details that I don't want you to see. Why do you think that is okay?
- deleted 2y ago[deleted]
- loeber 2y agoThe data is public by default. You know this when you sign up and use the service. This should inform your expectations of how the data will be used.