2 ms·
It's very simple to write a basic ActivityPub crawler (I have done this), but you'd go from outbox to outbox, not inbox to inbox. Regardless, it's all just JSON
by sfsdfsdfasfasd 4y ago
It's very simple to write a basic ActivityPub crawler (I have done this), but you'd go from outbox to outbox, not inbox to inbox. Regardless, it's all just JSON, so it is more straightforward to crawl than the web.
I tried writing a crawler to map the Fediverse (to discover homeservers) but I discovered quickly that most of the content in the Fediverse is extremist content, both left-wing and right-wing -- it's basically all tankies and Nazis, with a few techies in a bubble using mastodon.social, who think Fediblock is a solution to this problem.
Oh right, and there are a ton of pedo instances too, mostly based in Japan where "loli" is legal. Due to the way ActivityPub publishes federated content, I didn't feel comfortable running a homeserver, because a pedo instance could federate illegal content to my homeserver and get me arrested and charged for possession of content I did not request and do not want to store on my computers, and if I'm not monitoring what content is getting federated to my timeline, I might not even know it's there. Too risky.
Demoralized, I abandoned the project, and I don't think much about ActivityPub anymore.
- mariusor 4y ago> because a pedo instance could federate illegal content to my homeserver and get me arrested and charged for possession of content I did not request and do not want to store on my computers, and if I'm not monitoring what content is getting federated to my timeline, I might not even know it's there. Too risky. I also work in the activitypub space and my solution to avoiding unlawful content is to make the "federating" relationship between two servers a mutual one. One asks to follow, the other replies favourably (or not). This puts a damper on the "wild west" of everyone federating with everyone, but ensures content is curated at a basic level by a SysOP so there are no surprises.