4 ms·

It's difficult to impossible to implement bulk level social media algorithms on decentralized networks because no single node has all the data and views/interac

by api 3y ago

It's difficult to impossible to implement bulk level social media algorithms on decentralized networks because no single node has all the data and views/interactions are largely private. This is a feature and is perhaps an even more compelling reason to go federated or decentralized than the other autonomy and privacy related reasons. Social media algorithms are cultural lobotomy machines. A great example of technological limitations improving content is podcasts. Podcasts are one-way, still fairly simple in their distribution methods (RSS and a few major apps but no clear monopoly), and mostly non-interactive. This limits the ability of platforms and advertisers to ruin them, which is why podcasting is still a bastion of quality media online.

lorean_victor 3y ago
it was quite easy for a small team to crawl and index a good portion of the internet, enough to become the de facto gateway (talking about Google). it was similarly possible for a relatively small team to crawl a good chunk of the available internet and train some of the most sophisticated "algorithms" we've seen on them (talking about Open AI). if there is an incentive, this problem can be solved. if this was actually a hard problem, most current social media companies wouldn't put so much effort in restricting crawling to force everyone through restricted API access (look at Twitter, Reddit, Instagram or Facebook, as examples).
PaulHoule 3y ago
I make algorithms that filter RSS feeds and other social media content. You don't need a global view of the system at all to do this, you just need a point of view. That is, if you have a few thousand posts and a thumbs up/thumbs down judgement you can train an ML model that will predict those judgements. With about two days looking at toots I could make a model that shows you nothing but angry toots about politics or one that removes angry toots (could take down that keyword filter that means I never hear if somebody is having trouble with the transmission in their truck or that Transnistria got invaded.) The main reason I haven't developed a social media (as opposed to product/service) sentiment model like this is that it would involve looking at a few thousand angry toots. (1) The reason I want it is that I don't want to read those toots, (2) it would cause me great suffering to look at those toots. Social media moderators at companies like Facebook have been traumatized, it's no joke. If it was my social network, I'd use that filter to put a brake on selfish angry memes spreading so that the pain of one person reading angry toots gives relief to so many more. I have a model that predicts the probability of a headline getting a lot of comments relative to votes on hacker news: some high scoring headlines in my RSS reader right now are: Why do women commit far less crime than men? Study suggests anti-Black racism may account for conservatives' negative reactions to jobs requiring DEI statements Checking a bag will cost you more on United Airlines, which is copying a similar move by American Everyone seems to forget why GNOME and GNOME 3 and Unity happened (2022) Three of those are clickbait, the last one is a good HN submission. A social media sentiment model can give a larger algorithm a "superego". There are other ways to pursue engagement other than selfish angry memes.

by api 3y ago

lorean_victor 3y ago

it was quite easy for a small team to crawl and index a good portion of the internet, enough to become the de facto gateway (talking about Google). it was similarly possible for a relatively small team to crawl a good chunk of the available internet and train some of the most sophisticated "algorithms" we've seen on them (talking about Open AI). if there is an incentive, this problem can be solved. if this was actually a hard problem, most current social media companies wouldn't put so much effort in restricting crawling to force everyone through restricted API access (look at Twitter, Reddit, Instagram or Facebook, as examples).

PaulHoule 3y ago

I make algorithms that filter RSS feeds and other social media content. You don't need a global view of the system at all to do this, you just need a point of view. That is, if you have a few thousand posts and a thumbs up/thumbs down judgement you can train an ML model that will predict those judgements. With about two days looking at toots I could make a model that shows you nothing but angry toots about politics or one that removes angry toots (could take down that keyword filter that means I never hear if somebody is having trouble with the transmission in their truck or that Transnistria got invaded.) The main reason I haven't developed a social media (as opposed to product/service) sentiment model like this is that it would involve looking at a few thousand angry toots. (1) The reason I want it is that I don't want to read those toots, (2) it would cause me great suffering to look at those toots. Social media moderators at companies like Facebook have been traumatized, it's no joke. If it was my social network, I'd use that filter to put a brake on selfish angry memes spreading so that the pain of one person reading angry toots gives relief to so many more. I have a model that predicts the probability of a headline getting a lot of comments relative to votes on hacker news: some high scoring headlines in my RSS reader right now are: Why do women commit far less crime than men? Study suggests anti-Black racism may account for conservatives' negative reactions to jobs requiring DEI statements Checking a bag will cost you more on United Airlines, which is copying a similar move by American Everyone seems to forget why GNOME and GNOME 3 and Unity happened (2022) Three of those are clickbait, the last one is a good HN submission. A social media sentiment model can give a larger algorithm a "superego". There are other ways to pursue engagement other than selfish angry memes.