13 ms·

What would open sourcing the Twitter algorithm actually look like?

142 points by transitivebs 4y ago

bombcar 4y ago
Even better than open-sourcing the algorithm would be giving people options on which algorithm they get. I bet you could get people to pay for Twitter blue just to get reverse chronological back or whatever it is.
- transitivebs 4y ago
  Agreed. The goal should be more transparency and optionality around twitter's algorithmic feed. Note that everyone has access to the reverse chronological feed called "Latest tweets". See here for how to switch: https://twitter.com/TwitterSupport/status/1447971179185295364 https://twitter.com/TwitterSupport/status/144797117918529536...? Also agreed that twitter pro would be a great place for twitter to add more options.
- shafyy 4y ago
  Still, defaults matter and most people will not change their algorithm settings. Ironically, those are exactly the people who are impacted most from an algorithmic timeline that tries to maximize engagement.
transitivebs 4y ago
Author here. I needed to split this up into multiple posts because it required a lot deeper dive than I had originally expected. What I'm really interested in, however, is help with answering the following questions: What would open sourcing the Twitter algorithm actually look like? Would it be possible to abstract away all of the engineering complexity that it takes to run a global, production system like Twitter and produce an OSS spec or API that is actually useful? Would it be possible to produce meaningful results without access to Twitter’s full data set? What does meaningful even mean here? How would we define success? What would need to happen in order to make this a reality? What are some practical proposals that would help improve the status quo?
- deleted 4y ago
  [deleted]
- karpierz 4y ago
  What do you mean by "open sourcing" the Twitter algorithm? What are the actual requirements that you're looking to satisfy by doing this?
  - transitivebs 4y ago
    Great questions. These are exactly the types of questions I'm trying to get more people to think about — and the reason I wrote the article above. It's not well-defined, but the goal imho should be to allow more transparency and optionality around Twitter's main feed UX. Open sourcing twitter's algorithmic feed would be a monumental task that's honestly unlikely to ever come to fruition, Elon or no Elon. BUT twitter could certainly do more to move in that direction. > What are the actual requirements that you're looking to satisfy by doing this? If it improves transparency around how twitter's algorithm works.. if it introduces more optionality into twitter's main UX (and isn't just a chrome extension, for instance), if it's coming from twitter itself and not a third-party.. if the effects of different factors on relevancy signals are made more transparent (like being able to play around with a simulated tweet under different conditions and seeing the resulting relevancy scores for a set of example users).. there's just so much that twitter could do here to improve transparency around how their feed works. And I'm generally in favor of anything what will move the status quo. In Elon's words “Civilizational risk is decreased the more we can increase the trust in Twitter as a public platform.”
    - MrMan 4y ago
      Actually the more I trust Twitter the more risk I am exposed to. Avoiding Twitter and discounting anything to do with Twitter is a better risk management policy for me.
- gmm1990 4y ago
  Some sort of explanation of why am I seeing this tweet would go a long way. like tweet was displayed because 50% interest in dogs 20% friends liked it 30% generally popular
Ourgon 4y ago
How it works is interesting in itself but it is not the reason why these 'algorithmic feeds' have gained the nefarious reputation they have. The real problem with this type of technology is the way it allows the proprietor of the service to present its users a tailored view of whatever content the service offers. The way this view is tailored depends on what the proprietors' aim is: Twitter uses algorithmic feeds to present its users with an ideologically driven view of whatever is being discussed, this can easily be seen by reading the first comments on contentious issues where they nearly always push some 'progressive' comments to the front, even when those comments are clearly less popular with users than whatever other comments follow. Comments critical of 'progressive' issues often require one or more extra clicks to be made visible. Sometimes this leads to the first page being devoid of comments, requiring a click to access whatever comments are present where those comments are nearly invariably critical of the 'progressive' issue being discussed. This does not happen when there are positive comments on those 'progressive' issues since these are presented directly on the first page. TikTok and Youtube also use algorithmic feeds to channel users but their algorithms seem to be tailored to keep users on the site/app for as long as possible.
- transitivebs 4y ago
  I agree with your sentiments. Just keep in mind that the whole point of this discussion is to move towards more transparency and optionality around how Twitter works. As you pointed out, not everything is algorithmic. There are definitely human factors at play, but without a more solid understanding of how twitter's algorithmic feed actually works, the discussion will remain hand wavy at best. Just my 2 cents :)
- cpr 4y ago
  This was abundantly clear when, e.g., every Trump tweet had the first (always highly negative) replies from a couple of crazy doctor brothers (twins?).
  - mikkergp 4y ago
    I always interpreted this as someone wrote a script and learned to game the algorithm, you think there’s a line of code in there that says trump_replies.unshift(crazy_doctor_brother_tweets) in there somewhere?
giancarlostoro 4y ago
I honestly just use Twitter’s historic feed. The day they remove this ability is the day I delete my account.
- AzzieElbab 4y ago
  I use lists which is something they seemed to have forgotten about. If Twitter removes lists I will use something like hootsuite
  - adolph 4y ago
    Lists and list sharing is reminiscent of Google Reader in some way I can't quite remember. The lack of ads is striking whenever I follow a link to a normal timeline.
- prawn 4y ago
  I use https://tweetdeck.twitter.com/ https://tweetdeck.twitter.com/ which has the ordering I expect, allows arbitrary columns and has no ads (though this might be because of ad blocking). I have five columns: personal account, replies to it, side project, replies to that, messages. Only thing I miss from the old OSX Twitter client is the ability to copy and paste in images.
twtrgrunt 4y ago
still speculation without being an employee (even if it did some research from blogs which seems to be more then most do) twitter should just write their own post and shut everyone up already. idk why the company is so secretive about everything. its clearly not doing them any favors
- transitivebs 4y ago
  I'd love for twitter to take a more active role in this, but from what I understand, twitter employees are forced to be pretty tight-lipped about these things. I tried to reference as many of the official sources as possible (in addition to drawing on my past experienced at Amazon + Facebook).
  - twtrgrunt 4y ago
    yeah these companies act like were all protecting the coca cola formula or building nukes or something. like really, its blocks of text being put in order. the hr, pr, comms, leaders or w/e seem so disconnected. im sure there is some bs reason like were protecting the company, i think its misguided being so closed off though and missing the bigger picture of building trust with users.
    - transitivebs 4y ago
      especially on a platform like twitter :)
      - twtrgrunt 4y ago
        its almost like its a place where people can interact directly. thanks for calling out scale and engineering problems btw! sometimes i think people think we're sitting around all day throwing darts at a wall of pictures of conservatives to pick who to ban next.
    - notriddle 4y ago
      > yeah these companies act like were all protecting the coca cola formula or building nukes or something. like really, its blocks of text being put in order The Coca-Cola formula seems like a pretty good analogy, honestly. Most people can't tell the difference between different cola recipes, and there probably isn't anything in the recipe itself that's responsible for the company's success. Not as much as marketing, and simply being in the right place at the right time, anyway.
Barrin92 4y ago
I always wonder what people expect when they talk about twitter's algorithm as if this some sort of Deus Ex level AI developed in the depths of Area 51. If anyone wants twitter's algorithm I guess all you need is to download some arxiv papers on state of the art recommender systems and it's probably going to look like that.
- transitivebs 4y ago
  Yes and no. Twitter's scale and real-time nature make it a difficult beast. Their network graph contains hundreds of millions of nodes and billions of edges. And it's constantly updating. So any graph ML algorithms you want to use have to deal an underlying graph that's eventually consistent at best — and oftentimes very sparse in terms of feature availability.
  - transitivebs 4y ago
    Also yeah, most people talk about "Twitter's algorithm" but have no idea what it is they're talking about — that's exactly why I wanted to write on this topic :)
  - mountainriver 4y ago
    Yup and temporal graphs are a super hard problem without a lot of great ML solutions currently
- axg11 4y ago
  I work on FAANG-scale recommender systems. Most of the time, due to the scale of the problem, the algorithms powering these systems are not anywhere near state of the art. On the other hand, they are highly optimised for the situation. You can bet that every aspect of the Twitter algorithmic feed has been thoroughly debated, thought through and most importantly A/B tested.
  - ceeplusplus 4y ago
    I worked at one of the FAANGs and the algorithms were definitely state of the art in some ways. Maybe they didn't use the latest models but the sheer size of, for example models, was huge. Easily 500+ features inputted and hundreds of millions of parameters for deep models.
icallbullshit 4y ago
Yet, any company who open sources an algorithm can simply apply any private alternative algorithms using any alternative/undisclosed data sources to any undisclosed subset of users as they wish. Hell, whose to say that some people aren't chosen for long term "experiments" with different behavioral/belief modification goals? That's what I would do.
- transitivebs 4y ago
  Good point. Even if there were some sort of open source spec like the W3C specs, for instance, Twitter would still be a black box that could choose to conform to the spec or not choose to. What we need is something more enforceable. So either you do something completely green field like https://blueskyweb.xyz/ https://blueskyweb.xyz/, or you spend $43B & roll up your sleeves to try and fix the problems with the current platform.
politician 4y ago
The easiest way to implement “open source the algorithm” is to delete the AI algorithm, revert the defaults to reverse-chronological feed, then release that.
- transitivebs 4y ago
  Sounds great. But it will never happen because of business incentives. I'm focused on trying to move the conversation towards more practical solutions.
  - politician 4y ago
    If Elon takes Twitter private, I doubt that you can argue that the quickest practical solution is other than as I’ve suggested. In the secretive context of the current non-profitable public Twitter model, yes, I agree, they cannot do anything useful. Not sure why you got a downvote though, so have my upvote.
    - transitivebs 4y ago
      Oh yeah, if Elon takes twitter private, then all bets are off :) Though I personally think throwing out any form of algorithmic feed wouldn't happen. There are just too many business and UX reasons it makes sense, which is why every major social platform uses them (https://www.socialmediatoday.com/social-networks/truth-about-algorithms-and-why-you-should-expect-see-more-them https://www.socialmediatoday.com/social-networks/truth-about...).
- prawn 4y ago
  I don't follow many people on any social network and endeavour to read everything those people post. Like you, I want a straight sort of those items. However, I assume the majority of people follow more than they can read, so a huge effort goes in to sifting through the fire hose to provide a filtered taste. Someone with a short following list can see the drama if they look at the huge list of replies that a popular account might get. So much cruft.
creakingstairs 4y ago
I've tinkered with coming up with my own algorithm for YouTube using their API (albeit very basic ones) and I quite liked the results. I _tried_to make a front end and gave up because of the scope creep. I wonder why platforms wouldn't just offer different algorithms. e.g. historical, maximise-time-on-screen or exploration. Most of the users would be nudged into what the platform wants anyway, and they will be able to claim moral superiority over other platforms that doesn't offer this. Heck, you could even charge for this feature.
- transitivebs 4y ago
  https://stratechery.com/2022/back-to-the-future-of-twitter/ https://stratechery.com/2022/back-to-the-future-of-twitter/ has a great breakdown of Twitter's history in this respect. TL;DR they used to have an open API and various third-party clients, each of which had their own flavor. BUT this made it really hard for twitter to guarantee a good UX and even harder for twitter to implement an ad-driven business model.
  - creakingstairs 4y ago
    I'm not arguing for an open API, I'm arguing for more options in content algorithms. These would be under their sole control. Aside from that, thanks for the link. That is a great post!
    - transitivebs 4y ago
      I think at a practical level, opening up their API for non-watered-down third-party clients would be the only true way to add more optionality to twitter's feed UX. Since I don't see open sourcing the actual feed algorithm to be something that would practically happen any time soon (which makes me a sad panda).
- bobthepanda 4y ago
  I would imagine Twitter spends a lot of time maintaining the one algorithm. More might involve a lot more work for diminishing upside.
tomschlick 4y ago
Other than a lot of people preferring the reverse chronological feed, I think the biggest complaint I see these days is the perception that the algorithm is being manually manipulated to promote/not promote certain topics and/or people. Given that Twitter/Facebook/Reddit/Tiktok are the behemoths in the room that all have the power to sway public opinion by either promoting or suppressing information, I think they should be regulated so that any topic that was promoted or suppressed by the algorithm or manual human intervention should show itself on the content just like an ad does. "This content has been quarantined by [Twitter Staff / Automated Review] and is only available to those with the direct url." would be a perfect example of such a message. From there, users should have control over if they want to see such quarantined content or if they want a "safe" experience that Twitter curates for them.
- mschuster91 4y ago
  > I think the biggest complaint I see these days is the perception that the algorithm is being manually manipulated to promote/not promote certain topics and/or people. The scary thing is, this isn't a conspiracy myth. Back in 2014 FB got caught at experimenting at how algorithm changes affected the mood of users [1]. [1] https://www.forbes.com/sites/kashmirhill/2014/06/28/facebook-manipulated-689003-users-emotions-for-science/ https://www.forbes.com/sites/kashmirhill/2014/06/28/facebook...
- axg11 4y ago
  This is only possible for content that is flagged in a binary way. Most of these systems will also have a score for content and up/down rank the content based on the score. Using a score is actually a way to reduce the harm done by the algorithm. A low score will reduce the number of people that see content but not remove it completely, preventing false positive censorship.
- hooande 4y ago
  This is wildly unlikely. the content scope of twitter is huge, the data scale is huge and they have so many engineers that it's difficult to keep "manual human intervention" confidential. Even something as simple as a keyword search, ie "hunter biden laptop", would require searching tens of millions of strings per hour. Distributed across thousands of servers, integrated with whatever dev ops set up they have. It just seems like a huge pain, for little practical gain. Their scale is so huge that I doubt twitter has much manual control over what people see, even if they wanted it. It seems much easier to just multiply weights
fma 4y ago
I'm not sure how it works but it works in such a way to frustrate me. I rarely see the tweets of those I follow. Instead I see the ones they liked, I see the tweets of others that follow them, I see tweets of others who are similar to them. I see the tweets of everyone BUT the ones I follow. Hence I have a chrome plug in that cleans up their junk into a usable state.
- transitivebs 4y ago
  You can switch to the old reverse chronological feed (called "Latest Tweets") in twitter itself. I also use https://github.com/giuseppeg/refined-twitter-lite https://github.com/giuseppeg/refined-twitter-lite
  - sneak 4y ago
    But it automatically switches back to algo mode after a while, diametrically against your expressed wishes. I sincerely hope I never learn the names of the people at Twitter responsible for this user-abusing product decision.
    - saagarjha 4y ago
      It no longer does this.
  - fma 4y ago
    That is a very handy and unfortunately hidden feature. I just enabled it, and even knowing the feature exist I had to Google to know I need to tap that funky icon on mobile. Unfortunately for Twitter I will keep using the browser plug-in which also hides their ads (maybe they get paid for the display, even if hidden)
- Cipater 4y ago
  My way around that nonsense is ; 1. I use Tweetdeck on PC (browser). 2. I also prefer to add people to lists rather than following them. So on Tweetdeck I have multiple columns open at once, the first column is the people I follow (Tweetdeck presents them in chronological order, no messing about with similar users or liked tweets) and the rest of my columns are my lists. (F1, MotoGP, NBA, Space/Rocketry, Tech). I mainly use Twitter on PC but whenever I use it on mobile it's to look through my lists. The timeline is pretty much useless on all apps.
gotaquestion 4y ago
Great technical article, but why did you feel the need to add speculative ideological polarization too it? ``` Elon’s motivation is clear and consistent with his modus operandi. It’s the same reason he’s working so hard to build a sustainable colony on Mars, why he’s devoted resources to understanding the potential dangers of AI super-intelligence, and why he’s so insistent on combatting climate change. His guiding motivation is to improve humanity’s chances at a positive future. ``` That's certainly one interpretation.
- mhh__ 4y ago
  You could literally get away with murder with that kind of assumption of good-will.
- transitivebs 4y ago
  Fair enough. What I was trying to do was explain my motivation for digging deep into this topic. It has nothing to do with ideological polarization, though it may be a bit speculative.. this is mainly based on Tim Urban's excellent series of articles on why Elon Musk chooses to work on the problems he does. https://waitbutwhy.com/2017/03/elon-musk-post-series.html https://waitbutwhy.com/2017/03/elon-musk-post-series.html
russdill 4y ago
Open sourcing the algorithm would be such a huge boon to those wishing to game it. Those with the sufficient compute resources could run billions of variations of Twitter activity to perfect their messaging to get it trending. It'd be an arms race for those with enough money to keep up. Any chance of anything ever making it into trending in an organic way would be lost. For transparency, it might be nice for Twitter to continually release their algorithm from a certain time period ago, say 6 months
- mgraczyk 4y ago
  Unfortunately it wouldn't even look anything like that. "The algorithm" is probably a bunch of ML with some shitty product driven hacks on top, and those hacks will be trivial to game. Things like "if follower_count > X: spam_model_weight *= 0.1"
- mountainriver 4y ago
  I really like the idea of being able to select my own algorithms
jollybean 4y ago
Elon Musk is a free speech absolutist and does not understand information in the modern age, these kinds of abstractions seem to elude him. That, or he does, and he's a sinister operator, I believe the former. 'The Algorithm' is not that important to Twitter. What makes it work, is a semi-coordinated mass of users on a reliable platform. If that platform could be distributed, and, there was a critical mass, it could work, maybe. But 'open sourcing' is really not the issue at all. There isn't a line of code at Twitter that's valuable on it's own to anyone. You could hand over the Source Code to Trump & Co. for their deluded 'Truth Net' or whatever and it wouldn't make a difference.
- dale_glass 4y ago
  I don't think there are very many true free speech absolutists. We mostly hold positions as a means to an end, not as an end in themselves. Meaning, people declaring themselves free speech absolutists do so not because for them it's Free Speech, consequences be damned. But because they believe Free Speech is the way to the society they want to see. A good thought exercise is to try to imagine what would happen if an absolutist of any kind got their way in every way possible, and still hated the results. Really committing to free speech absolutism requires a very strong stomach very few people truly have. Back when Voat was a thing it was such a cess-pit that when r/TheDonald tried to move there, they couldn't take it. Voat disdained moderation, and as a result filtered for the kind of person who can look at a feed full of conspiracies, the most open and clear hate, fascism, racism and antisemitism, and every kind of porn that's banned on Reddit, and not even flinch at it. The denizens of r/TheDonald were used to setting rules on their turf, and were relentlessly mocked for it until they crawled back to Reddit. Voat eventually died. I think probably because in the end even the creators grew disillusioned with the community they had created, or because they realized it was only a matter of time before they got into serious legal trouble.
spullara 4y ago
It would be pretty easy if they just reverted it to reverse chronological and lose the engagement win they get by having it.
- saagarjha 4y ago
  Ok, but would mean people who use “the algorithm” wouldn’t get it anymore.
  - spullara 4y ago
    Correct. Which is what most of the complaints are about, knowingly or not.
Danielle69 4y ago
[dead]
saagarjha 4y ago
One thing this blog post doesn’t seem to mention, perhaps because it’s not Twitter-specific, is that all the code that drives “the algorithm” is heavily tied to everything else at Twitter and not easy to extricate and use by yourself. It might be fun to look at, as much as it is fun to look at a bunch of weights, but you probably are not going to be able to run it yourself.
- omegalulw 4y ago
  It's likely big neural nets, looking at the weights won't tell you much. What you can use instead to find biases is the training data, especially what sort of data the explicitly exclude/include. It's unlikely Twitter releases that.
throwaway5371 4y ago
SELECT t.* FROM tweets t, followers f WHERE t.author_id = f.author_id AND f.user_id = ? ORDER BY rand() there its open source now
- jdrc 4y ago
  i prefer date DESC
- transitivebs 4y ago
  reminds me of the infamous Conway sort (https://transitivebullsh.it/gaming-cs-interviews#083392b655d342aa8c85e34f86294e4b https://transitivebullsh.it/gaming-cs-interviews#083392b655d...)
ComradePhil 4y ago
"Open sourcing" the algorithm to a public platform will be very interesting... but even getting Twitter out of influence of foreign agents, whether they are bots [1], the 50 cent army [2] or powerful foreign state-level actors [3] will be a good start for Twitter. [1] https://twitter.com/johnkrausphotos/status/1517215349724352514 https://twitter.com/johnkrausphotos/status/15172153497243525... [2] https://www.washingtonpost.com/news/monkey-cage/wp/2016/05/19/the-chinese-government-fakes-nearly-450-million-social-media-comments-a-year-this-is-why/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/05/1... [3] https://twitter.com/Alwaleed_Talal/status/1514615956986757127 https://twitter.com/Alwaleed_Talal/status/151461595698675712...
JanneVee 4y ago
"The algorithm" is irredeemable. Its sole purpose is to prolong engagement so they can sneak ads in front of your eyeballs. Open sourcing it would not change its purpose and intended design. The design does not take into account that it can distort users worldview to the point of delusion and hinder or even set back progress of society if enough people are deluded in a group. I left twitter for the sole reason of that I saw a huge group of people feeding their worst tribal tendencies and celebrate the death of a human being. This is the very same mechanisms that has fed into pogroms and other atrocities through out history. "The algorithm" presented me with these celebratory tweets as a recommendation in my feed of only following highly technical people and projects with in software.
- UweSchmidt 4y ago
  But what if "the algorithm" is turns out to be rather basic? Maybe it just gives people more of what they like. Filter bubbles, polarization and herd mentality could be a natural, inevitable consequence of any Twitter-like UI and a central database.
  - pasiaj 4y ago
    The algo surely is quite basic. I believe JanneVee is saying that a "rather basic" algorithm that tries to maximize engagement drifts towards promoting the more divisive posts that amplify our tribalistic tendencies. And by giving more visibility to polarizing content it incentivises others to create more polarising content. The algorithm therefore is a positive feedback loop of polarization. It doesn't just show what we are, but amplifies our worst (or tribalistic) tendencies. Relevant Howard Stern reference: https://www.youtube.com/watch?v=9G6xu-J_Dmc https://www.youtube.com/watch?v=9G6xu-J_Dmc
  - JanneVee 4y ago
    The problem is that I tried to put myself into a software development filter bubble but yet the herd mentality and polarization bled through the filter. It is not a single issue of "algorithm = bad". It is multiple issues intersecting with psychology. It is the subversion of willpower by hijacking the dopamine that you feel rewarded to keep scrolling, it is the fracturing the shared reality by presenting different content to different people depending on what they engaged with and a number of other issues... all this makes it a complicated mess where one set of people starve themselves to death because they think they are fat according to their feed and other where people eat themselves to death because fat is beautiful according to their reality. You can fix one issue but then the next issue props up and all of a sudden the complications of human psychology makes the unintended consequences even more dire. All this to make you watch ads and buy things.
jdrc 4y ago
That would imply that twitter becomes a nonprofit
- hef19898 4y ago
  Isn't that Elon's plan? At least he claims it is, so it must be true.
  - jdrc 4y ago
    i may have missed that part
    - hef19898 4y ago
      I exaggerated, but he said that he doesn't care about Twitter being profitable once he took it private. And he's all about free speech. Not that I believe any of this crap so.
- dbrueck 4y ago
  It already IS a nonprofit. /rimshot
renewiltord 4y ago
Scenes when Twitter algo is literally just what you and your friends like and retweet - not even clicks or anything. Wonder if Jack Dorsey will have been responsible for the end of democracy then.
lysecret 4y ago
I have a theory about current state of the are of recommendations, automation or ML in general. I think a lot of the valid criticisms we see with recommendation systems (building of bubbles, extremization etc.) is due to the simplicity of the algorithms. In reinforcement learning there is a notion of exploration and exploitation. All of supervised ML can in a way be seen as exploitation. Incorporating ideas of exploration in a clever way into the standard recommendation algos, I think could be the solution to a lot of the mentioned problems.
philliphaydon 4y ago
Imagine if it was just a chronological order of tweets and retweets from your followers. And that’s it. No algorithm. Twitter would be so much better.
- Avshalom 4y ago
  there's literally a button for that, you click the three stars at the top and select "see latest tweets"
  - philliphaydon 4y ago
    Any idea how to do this on desktop? I only really reply to people on mobile and rarely browse Twitter on there.
    - Avshalom 4y ago
      On the non-mobile website there's the same icon at the top of your feed. I don't know about any of the desktop clients
- dr_dshiv 4y ago
  Agree. And imagine if, instead of a single “good for all” black box algorithm, one could simply apply custom filters. “See less of this” or “always show” or “no retweets”. People are obsessed with personalization algorithms and disappointed when this involves giving people control to personalize. As though personal control isn’t magical enough (in comparison to the magic of the algorithm).
- megous 4y ago
  That's how I use twitter. I had to implement this complicated concept myself by scraping tweets from accounts I follow into local database, and then displaying data from that database in chronological order.
- Saint_Genet 4y ago
  That's what the timeline setting does the mobile client at least (not counting the occasional ad)
lvl102 4y ago
This notion that open sourcing algo would resolve “speech” issue on social media is laughable. All it will do is reduce the contention down to said algo and how it could be manipulated.
- transitivebs 4y ago
  Yes and no. The goal here is to increase transparency and optionality around twitter's core feed. There will always be difficult, contentious human problems with a platform like twitter — but instead of writing it off as "laughable", there's a lot that can be done to improve the status quo. And imho that's a very impactful goal worth striving for.
p5v 4y ago
Sad to see another blog that has no RSS feed :( People, please, I am begging you - turn those back on. It's the only way we can stay up-to-date with your amazing content. Anyway, this is what I came to say. A great post, full of insights. We have discovered quite a few of them ourselves, while building https://murmel.social https://murmel.social, and I envy the author a little for publishing the knowledge first. Next time ;)
- transitivebs 4y ago
  Hey haha I literally added this yesterday :) https://github.com/transitive-bullshit/nextjs-notion-starter-kit/pull/273 https://github.com/transitive-bullshit/nextjs-notion-starter... https://transitivebullsh.it/feed.xml https://transitivebullsh.it/feed.xml Great call!

142 points by transitivebs 4y ago

Even better than open-sourcing the algorithm would be giving people options on which algorithm they get. I bet you could get people to pay for Twitter blue just to get reverse chronological back or whatever it is.

transitivebs 4y ago
Agreed. The goal should be more transparency and optionality around twitter's algorithmic feed. Note that everyone has access to the reverse chronological feed called "Latest tweets". See here for how to switch: https://twitter.com/TwitterSupport/status/1447971179185295364 https://twitter.com/TwitterSupport/status/144797117918529536...? Also agreed that twitter pro would be a great place for twitter to add more options.
shafyy 4y ago
Still, defaults matter and most people will not change their algorithm settings. Ironically, those are exactly the people who are impacted most from an algorithmic timeline that tries to maximize engagement.

transitivebs 4y ago

Author here. I needed to split this up into multiple posts because it required a lot deeper dive than I had originally expected. What I'm really interested in, however, is help with answering the following questions: What would open sourcing the Twitter algorithm actually look like? Would it be possible to abstract away all of the engineering complexity that it takes to run a global, production system like Twitter and produce an OSS spec or API that is actually useful? Would it be possible to produce meaningful results without access to Twitter’s full data set? What does meaningful even mean here? How would we define success? What would need to happen in order to make this a reality? What are some practical proposals that would help improve the status quo?

deleted 4y ago
[deleted]
karpierz 4y ago
What do you mean by "open sourcing" the Twitter algorithm? What are the actual requirements that you're looking to satisfy by doing this?
- transitivebs 4y ago
  Great questions. These are exactly the types of questions I'm trying to get more people to think about — and the reason I wrote the article above. It's not well-defined, but the goal imho should be to allow more transparency and optionality around Twitter's main feed UX. Open sourcing twitter's algorithmic feed would be a monumental task that's honestly unlikely to ever come to fruition, Elon or no Elon. BUT twitter could certainly do more to move in that direction. > What are the actual requirements that you're looking to satisfy by doing this? If it improves transparency around how twitter's algorithm works.. if it introduces more optionality into twitter's main UX (and isn't just a chrome extension, for instance), if it's coming from twitter itself and not a third-party.. if the effects of different factors on relevancy signals are made more transparent (like being able to play around with a simulated tweet under different conditions and seeing the resulting relevancy scores for a set of example users).. there's just so much that twitter could do here to improve transparency around how their feed works. And I'm generally in favor of anything what will move the status quo. In Elon's words “Civilizational risk is decreased the more we can increase the trust in Twitter as a public platform.”
  - MrMan 4y ago
    Actually the more I trust Twitter the more risk I am exposed to. Avoiding Twitter and discounting anything to do with Twitter is a better risk management policy for me.
gmm1990 4y ago
Some sort of explanation of why am I seeing this tweet would go a long way. like tweet was displayed because 50% interest in dogs 20% friends liked it 30% generally popular

Ourgon 4y ago

How it works is interesting in itself but it is not the reason why these 'algorithmic feeds' have gained the nefarious reputation they have. The real problem with this type of technology is the way it allows the proprietor of the service to present its users a tailored view of whatever content the service offers. The way this view is tailored depends on what the proprietors' aim is: Twitter uses algorithmic feeds to present its users with an ideologically driven view of whatever is being discussed, this can easily be seen by reading the first comments on contentious issues where they nearly always push some 'progressive' comments to the front, even when those comments are clearly less popular with users than whatever other comments follow. Comments critical of 'progressive' issues often require one or more extra clicks to be made visible. Sometimes this leads to the first page being devoid of comments, requiring a click to access whatever comments are present where those comments are nearly invariably critical of the 'progressive' issue being discussed. This does not happen when there are positive comments on those 'progressive' issues since these are presented directly on the first page. TikTok and Youtube also use algorithmic feeds to channel users but their algorithms seem to be tailored to keep users on the site/app for as long as possible.

transitivebs 4y ago
I agree with your sentiments. Just keep in mind that the whole point of this discussion is to move towards more transparency and optionality around how Twitter works. As you pointed out, not everything is algorithmic. There are definitely human factors at play, but without a more solid understanding of how twitter's algorithmic feed actually works, the discussion will remain hand wavy at best. Just my 2 cents :)
cpr 4y ago
This was abundantly clear when, e.g., every Trump tweet had the first (always highly negative) replies from a couple of crazy doctor brothers (twins?).
- mikkergp 4y ago
  I always interpreted this as someone wrote a script and learned to game the algorithm, you think there’s a line of code in there that says trump_replies.unshift(crazy_doctor_brother_tweets) in there somewhere?

giancarlostoro 4y ago

I honestly just use Twitter’s historic feed. The day they remove this ability is the day I delete my account.

AzzieElbab 4y ago
I use lists which is something they seemed to have forgotten about. If Twitter removes lists I will use something like hootsuite
- adolph 4y ago
  Lists and list sharing is reminiscent of Google Reader in some way I can't quite remember. The lack of ads is striking whenever I follow a link to a normal timeline.
prawn 4y ago
I use https://tweetdeck.twitter.com/ https://tweetdeck.twitter.com/ which has the ordering I expect, allows arbitrary columns and has no ads (though this might be because of ad blocking). I have five columns: personal account, replies to it, side project, replies to that, messages. Only thing I miss from the old OSX Twitter client is the ability to copy and paste in images.

twtrgrunt 4y ago

still speculation without being an employee (even if it did some research from blogs which seems to be more then most do) twitter should just write their own post and shut everyone up already. idk why the company is so secretive about everything. its clearly not doing them any favors

transitivebs 4y ago
I'd love for twitter to take a more active role in this, but from what I understand, twitter employees are forced to be pretty tight-lipped about these things. I tried to reference as many of the official sources as possible (in addition to drawing on my past experienced at Amazon + Facebook).
- twtrgrunt 4y ago
  yeah these companies act like were all protecting the coca cola formula or building nukes or something. like really, its blocks of text being put in order. the hr, pr, comms, leaders or w/e seem so disconnected. im sure there is some bs reason like were protecting the company, i think its misguided being so closed off though and missing the bigger picture of building trust with users.
  - transitivebs 4y ago
    especially on a platform like twitter :)
    - twtrgrunt 4y ago
      its almost like its a place where people can interact directly. thanks for calling out scale and engineering problems btw! sometimes i think people think we're sitting around all day throwing darts at a wall of pictures of conservatives to pick who to ban next.
  - notriddle 4y ago
    > yeah these companies act like were all protecting the coca cola formula or building nukes or something. like really, its blocks of text being put in order The Coca-Cola formula seems like a pretty good analogy, honestly. Most people can't tell the difference between different cola recipes, and there probably isn't anything in the recipe itself that's responsible for the company's success. Not as much as marketing, and simply being in the right place at the right time, anyway.

Barrin92 4y ago

I always wonder what people expect when they talk about twitter's algorithm as if this some sort of Deus Ex level AI developed in the depths of Area 51. If anyone wants twitter's algorithm I guess all you need is to download some arxiv papers on state of the art recommender systems and it's probably going to look like that.

transitivebs 4y ago
Yes and no. Twitter's scale and real-time nature make it a difficult beast. Their network graph contains hundreds of millions of nodes and billions of edges. And it's constantly updating. So any graph ML algorithms you want to use have to deal an underlying graph that's eventually consistent at best — and oftentimes very sparse in terms of feature availability.
- transitivebs 4y ago
  Also yeah, most people talk about "Twitter's algorithm" but have no idea what it is they're talking about — that's exactly why I wanted to write on this topic :)
- mountainriver 4y ago
  Yup and temporal graphs are a super hard problem without a lot of great ML solutions currently
axg11 4y ago
I work on FAANG-scale recommender systems. Most of the time, due to the scale of the problem, the algorithms powering these systems are not anywhere near state of the art. On the other hand, they are highly optimised for the situation. You can bet that every aspect of the Twitter algorithmic feed has been thoroughly debated, thought through and most importantly A/B tested.
- ceeplusplus 4y ago
  I worked at one of the FAANGs and the algorithms were definitely state of the art in some ways. Maybe they didn't use the latest models but the sheer size of, for example models, was huge. Easily 500+ features inputted and hundreds of millions of parameters for deep models.

icallbullshit 4y ago

Yet, any company who open sources an algorithm can simply apply any private alternative algorithms using any alternative/undisclosed data sources to any undisclosed subset of users as they wish. Hell, whose to say that some people aren't chosen for long term "experiments" with different behavioral/belief modification goals? That's what I would do.

transitivebs 4y ago
Good point. Even if there were some sort of open source spec like the W3C specs, for instance, Twitter would still be a black box that could choose to conform to the spec or not choose to. What we need is something more enforceable. So either you do something completely green field like https://blueskyweb.xyz/ https://blueskyweb.xyz/, or you spend $43B & roll up your sleeves to try and fix the problems with the current platform.

politician 4y ago

The easiest way to implement “open source the algorithm” is to delete the AI algorithm, revert the defaults to reverse-chronological feed, then release that.

transitivebs 4y ago
Sounds great. But it will never happen because of business incentives. I'm focused on trying to move the conversation towards more practical solutions.
- politician 4y ago
  If Elon takes Twitter private, I doubt that you can argue that the quickest practical solution is other than as I’ve suggested. In the secretive context of the current non-profitable public Twitter model, yes, I agree, they cannot do anything useful. Not sure why you got a downvote though, so have my upvote.
  - transitivebs 4y ago
    Oh yeah, if Elon takes twitter private, then all bets are off :) Though I personally think throwing out any form of algorithmic feed wouldn't happen. There are just too many business and UX reasons it makes sense, which is why every major social platform uses them (https://www.socialmediatoday.com/social-networks/truth-about-algorithms-and-why-you-should-expect-see-more-them https://www.socialmediatoday.com/social-networks/truth-about...).
prawn 4y ago
I don't follow many people on any social network and endeavour to read everything those people post. Like you, I want a straight sort of those items. However, I assume the majority of people follow more than they can read, so a huge effort goes in to sifting through the fire hose to provide a filtered taste. Someone with a short following list can see the drama if they look at the huge list of replies that a popular account might get. So much cruft.

creakingstairs 4y ago

I've tinkered with coming up with my own algorithm for YouTube using their API (albeit very basic ones) and I quite liked the results. I _tried_to make a front end and gave up because of the scope creep. I wonder why platforms wouldn't just offer different algorithms. e.g. historical, maximise-time-on-screen or exploration. Most of the users would be nudged into what the platform wants anyway, and they will be able to claim moral superiority over other platforms that doesn't offer this. Heck, you could even charge for this feature.

transitivebs 4y ago
https://stratechery.com/2022/back-to-the-future-of-twitter/ https://stratechery.com/2022/back-to-the-future-of-twitter/ has a great breakdown of Twitter's history in this respect. TL;DR they used to have an open API and various third-party clients, each of which had their own flavor. BUT this made it really hard for twitter to guarantee a good UX and even harder for twitter to implement an ad-driven business model.
- creakingstairs 4y ago
  I'm not arguing for an open API, I'm arguing for more options in content algorithms. These would be under their sole control. Aside from that, thanks for the link. That is a great post!
  - transitivebs 4y ago
    I think at a practical level, opening up their API for non-watered-down third-party clients would be the only true way to add more optionality to twitter's feed UX. Since I don't see open sourcing the actual feed algorithm to be something that would practically happen any time soon (which makes me a sad panda).
bobthepanda 4y ago
I would imagine Twitter spends a lot of time maintaining the one algorithm. More might involve a lot more work for diminishing upside.

tomschlick 4y ago

Other than a lot of people preferring the reverse chronological feed, I think the biggest complaint I see these days is the perception that the algorithm is being manually manipulated to promote/not promote certain topics and/or people. Given that Twitter/Facebook/Reddit/Tiktok are the behemoths in the room that all have the power to sway public opinion by either promoting or suppressing information, I think they should be regulated so that any topic that was promoted or suppressed by the algorithm or manual human intervention should show itself on the content just like an ad does. "This content has been quarantined by [Twitter Staff / Automated Review] and is only available to those with the direct url." would be a perfect example of such a message. From there, users should have control over if they want to see such quarantined content or if they want a "safe" experience that Twitter curates for them.

mschuster91 4y ago
> I think the biggest complaint I see these days is the perception that the algorithm is being manually manipulated to promote/not promote certain topics and/or people. The scary thing is, this isn't a conspiracy myth. Back in 2014 FB got caught at experimenting at how algorithm changes affected the mood of users [1]. [1] https://www.forbes.com/sites/kashmirhill/2014/06/28/facebook-manipulated-689003-users-emotions-for-science/ https://www.forbes.com/sites/kashmirhill/2014/06/28/facebook...
axg11 4y ago
This is only possible for content that is flagged in a binary way. Most of these systems will also have a score for content and up/down rank the content based on the score. Using a score is actually a way to reduce the harm done by the algorithm. A low score will reduce the number of people that see content but not remove it completely, preventing false positive censorship.
hooande 4y ago
This is wildly unlikely. the content scope of twitter is huge, the data scale is huge and they have so many engineers that it's difficult to keep "manual human intervention" confidential. Even something as simple as a keyword search, ie "hunter biden laptop", would require searching tens of millions of strings per hour. Distributed across thousands of servers, integrated with whatever dev ops set up they have. It just seems like a huge pain, for little practical gain. Their scale is so huge that I doubt twitter has much manual control over what people see, even if they wanted it. It seems much easier to just multiply weights

fma 4y ago

I'm not sure how it works but it works in such a way to frustrate me. I rarely see the tweets of those I follow. Instead I see the ones they liked, I see the tweets of others that follow them, I see tweets of others who are similar to them. I see the tweets of everyone BUT the ones I follow. Hence I have a chrome plug in that cleans up their junk into a usable state.

transitivebs 4y ago
You can switch to the old reverse chronological feed (called "Latest Tweets") in twitter itself. I also use https://github.com/giuseppeg/refined-twitter-lite https://github.com/giuseppeg/refined-twitter-lite
- sneak 4y ago
  But it automatically switches back to algo mode after a while, diametrically against your expressed wishes. I sincerely hope I never learn the names of the people at Twitter responsible for this user-abusing product decision.
  - saagarjha 4y ago
    It no longer does this.
- fma 4y ago
  That is a very handy and unfortunately hidden feature. I just enabled it, and even knowing the feature exist I had to Google to know I need to tap that funky icon on mobile. Unfortunately for Twitter I will keep using the browser plug-in which also hides their ads (maybe they get paid for the display, even if hidden)
Cipater 4y ago
My way around that nonsense is ; 1. I use Tweetdeck on PC (browser). 2. I also prefer to add people to lists rather than following them. So on Tweetdeck I have multiple columns open at once, the first column is the people I follow (Tweetdeck presents them in chronological order, no messing about with similar users or liked tweets) and the rest of my columns are my lists. (F1, MotoGP, NBA, Space/Rocketry, Tech). I mainly use Twitter on PC but whenever I use it on mobile it's to look through my lists. The timeline is pretty much useless on all apps.

gotaquestion 4y ago

Great technical article, but why did you feel the need to add speculative ideological polarization too it? ``` Elon’s motivation is clear and consistent with his modus operandi. It’s the same reason he’s working so hard to build a sustainable colony on Mars, why he’s devoted resources to understanding the potential dangers of AI super-intelligence, and why he’s so insistent on combatting climate change. His guiding motivation is to improve humanity’s chances at a positive future. ``` That's certainly one interpretation.

mhh__ 4y ago
You could literally get away with murder with that kind of assumption of good-will.
transitivebs 4y ago
Fair enough. What I was trying to do was explain my motivation for digging deep into this topic. It has nothing to do with ideological polarization, though it may be a bit speculative.. this is mainly based on Tim Urban's excellent series of articles on why Elon Musk chooses to work on the problems he does. https://waitbutwhy.com/2017/03/elon-musk-post-series.html https://waitbutwhy.com/2017/03/elon-musk-post-series.html

russdill 4y ago

Open sourcing the algorithm would be such a huge boon to those wishing to game it. Those with the sufficient compute resources could run billions of variations of Twitter activity to perfect their messaging to get it trending. It'd be an arms race for those with enough money to keep up. Any chance of anything ever making it into trending in an organic way would be lost. For transparency, it might be nice for Twitter to continually release their algorithm from a certain time period ago, say 6 months

mgraczyk 4y ago
Unfortunately it wouldn't even look anything like that. "The algorithm" is probably a bunch of ML with some shitty product driven hacks on top, and those hacks will be trivial to game. Things like "if follower_count > X: spam_model_weight *= 0.1"
mountainriver 4y ago
I really like the idea of being able to select my own algorithms

jollybean 4y ago

Elon Musk is a free speech absolutist and does not understand information in the modern age, these kinds of abstractions seem to elude him. That, or he does, and he's a sinister operator, I believe the former. 'The Algorithm' is not that important to Twitter. What makes it work, is a semi-coordinated mass of users on a reliable platform. If that platform could be distributed, and, there was a critical mass, it could work, maybe. But 'open sourcing' is really not the issue at all. There isn't a line of code at Twitter that's valuable on it's own to anyone. You could hand over the Source Code to Trump & Co. for their deluded 'Truth Net' or whatever and it wouldn't make a difference.

dale_glass 4y ago
I don't think there are very many true free speech absolutists. We mostly hold positions as a means to an end, not as an end in themselves. Meaning, people declaring themselves free speech absolutists do so not because for them it's Free Speech, consequences be damned. But because they believe Free Speech is the way to the society they want to see. A good thought exercise is to try to imagine what would happen if an absolutist of any kind got their way in every way possible, and still hated the results. Really committing to free speech absolutism requires a very strong stomach very few people truly have. Back when Voat was a thing it was such a cess-pit that when r/TheDonald tried to move there, they couldn't take it. Voat disdained moderation, and as a result filtered for the kind of person who can look at a feed full of conspiracies, the most open and clear hate, fascism, racism and antisemitism, and every kind of porn that's banned on Reddit, and not even flinch at it. The denizens of r/TheDonald were used to setting rules on their turf, and were relentlessly mocked for it until they crawled back to Reddit. Voat eventually died. I think probably because in the end even the creators grew disillusioned with the community they had created, or because they realized it was only a matter of time before they got into serious legal trouble.

spullara 4y ago

It would be pretty easy if they just reverted it to reverse chronological and lose the engagement win they get by having it.

saagarjha 4y ago
Ok, but would mean people who use “the algorithm” wouldn’t get it anymore.
- spullara 4y ago
  Correct. Which is what most of the complaints are about, knowingly or not.

Danielle69 4y ago

[dead]

saagarjha 4y ago

One thing this blog post doesn’t seem to mention, perhaps because it’s not Twitter-specific, is that all the code that drives “the algorithm” is heavily tied to everything else at Twitter and not easy to extricate and use by yourself. It might be fun to look at, as much as it is fun to look at a bunch of weights, but you probably are not going to be able to run it yourself.

omegalulw 4y ago
It's likely big neural nets, looking at the weights won't tell you much. What you can use instead to find biases is the training data, especially what sort of data the explicitly exclude/include. It's unlikely Twitter releases that.

throwaway5371 4y ago

SELECT t.* FROM tweets t, followers f WHERE t.author_id = f.author_id AND f.user_id = ? ORDER BY rand() there its open source now

jdrc 4y ago
i prefer date DESC
transitivebs 4y ago
reminds me of the infamous Conway sort (https://transitivebullsh.it/gaming-cs-interviews#083392b655d342aa8c85e34f86294e4b https://transitivebullsh.it/gaming-cs-interviews#083392b655d...)

ComradePhil 4y ago

"Open sourcing" the algorithm to a public platform will be very interesting... but even getting Twitter out of influence of foreign agents, whether they are bots [1], the 50 cent army [2] or powerful foreign state-level actors [3] will be a good start for Twitter. [1] https://twitter.com/johnkrausphotos/status/1517215349724352514 https://twitter.com/johnkrausphotos/status/15172153497243525... [2] https://www.washingtonpost.com/news/monkey-cage/wp/2016/05/19/the-chinese-government-fakes-nearly-450-million-social-media-comments-a-year-this-is-why/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/05/1... [3] https://twitter.com/Alwaleed_Talal/status/1514615956986757127 https://twitter.com/Alwaleed_Talal/status/151461595698675712...

JanneVee 4y ago

"The algorithm" is irredeemable. Its sole purpose is to prolong engagement so they can sneak ads in front of your eyeballs. Open sourcing it would not change its purpose and intended design. The design does not take into account that it can distort users worldview to the point of delusion and hinder or even set back progress of society if enough people are deluded in a group. I left twitter for the sole reason of that I saw a huge group of people feeding their worst tribal tendencies and celebrate the death of a human being. This is the very same mechanisms that has fed into pogroms and other atrocities through out history. "The algorithm" presented me with these celebratory tweets as a recommendation in my feed of only following highly technical people and projects with in software.

UweSchmidt 4y ago
But what if "the algorithm" is turns out to be rather basic? Maybe it just gives people more of what they like. Filter bubbles, polarization and herd mentality could be a natural, inevitable consequence of any Twitter-like UI and a central database.
- pasiaj 4y ago
  The algo surely is quite basic. I believe JanneVee is saying that a "rather basic" algorithm that tries to maximize engagement drifts towards promoting the more divisive posts that amplify our tribalistic tendencies. And by giving more visibility to polarizing content it incentivises others to create more polarising content. The algorithm therefore is a positive feedback loop of polarization. It doesn't just show what we are, but amplifies our worst (or tribalistic) tendencies. Relevant Howard Stern reference: https://www.youtube.com/watch?v=9G6xu-J_Dmc https://www.youtube.com/watch?v=9G6xu-J_Dmc
- JanneVee 4y ago
  The problem is that I tried to put myself into a software development filter bubble but yet the herd mentality and polarization bled through the filter. It is not a single issue of "algorithm = bad". It is multiple issues intersecting with psychology. It is the subversion of willpower by hijacking the dopamine that you feel rewarded to keep scrolling, it is the fracturing the shared reality by presenting different content to different people depending on what they engaged with and a number of other issues... all this makes it a complicated mess where one set of people starve themselves to death because they think they are fat according to their feed and other where people eat themselves to death because fat is beautiful according to their reality. You can fix one issue but then the next issue props up and all of a sudden the complications of human psychology makes the unintended consequences even more dire. All this to make you watch ads and buy things.

jdrc 4y ago

That would imply that twitter becomes a nonprofit

hef19898 4y ago
Isn't that Elon's plan? At least he claims it is, so it must be true.
- jdrc 4y ago
  i may have missed that part
  - hef19898 4y ago
    I exaggerated, but he said that he doesn't care about Twitter being profitable once he took it private. And he's all about free speech. Not that I believe any of this crap so.
dbrueck 4y ago
It already IS a nonprofit. /rimshot

renewiltord 4y ago

Scenes when Twitter algo is literally just what you and your friends like and retweet - not even clicks or anything. Wonder if Jack Dorsey will have been responsible for the end of democracy then.

lysecret 4y ago

I have a theory about current state of the are of recommendations, automation or ML in general. I think a lot of the valid criticisms we see with recommendation systems (building of bubbles, extremization etc.) is due to the simplicity of the algorithms. In reinforcement learning there is a notion of exploration and exploitation. All of supervised ML can in a way be seen as exploitation. Incorporating ideas of exploration in a clever way into the standard recommendation algos, I think could be the solution to a lot of the mentioned problems.

philliphaydon 4y ago

Imagine if it was just a chronological order of tweets and retweets from your followers. And that’s it. No algorithm. Twitter would be so much better.

Avshalom 4y ago
there's literally a button for that, you click the three stars at the top and select "see latest tweets"
- philliphaydon 4y ago
  Any idea how to do this on desktop? I only really reply to people on mobile and rarely browse Twitter on there.
  - Avshalom 4y ago
    On the non-mobile website there's the same icon at the top of your feed. I don't know about any of the desktop clients
dr_dshiv 4y ago
Agree. And imagine if, instead of a single “good for all” black box algorithm, one could simply apply custom filters. “See less of this” or “always show” or “no retweets”. People are obsessed with personalization algorithms and disappointed when this involves giving people control to personalize. As though personal control isn’t magical enough (in comparison to the magic of the algorithm).
megous 4y ago
That's how I use twitter. I had to implement this complicated concept myself by scraping tweets from accounts I follow into local database, and then displaying data from that database in chronological order.
Saint_Genet 4y ago
That's what the timeline setting does the mobile client at least (not counting the occasional ad)

lvl102 4y ago

This notion that open sourcing algo would resolve “speech” issue on social media is laughable. All it will do is reduce the contention down to said algo and how it could be manipulated.

transitivebs 4y ago
Yes and no. The goal here is to increase transparency and optionality around twitter's core feed. There will always be difficult, contentious human problems with a platform like twitter — but instead of writing it off as "laughable", there's a lot that can be done to improve the status quo. And imho that's a very impactful goal worth striving for.

p5v 4y ago

Sad to see another blog that has no RSS feed :( People, please, I am begging you - turn those back on. It's the only way we can stay up-to-date with your amazing content. Anyway, this is what I came to say. A great post, full of insights. We have discovered quite a few of them ourselves, while building https://murmel.social https://murmel.social, and I envy the author a little for publishing the knowledge first. Next time ;)

transitivebs 4y ago
Hey haha I literally added this yesterday :) https://github.com/transitive-bullshit/nextjs-notion-starter-kit/pull/273 https://github.com/transitive-bullshit/nextjs-notion-starter... https://transitivebullsh.it/feed.xml https://transitivebullsh.it/feed.xml Great call!