4 ms·
There's not an algorithm to order timelines?
by influx 4y ago
There's not an algorithm to order timelines?
- jonwinstanley 4y agoIf it’s a list of tweets ordered based on a kajillion ML data points that varies per user is it still an algorithm? And does every user have their own algorithm? And could it be made readable to a human?
- abduhl 4y agoYes, irrelevant, and the code somewhere is readable. Take an example like gravity: is there an algorithm for how gravity acts even though it varies per each molecule based on a kajillion other molecules? Of course there is.
- pavlov 4y ago
- zbentley 4y agoComputer systems, even AI/ML-driven ones, aren't uncomprehended the way some areas of science that we haven't developed a strong understanding of are. Systems comprised of (or generated by) comprehensible parts can be comprehended, we just often opt not to put in that work. There's a reason model interpretability is such a significant and increasingly-demanded area of expertise.
- marcinzm 4y agoIf I have your DNA can I recreate a perfect copy of you as you are now including your mind? It is after all the code you were made with.
- vasco 4y agoThe point is not to recreate it or explain it, or to have a way to say "tweet X will get rank Y" but to open source what there is. It doesn't matter if no single person can explain it, it'll still be open source and readable and the types of inputs will be known. But this doesn't look like a discussion on open source and more about politics at this point. I for one haven't worked at FAANG and think it'll be super interesting to read this code, I can't believe software engineers are complaining about a potentially super complex bleeding edge codebase for timeline recommendations / ranking being open sourced. This is going to be great reading material if it ever gets published!
- abduhl 4y agoNobody is trying to “recreate a perfect copy” of Twitter. Do you think that code which produces random numbers is also not an algorithm because it doesn’t recreate a perfect copy of a previous random number?
- stjohnswarts 4y agoNo you can't, you can only make a twin, but the mind is shaped by experience also and not just genetics. As is the body.
- ironmagma 4y agoIn that case, the definition of “the algorithm” is the training set.
- JumpCrisscross 4y ago> the definition of “the algorithm” is the training set Or the trained model itself. There are people looking for intentional bias. But the insidiousness of the problem likely arises from unintentional bias. Letting researchers brute force the ranking models with hypotheticals could be a win win.
- ironmagma 4y agoIs there any reason to not think that like most human-created systems it has a mixture of intentional and unintentional bias?
- lamontcg 4y agoTwitter itself is probably the training set for twitter.
- ironmagma 4y agoIn that case, the algorithm is the labels.
- deleted 4y ago[deleted]
- dmonitor 4y agoTurns out, it's been RNG this entire time.
- krapp 4y agoThere probably is not "an algorithm" on a site as large and complex as Twitter, no. There are probably dozens if not hundreds of algorithms spread throughout the codebase which affect the timeline for individual users, possibly even code entirely self-generated by ML systems.
- onlyrealcuzzo 4y agoIn the end, it's still code that is being executed, which can be called "an algorithm". Algorithms are under no obligation to be simple or only include "one algorithm".
- krapp 4y agoSo then "open sourcing the algorithm" likely means opens sourcing the entire Twitter codebase?
- baryphonic 4y agoAt this point, what harm would that do to Twitter? The company's tech has been stagnant for years, and the only real features recently have been attempts to exploit Twitter's network effects to ape Clubhouse & Substack. (Seems successful for the former, kind of a failure for the latter.)
- stjohnswarts 4y agodo we know? has any angy ex-twitter engineer revealed the secret sauce to the world?
- captainmuon 4y agoWell sure there is "ORDER BY" something, and there are batch jobs to calculate this something from different variables. Maybe it is partially a ML black box, but I'm pretty sure they are not flying completely blind. There's got to be an internal Wiki or some PPTs where they list what goes into the mix. Number of likes, retweets. Attached media, hashtags, trending or not. Do they do sentiment analysis and use positivity as a factor? Do they do PageRank so that likes from important people have more weight? Are there manual debuffs for certain topics? Are certain posts removed or added in a second pass? These are all answerable questions in principle.
- Hamuko 4y agoORDER BY timestamp DESC;
- talideon 4y agoThe one true algorithm!
- dylan604 4y agono self respecting PM would ever allow something so trite and simple. this was obviously copy&pasted from stackoverflow. something like this would never web scale, so obviously they need somethign written from scratch preferably in Go/Rust/Node/anythingButSQL
- jdrc 4y agojust birds poking random tweets
- apgwoz 4y agoTurns out they never actually scaled their mailbox pattern. As a result, what you see is what they delivered before timeout.