3 ms·

> opens sourcing Twitter algorithm People need to stop saying this. It's nonsense. Or maybe keep saying it so we know your clueless about how anything works th

by tayo42 5y ago

> opens sourcing Twitter algorithm People need to stop saying this. It's nonsense. Or maybe keep saying it so we know your clueless about how anything works that your commenting on

JumpCrisscross 5y ago
> It's nonsense Why? (Honest question.)
- burnished 5y ago
  Also genuine, why would a person believe that would happen? The foremost thing that comes to mind is they'd be giving something of great value and receiving nothing in return.
- tayo42 5y ago
  which part do you want open sourced? timelines are generated after going through machine learning and then multiple services. even then, some of the infrastructure limits what you see in the end. https://blog.twitter.com/engineering/en_us/topics/infrastructure/2017/building-and-serving-conversations-on-twitter https://blog.twitter.com/engineering/en_us/topics/infrastruc... > Unfortunately, Haplo has its limits. Since we try to pull every single Tweet ID in a conversation every time a conversation is loaded in real-time, this is potentially an unbounded amount of data. We’ve found that if our platform tries to load more than a certain number of entries from our cache at once, our cache request latency will spike, and we begin to time-out on a significant fraction of read requests. Because of this, there is a limit of thousands of Tweets per conversation tree in this cache. even just showing an ad is a ton work has a bunch of services involved https://blog.twitter.com/engineering/en_us/topics/infrastructure/2021/sharding-simplification-and-twitters-ads-serving-platform https://blog.twitter.com/engineering/en_us/topics/infrastruc... Other stuff that I think explains the complexity https://blog.twitter.com/engineering/en_us/topics/insights/2017/using-deep-learning-at-scale-in-twitters-timelines https://blog.twitter.com/engineering/en_us/topics/insights/2... https://blog.twitter.com/engineering/en_us/topics/insights/2020/using-machine-learning-to-predict-the-value-of-ad-requests https://blog.twitter.com/engineering/en_us/topics/insights/2... https://blog.twitter.com/engineering/en_us/topics/insights/2018/twittertensorflow https://blog.twitter.com/engineering/en_us/topics/insights/2... there's no single algorithm, its a bunch of parts coming together from hundreds of engineers and multiple services.
VirusNewbie 5y ago
You think there is no such thing as OSS models?
pbreit 5y ago
Please explain. I can easily envision much more transparency on how the feed works.
- tayo42 5y ago
  Transparency i can agree with, i think twitter does a horrible job at that. especially explaining the rumored shadow bans and stuff. But i think thats different then open source code

by tayo42 5y ago

> opens sourcing Twitter algorithm People need to stop saying this. It's nonsense. Or maybe keep saying it so we know your clueless about how anything works that your commenting on

JumpCrisscross 5y ago

> It's nonsense Why? (Honest question.)

burnished 5y ago
Also genuine, why would a person believe that would happen? The foremost thing that comes to mind is they'd be giving something of great value and receiving nothing in return.
tayo42 5y ago
which part do you want open sourced? timelines are generated after going through machine learning and then multiple services. even then, some of the infrastructure limits what you see in the end. https://blog.twitter.com/engineering/en_us/topics/infrastructure/2017/building-and-serving-conversations-on-twitter https://blog.twitter.com/engineering/en_us/topics/infrastruc... > Unfortunately, Haplo has its limits. Since we try to pull every single Tweet ID in a conversation every time a conversation is loaded in real-time, this is potentially an unbounded amount of data. We’ve found that if our platform tries to load more than a certain number of entries from our cache at once, our cache request latency will spike, and we begin to time-out on a significant fraction of read requests. Because of this, there is a limit of thousands of Tweets per conversation tree in this cache. even just showing an ad is a ton work has a bunch of services involved https://blog.twitter.com/engineering/en_us/topics/infrastructure/2021/sharding-simplification-and-twitters-ads-serving-platform https://blog.twitter.com/engineering/en_us/topics/infrastruc... Other stuff that I think explains the complexity https://blog.twitter.com/engineering/en_us/topics/insights/2017/using-deep-learning-at-scale-in-twitters-timelines https://blog.twitter.com/engineering/en_us/topics/insights/2... https://blog.twitter.com/engineering/en_us/topics/insights/2020/using-machine-learning-to-predict-the-value-of-ad-requests https://blog.twitter.com/engineering/en_us/topics/insights/2... https://blog.twitter.com/engineering/en_us/topics/insights/2018/twittertensorflow https://blog.twitter.com/engineering/en_us/topics/insights/2... there's no single algorithm, its a bunch of parts coming together from hundreds of engineers and multiple services.

VirusNewbie 5y ago

You think there is no such thing as OSS models?

pbreit 5y ago

Please explain. I can easily envision much more transparency on how the feed works.

tayo42 5y ago
Transparency i can agree with, i think twitter does a horrible job at that. especially explaining the rumored shadow bans and stuff. But i think thats different then open source code