47 ms·
Why Twitter's Engineers Hate the @replies feature
- nop 18y ago"What have I missed here?" They don't have to check 1.7 million users to see if they have @replies on, if they maintain a list with "@reply subscribers", there's little overhead as far as I can see. If a something rarely changes but is checked a lot, it's a pretty good idea to have it pre-calculated. Edited: "No overhead", was a bit to absolute thanks jpwagner. Re-Edited: Oh look there was a commenting feature on the blog, was pretty hard to find.
- jpwagner 18y agoagreed. actually that's not NO overhead, but it's trivial and the right solution. instead of just one flag link on an HN post, there should be flag as inappropriate vs. flag as stupid *edit: make that "flag as waste of time"
- Carnage4Life 18y agoThat solution doesn't address my question. Caching @reply subscribers only addresses the 3% of users who have opted-in to receiving all @replies. For the remaining 97% whether they receive @replies from someone they are following or not is a function of which user the reply is directed to and whether that user is also their friend. You can't cache that, at best you can optimize how you calculate who should receive replies. In fact, what you've pointed out is that building an implementation to address the 3% case is straightforward which was the point of my post. PS: My blog does have a commenting feature. In fact there are 3 comments in response to the post.
- joshwa 18y agothose comments are very well hidden! Why not just display them by default they way the other 99% of blogs do? Or if you're going to hide them, at least put the view comments link at the bottom of the post where I'd expect the comments to be.
- jpwagner 18y agoPerson X member of 97% Person Y member of 97% Person Z member of 3% Person W member of 3% ________________________ Person_A_@_List: X, Y, Z Person_B_@_List: X, W ________________________ Person_A writes: @Person_B you be cool! Received by X and Z...not Y or W
- Carnage4Life 18y agoI've updated my post based on your comment above. Thanks, it was rather helpful.
- ck113 18y agoIf you're interested in wild speculation (I have no idea how Twitter's database works), here's how I interpreted Biz's explanation: With the new system, say user Foo writes "@Bar lol me too!". Then Twitter can take Foo's follower list, join it with Bar's follower list, and send the message to everyone in the resulting list. Relational databases are very good at joins. On the other hand, with the old system, they'd have to do a deep inspection of the record for each of Foo's followers to know if they should send the message to that follower. Relational databases are much less good at this. But, as you and others have pointed out, if the number of users that use the "all @-replies" feature is really so small, it would be fairly inexpensive to cache the list of all of Foo's followers who use that feature, and join them in as well. I don't know why they don't do that -- maybe it adds up (like, if only 3% of users use the feature, but those users follow a lot of other users, they'll each end up in a lot of other users' caches).
- dws 18y agoThe 3% (of users affected) number has been trotted out a few times. Has twitter ever backed that up? It feels like a way of marginalizing the people who're complaining about the change, by painting them (us) as a vocal minority. (Following up: A trusted source confirms 3%)
- ubernostrum 18y agoTaking into consideration the fact that the percentage of people likely to put in the time to explore documentation and settings and discover the original configurability is probably quite small, and a reasonable assumption that not everyone who discovers the functionality will use it, yeah, the number sounds all right. Note, btw, that the number they seem to be going for is the above -- e.g., "according to the accounts database, only 3% of users ever changed this setting", _not_ "only 3% of users are claiming to care about this now that it's been removed".
- gaius 18y agoThe thing you're missing is that everyone at Twitter slept through CS 101.
- dasil003 18y agoGive me a break, scaling Twitter is not easy. CS principles do not magically allow you to construct an optimal solution for real world engineering problems. At Twitter's volume, the tiniest bit of extra latency in some middleware or network hardware can be the bottleneck. It's so easy to sit here and play armchair engineer and diagnose Twitter's (mostly past, btw) problems with a nauseating mix of hubris and ignorance.
- serhei 18y ago> ... with a nauseating mix of hubris and ignorance. That, and the pointing out of quite natural questions which Twitter has failed to address in its explanation of how it is acting. I mean, once you know what you are doing, being able to satisfy people asking ignorant and hubristic questions is a pretty sweet bonus.
- gaius 18y agonauseating mix of hubris and ignorance I'm afraid it's a nauseating mix of having operated real pub-sub systems such as TIBCO Rendezvous on major stock exchanges. What Twitter does is not difficult and they do it badly.
- ubernostrum 18y agoSo now you bait and switch? You started out with a condescending post which implied that anyone who'd stayed awake in CS 101 would find Twitter trivially easy to scale. Then you switched course to claiming that you know how to do this because of experience working a gigantic installation. Or are you asserting that the average CS 101 project consists of running the production messaging systems for major stock exchanges?
- gaius 18y ago
- cedsav 18y agoWhat if they have a fast way to get the intersect between followers of 2 different users? In this case, if a message is a @reply they could ignore the follower lists of each users and use the intersect list only (which can be cached). Supporting the 'get all replies' option means that they also need to consider the full follower lists and then deduplicate. .. just guessing here
- Tichy 18y agoIt really does not make any sense to me, given that @replies were not even a feature of twitter originally. Isn't the folk story that they only made @replies into a link when all people started marking replies in such a way? The way it is now (from my understanding) is that for every reply they have to check if somebody is following the person that is being replied to. How can this be more efficient than simply sending the reply-tweets to all followers (which they have to do anyway with all the other tweets that are not replies)?
- zimbabwe 18y agoIsn't the folk story that they only made @replies into a link when all people started marking replies in such a way? Considering they've just recently started turning #hashtags into links, I would not at all be surprised if @replies were birthed in a similar manner.
- gojomo 18y agoFirst, I sympathize with the Twitter engineers who have had to build a scaling system improvisationally under incredible growth and time constraints. I doubt they have time to reevaluate early limiting decisions except when things are on fire, and even then they probably only have enough time to go for the quick fix. Ultimately, though, I'm sure they'll figure out a way for this to work -- especially as it only requires "3%" of users to get @unacquiantanted-replies the same way they get broadcast messages. One approach: split accounts into two inner accounts, for example, aplusk and aplusk(all). Only the original name is ever publicly shown -- though the account's public page shows the all tweets, as now. However, which one people actually 'follow' depends on their 'show all @unacquianted replies' setting. Thus every aplusk(all) tweet is handled as a broadcast, because all those followers -- only 3% of the total -- prefer it that way.
- varenc 18y ago"In that case I’d expect Twitter to argue that the feature they want to remove for engineering reasons is filtering out some of the tweets you see based on whether you are a follower of the person the message is directed to not the other way around. " That would be easier for the twitter engineers, but thats not what 97% of twitter users want. The author doesn't acknowledge this.
- jpwatts 18y agoLies, damned lies, and statistics. That 97% only indicates that most people don't change the defaults; it says nothing about what Twitter users want. In this case it's particularly misleading, since we're talking about an option to turn off silent data loss. What would make someone track down a setting related to a problem they don't know they have? A more honest representation would be: "A small number of our users were aware that we filter @replies by default, some were happy about it and a few (3% of total users) changed their settings to remove the filtering."
- SMrF 18y agoIs anyone else getting redirected from the article to rotten.com?
- aardvark 18y agoAll I'm getting is "service unavailable"