4 ms·
Twitter's peak load is only 200 tweets per second
- est 17y ago80KBps, 0.5 billion tweets per month.
- icey 17y agoI'm confused... Where is this figure actually coming from? (Other than Scoble saying it.)
- adbachman 17y agoAccording to him, it's the number given by "people who are getting the firehose feed." http://friendfeed.com/scobleizer/50e673d8/some-stats-from-twitter-conference-compared-to#c-6154ef056a9246b69ed763d361ba1c38 http://friendfeed.com/scobleizer/50e673d8/some-stats-from-tw...
- icey 17y agoThat makes the headline seem a bit linkbaity to me. It also makes me question the validity of the data.
- Maciek416 17y agoAnother way to extract this figure is to monitor the ids of tweets like the Twitpocalypse site is doing. We've been monitoring Twitpocalypse for a while, and the data yields the following chart: http://img.skitch.com/20090529-xhhnr83xsy67burr7w24kycp4u.png http://img.skitch.com/20090529-xhhnr83xsy67burr7w24kycp4u.pn... From the slope, we saw about 200 tweets/second, sometimes peaking at 250 as well. The right axis in this chart is the number of tweets per minute. You can see it rising and falling through the day.
- icey 17y agoHold on a minute. Are we talking about peak load or average load? Saying Twitter averages 200 tweets/second is vastly different than peaking at 200 tweets a second. Am I just being slow? Edit: Nevermind, Robert is saying that the average is 200 tweets/second during periods of peak load. The headline here makes it seem like the maximum capacity is 200 tweets/second which is misleading.
- Maciek416 17y agoI think what the data shows is that the average sort of wobbles around a bit. Check out the update to my post for a new image link to see it varying throughout the day.
- adbachman 17y agoBUT, that's via the firehose, which doesn't include direct messages or twitterers who've locked their feeds. The number also doesn't reflect what has to happen with each of those 200. Sure it's not impressive if all they have to do is append a 140 char message to a flat file or DB table. It's all the other manipulation that's interesting.
- Kaizyn 17y agoWhat kind of 'manipulation'? They're basically routing messages, with a protocol change in the middle when the recipients' designated protocol differs (i.e. IM to text message). This can't be all that computationally expensive when the system is engineered right. Knowing how little activity twitter has really makes them look incompetent in light of the service outages they experienced over the last year. Either Ruby on Rails is really really non-performant or the twitter code-monkies got the system architecture wrong.
- jerf 17y agoThe correct metric is probably not "tweets incoming", but "tweets delivered". There's some multiplicative factors that start to bite. I say "probably" because I'm an outsider guessing, but I'm reasonably confident about that statement. Just not quite 100%. (In the colloquial sense of 100%, of course; I'm not mathematically 100% sure Twitter exists.)
- adbachman 17y agoFor starters, there's a few interesting links at http://highscalability.com/scaling-twitter-making-twitter-10000-percent-faster http://highscalability.com/scaling-twitter-making-twitter-10... summary: "Rails and Ruby haven’t been stumbling blocks... The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January" They're dealing with duplication, consistency, searchability, etc. across distributed storage systems and a variety of service mechanisms (more than just protocols). The 1 000 000 follower user has every message duplicated and cached at multiple layers up to 1000000 times. The message can show up on the web, API, or across any of the protocols, and it has to persist. They're not incompetent code-monkies, they just guessed way low when they designed the architecture. It looks like the big move was from a CMS model (not at all unreasonable for a "microblogging" service) to a messaging model. In hindsight, targeting messaging to begin with would've saved them some down time, but wouldn't have been practical in the short term. From http://gojko.net/2009/03/16/qcon-london-2009-upgrading-twitter-without-service-disruptions/ http://gojko.net/2009/03/16/qcon-london-2009-upgrading-twitt... it sounds like the insertion number averages around 9600 messages per second. That's avg follower count * avg tweets inbound, and that's only the input. Edit: real, solid numbers are hard to come by. I suspect it's proprietary.
- TweedHeads 17y agoMySql can handle that with both hands tied in the back. No sweat.
- poutine 17y agoUh, some of those 200 tweets per second are being sent to over a million people. Most are sent to hundreds or thousands.
- pietro 17y agoI don't think that actor guy posts every second, and the average number of followers is way lower than thousands.
- adbachman 17y agoThere are 13 users with 1000000+ and the top 100 all have over 450000. (http://twitterholic.com/ http://twitterholic.com/) But, long tail and all that, I hear average follower count is in the 120 range, you're right there. (http://gojko.net/2009/03/16/qcon-london-2009-upgrading-twitter-without-service-disruptions/ http://gojko.net/2009/03/16/qcon-london-2009-upgrading-twitt...)
- decode 17y agoI've always assumed that most people don't get every tweet SMS'd to them. So most of the time, a tweet isn't sent to anybody, it's just returned when people request the tweets that are relevant to them.
- daleharvey 17y agotwitter does not "INSERT INTO tweets VALUES ('me', 'new tweet', DATE())" what mysql can handle doing a completely different thing is pretty irrelevant
- deleted 17y ago[deleted]
- lallysingh 17y ago
- abstractwater 17y agoI don't see the value in Scoble's argument, if he has one. I don't see the point of comparing the amount of tweets/sec with the rate of Google queries. They are just very different things, in their essence, purpose, way of usage. It would be more interesting (maybe) comparing the number of Twitter search queries with Google's, at least we would be talking about similar operations.