4 ms·
Have you worked on other sites with a similar scale to Twitter? If not, how would you know?
by scribu 14y ago
Have you worked on other sites with a similar scale to Twitter? If not, how would you know?
- jrockway 14y agoI work at Google :)
- shanelja 14y agoWell, there's pretty much no response he can give to that then. Unless of course, you work in the marketing department or clean the buildings?
- gary4gar 14y ago<!-- insert *evil grin* here -->
- scribu 14y agoFair enough :)
- boyter 14y agoBased on that I would like to hear how you would do so. Would make an interesting blog post or comment here.
- sbuk 14y agoAnd..?
- kooshball 14y agoThen you should really know better than to make statements like "I think I could write a scalable Twitter in mostly Ruby. And I've never written a line of Ruby in my life."
- jrockway 14y agoThis isn't really a comment on Java, Ruby, Twitter, Google, or my programming ability. It's a comment about the fact that once you've designed a scalable distributed system, the constant factor runtime isn't so important. Twitter has clearly engineered a working distributed system (or you would have read stories like "5% of Twitter users lost their tweets when our servers caught on fire last night"), so the fact that they wrote it in Scala or Java is largely irrelevant. They need fewer computers than they would have if they wrote it in Ruby again, and that's not something to discount, but I think it's technically possible to have a full Ruby stack do what Twitter does. And, they would have needed even fewer computers if they used C or C++. So it's clear that Java was a social choice (it had the right books, libraries, and stack of resumes) rather than an absolute must. Besides, Ruby and Java both make the same C library calls to do I/O. So it all ends up being the same for everything except search, trending tweets, and other cpu-intensive analysis.
- thebluesky 14y agoI think I could write a scalable Twitter using mostly low-paid workers who manually write down the tweets and deliver them in person. It's just a matter of hiring enough workers now that I've designed this scalable distributed system.
- jrockway 14y agoComputers are easier (though perhaps less fun) to make than humans. Ruby is slow but it's close enough to Java that the numbers will still work out. Humans: probably not.
- thebluesky 14y ago30x slower seems a bit of a stretch for "close enough": http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=yarv&lang2=java http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...
- rimantas 14y agoYou should look at performance difference in relevant operations. I doubt that they calculate mandelbrots at twitter.
- thebluesky 14y ago> You should look at performance difference in relevant operations If you have a link to a Twitter-realistic benchmark I'd be interested see the results.
- irahul 14y agoCompared to the OP, your post is just random, non-funny sarcasm. The OP has highlighted relevant data points. 1. Writing a system the second time is easier since you already know the pitfalls and avoid them. 2. Ruby, Python, Perl, Java... provide thin wrappers for the underlying IO system calls, and hence, Java IO isn't very different from Ruby IO when it comes to performance. 3. There are only a few CPU intensive tasks. It goes without saying that Java will kick Ruby's(pure Ruby; native extensions will be a different story)ass when it comes to CPU intensive tasks.
- ilamparithi 14y agoI suppose it doesn't matter if you didn't work on similar scaling problems in Google.
- jrockway 14y agoYou're right and I know that "appeal to authority" is not a valid argument mechanism. With that in mind, I just couldn't pass up the opportunity to make an amusing comment.