5 ms·

Twitter Search is Now 3x Faster

17 points by denysonique 16y ago

tom_b 16y ago
Anyone from Twitter who can comment on the cultural/engineering ramifications of this? I would be tremendously interested to hear from former Ruby hackers who have been shifting (back?) to Java via the Netty tool mentioned in the OP. Definitely a different tack that we are used to hearing hear on HN . . .
Tawheed 16y ago
There are two ways to interpret this: OPTION A) Omg, I'm not going to use Rails, even Twitter is ditching it. OPTION B) Hmm.. They built something really fast using Rails, got it out there, got massive popularity, made boatloads of $$ and now they're maturing and moving to a more robust infrastructure. I go with Option B. If you're a new startup -- don't confuse this advice and start brewing your own Java-based server just because Twitter is doing this now.
- suking 16y ago
  Boatloads of $???
  - RobertKohr 16y ago
    AKA: Investors giving them money for worthless pieces of paper.
- jarin 16y ago
  I think that's the way to look at it. You have to remember that Twitter is at a completely different level of traffic than your company will likely ever hit, even if it becomes hugely and wildly popular. It's like looking at Google's server farm and saying "I can't be successful unless I too have a server farm like Google's."
- bonch 16y ago
  The implication of Option B is that Rails isn't robust. A lot of people are going to avoid Rails because of that.
  - michaelcampbell 16y ago
    That could very well be true in some abstract sense; but who among us honestly needs the robustness and/or scalability of Twitter, really?
- samstokes 16y ago
  This article is not about Twitter ditching Rails, because Twitter is not ditching Rails. This seems worth repeating [1] because this is the top-voted comment, even though the submission title no longer implies that. Absolutely go with Rails [2], because it's a great way to get started quickly and large successful startups like Twitter are still using it at scale. [1] http://news.ycombinator.com/item?id=2424393 http://news.ycombinator.com/item?id=2424393 [2] Or Django, or whatever
- jemond 16y ago
  To clarify, Twitter hasn't and doesn't make boatloads of money.
deleted 16y ago
[deleted]
bradly 16y ago
This is article is only talking about search.twitter.com, isn't it?
ChrisArchitect 16y ago
there's not alot in here about dropping Rails particularly.... it's more some Rails powered front end stuff for SEARCH that needed to be ditched to speed up the crazy insane search load they get.
rwolf 16y ago
Flagged. Ignore the gauche of sending me a link to your service instead of the story for a second--consider your audience! Why would a HN user who clicks on a link to a story want to read your summary first?
samstokes 16y ago
Submission title is misleading. The main Twitter website is still running Rails and the article gives no indication that that will change. The article is about the Twitter Search frontend moving away from an architecture where their application servers would synchronously query the search servers, which wasn't scaling with their load.
- denysonique 16y ago
  Actually you are right, I haven't read it carefully enough.
gry 16y ago
If your system looks like it did the first day you launched, you're doing it wrong. It's difficult to communicate to this... Am I the only one who reads it is only removing RoR from the search stack? It seems the rest of the front-end will probably exist as RoR in some form.
afsina 16y ago
Expect them to drop RoR completely from Twitter soon. As far as I know they only use it for some front-end stuff anyway.
PatrickTulskie 16y ago
Maybe I'm reading between the lines here, but the feel of the article was "We got this code base from Summize and it worked great when we got it but now it's full of rot so we scraped it." Twitter is big on the JVM and they have a bunch of Scala and Java engineers over there so this seems like the natural path. Besides, a lot of the big searching and map reduce frameworks out there are written in Java. This seems like a very natural transition. The annoying thing is you're going to find a bunch of people who skim the article and start banging the "Rails can't scale" drum louder than they were 3 years ago.

17 points by denysonique 16y ago

tom_b 16y ago

Anyone from Twitter who can comment on the cultural/engineering ramifications of this? I would be tremendously interested to hear from former Ruby hackers who have been shifting (back?) to Java via the Netty tool mentioned in the OP. Definitely a different tack that we are used to hearing hear on HN . . .

Tawheed 16y ago

There are two ways to interpret this: OPTION A) Omg, I'm not going to use Rails, even Twitter is ditching it. OPTION B) Hmm.. They built something really fast using Rails, got it out there, got massive popularity, made boatloads of $$ and now they're maturing and moving to a more robust infrastructure. I go with Option B. If you're a new startup -- don't confuse this advice and start brewing your own Java-based server just because Twitter is doing this now.

suking 16y ago
Boatloads of $???
- RobertKohr 16y ago
  AKA: Investors giving them money for worthless pieces of paper.
jarin 16y ago
I think that's the way to look at it. You have to remember that Twitter is at a completely different level of traffic than your company will likely ever hit, even if it becomes hugely and wildly popular. It's like looking at Google's server farm and saying "I can't be successful unless I too have a server farm like Google's."
bonch 16y ago
The implication of Option B is that Rails isn't robust. A lot of people are going to avoid Rails because of that.
- michaelcampbell 16y ago
  That could very well be true in some abstract sense; but who among us honestly needs the robustness and/or scalability of Twitter, really?
samstokes 16y ago
This article is not about Twitter ditching Rails, because Twitter is not ditching Rails. This seems worth repeating [1] because this is the top-voted comment, even though the submission title no longer implies that. Absolutely go with Rails [2], because it's a great way to get started quickly and large successful startups like Twitter are still using it at scale. [1] http://news.ycombinator.com/item?id=2424393 http://news.ycombinator.com/item?id=2424393 [2] Or Django, or whatever
jemond 16y ago
To clarify, Twitter hasn't and doesn't make boatloads of money.

deleted 16y ago

[deleted]

bradly 16y ago

This is article is only talking about search.twitter.com, isn't it?

ChrisArchitect 16y ago

there's not alot in here about dropping Rails particularly.... it's more some Rails powered front end stuff for SEARCH that needed to be ditched to speed up the crazy insane search load they get.

rwolf 16y ago

Flagged. Ignore the gauche of sending me a link to your service instead of the story for a second--consider your audience! Why would a HN user who clicks on a link to a story want to read your summary first?

samstokes 16y ago

Submission title is misleading. The main Twitter website is still running Rails and the article gives no indication that that will change. The article is about the Twitter Search frontend moving away from an architecture where their application servers would synchronously query the search servers, which wasn't scaling with their load.

denysonique 16y ago
Actually you are right, I haven't read it carefully enough.

gry 16y ago

If your system looks like it did the first day you launched, you're doing it wrong. It's difficult to communicate to this... Am I the only one who reads it is only removing RoR from the search stack? It seems the rest of the front-end will probably exist as RoR in some form.

afsina 16y ago

Expect them to drop RoR completely from Twitter soon. As far as I know they only use it for some front-end stuff anyway.

PatrickTulskie 16y ago

Maybe I'm reading between the lines here, but the feel of the article was "We got this code base from Summize and it worked great when we got it but now it's full of rot so we scraped it." Twitter is big on the JVM and they have a bunch of Scala and Java engineers over there so this seems like the natural path. Besides, a lot of the big searching and map reduce frameworks out there are written in Java. This seems like a very natural transition. The annoying thing is you're going to find a bunch of people who skim the article and start banging the "Rails can't scale" drum louder than they were 3 years ago.