4 ms·
Once you increase the scale sufficiently, it's more cost effective to hire a team that makes the code run faster rather than add large expensive servers. This i
by theoj 16y ago
Once you increase the scale sufficiently, it's more cost effective to hire a team that makes the code run faster rather than add large expensive servers. This is what's happening now at Twitter.
It's not that Java is now absolutely 100% better than Ruby on Rails, it's just that Java is better for a site of that scale.
- acangiano 16y agoYep. Ruby on Rails: suitable for apps with less than 1 billion queries per day.
- theoj 16y agoIt's more than just about queries. Ruby on Rails is an absolute pig in terms of memory used. That's something you find out pretty quickly if you build your own VPS setup.
- aphexairlines 16y agoGoing to the JVM because of memory usage issues is like going to KFC to lose weight.
- bballant 16y agoOn my team such remarks cost a dollar. The JVM generally does very well in terms of memory use compared to many languages. It's easy to test: write an a small program in both languages that creates boatloads of objects and then watch the memory. The JVM will use up all the memory it can (the amt is configurable with some runtime flags) before the GC kicks in and frees up space. It uses a couple of buckets for different types of objects, and can easily clean small short-lived objects with a bit of a CPU hit, but without affecting the performance of the application. The upshot is you'll see Java's mem use slowly creep up, you'll see a small spike in CPU, then the mem use will drop. If you don't see this, it means the code is poorly written and its keeping around object references. Ruby's GC is much less efficient. It has no concept of memory buckets like the JVM and will effectively stop execution and traverse every object in the heap twice, marking and then freeing up space. Early versions of Java (pre 5) had some substantial problems with memory management, but it's been very performant for quite some time and easily out-paces ruby.
- aphexairlines 16y agoThere are no benchmarks showing JVM-based programs using less memory than most any other roughly equivalent programs on different runtimes. The JVM's reputation as a pig also comes from people's personal experiences -- even a simple email proxy (davmail) is easily at 100mb resident, over 1gb virtual.
- bballant 16y agoMy main point is that the JVM manages memory better than ruby. Its dead easy to benchmark it yourself by writing a pair of tiny programs. As I said, the JVM will use up all the memory it can. You can probably tweak davmail's runtime flags to use less. There is typically a CPU tradeoff if you cut back on the memory a program can use, and usually a happy middle-ground. In my personal experience, I've re-written a few ruby utils in Java for an order of magnitude speed improvement. I've seen a ruby site crater under load because of memory issues that its java replacement easily manages (which is sorta what the OP is about). The JVM is especially well suited for handling "stateless" http requests because it's ability to quickly create and destroy small objects.
- wwkeyboard 16y agoIt's not just the quantity of queries, it's what the application does. Twitter needs to do specific things very quickly(in this case answer search queries). You could look at a more general high-traffic site(like Facebook) to find a different approach(highly optimized php compiler). I don't think looking to such a large team on such a large project for a role model is beneficial to small teams, they have different considerations and abilities.