6 ms·

I'm working on a web service (build on top of Scala and the JVM) that's handling between 1500 and 3000 reqs per second per c1.medium instance, with an average t

I'm working on a web service (build on top of Scala and the JVM) that's handling between 1500 and 3000 reqs per second per c1.medium instance, with an average time per request of under 15ms. This is real traffic, with the web service receiving between 16,000 and 30,000 total requests per second during the day. A c1.xlarge can do 7000 reqs per second or even more, but for the moment I felt like the difference in pricing is too big and it's cheaper and safer just starting more c1.medium instances (with auto-scaling based on latency), but in case we'll need more RAM, then we'll probably switch to c1.xlarge. If scalability matters, you should have picked a better platform. Ruby/Rails/Passenger is a terrible platform for scalability / performance. And even if AWS is slower than other solutions, the first problem you have is your own heavy-weight app and the platform you've chosen. 15 concurrent requests per second makes me chuckle.

nirvdrum 13y ago
I don't entirely disagree. All I can say is REE and Rails 2.3 were far lighter weight and faster than Ruby 1.9 and Rails 3.2. Given it's a 3.5 year old app, the landscape was pretty different back then. I looked at Lift and didn't like it. Django was still in a weird place. And ultimately Rails looked like the best option for a variety of reasons. Things evolve and whole hog rewrites are difficult. Nowadays we run in JRuby and things are quite a bit better. But we can't run on anything smaller than an m1.large. The low I/O and meager RAM in a c1.medium preclude its use. (BTW, that's where a lot of the original 15 came from -- with a process using 100 MB RAM and only 1.7 GB available, it's hard to squeeze much more out of that). But the larger point is with virtually any other provider you can pick a configuration that matches the needs of your app (rather than the other way around), don't have to fight with CPU steal, don't have to fight with over-subscribed hardware, and don't have to deal with machine configurations from 2006. Yeah, Rails is never going to outperform your Scala web service. But if the app would run just fine on the other N - 1 providers, then it's disingenuous to gloss over the execution environment as well.
- bad_user 13y ago
  Yes, JRuby should be much better. Run it on top of JDK 7 and use the CMS garbage collector, as JRuby (and Scala) tend to generate a lot of short-term garbage and experiment with the new generation proportion (something like -XX:+UseConcMarkSweepGC -XX:NewRatio=1 -XX:MaxGCPauseMillis=850). You can also profile memory usage (make sure you're not stressing the GC, as that can steal away CPU resources) and for that I believe you can use Java profilers (like YourKit which is pretty good). Also, try to do more stuff async, like in another thread, process or server. Use caching where it's easy, but don't over do it, as dealing with complex cache invalidation policies is a PITA.
nirvdrum 13y ago
I just wanted to add -- since you're not the first to point out the Rails part -- that I've also run a 42 node Cassandra cluster on a m1.xlarges and did a fair bit of CPU-bound operations (encryption and compression) on hundreds of TB of data on cc2.8xlarge. I just used the Rails one as an example. In the case of Cassandra, disk I/O was a constant issue. So, we grew the cluster much larger than would be necessary on another provider. We also lost instances pretty regularly. If we were lucky, Amazon would notify us about degraded hardware, but usually the instance would stay up but do things like drop 20% of its packets. Replacing a node in Cassandra is easy enough, but you quickly learn how much their I/O levels impact network performance as well. Nowadays Cassandra has the ability to compress data to reduce network load, but you then run into EC2's fairly low CPU performance. The CPU-bound application I mentioned wasn't so bad, but we paid heftily for that ($2.40 / hour - some volume discount). At the high end the hardware tends not to be over-subscribed. Performance, price, and reliability were all issues in all cases. Those are not EC2's strong suits and haven't been for a while.
- bad_user 13y ago
  Yeah, I feel you dude. I also experienced I/O issues. Unfortunately we also need auto-scaling capabilities and map-reducers and stuff. Maybe Google's new "Compute Engine" service will prove to be better.

nirvdrum 13y ago

I don't entirely disagree. All I can say is REE and Rails 2.3 were far lighter weight and faster than Ruby 1.9 and Rails 3.2. Given it's a 3.5 year old app, the landscape was pretty different back then. I looked at Lift and didn't like it. Django was still in a weird place. And ultimately Rails looked like the best option for a variety of reasons. Things evolve and whole hog rewrites are difficult. Nowadays we run in JRuby and things are quite a bit better. But we can't run on anything smaller than an m1.large. The low I/O and meager RAM in a c1.medium preclude its use. (BTW, that's where a lot of the original 15 came from -- with a process using 100 MB RAM and only 1.7 GB available, it's hard to squeeze much more out of that). But the larger point is with virtually any other provider you can pick a configuration that matches the needs of your app (rather than the other way around), don't have to fight with CPU steal, don't have to fight with over-subscribed hardware, and don't have to deal with machine configurations from 2006. Yeah, Rails is never going to outperform your Scala web service. But if the app would run just fine on the other N - 1 providers, then it's disingenuous to gloss over the execution environment as well.

bad_user 13y ago
Yes, JRuby should be much better. Run it on top of JDK 7 and use the CMS garbage collector, as JRuby (and Scala) tend to generate a lot of short-term garbage and experiment with the new generation proportion (something like -XX:+UseConcMarkSweepGC -XX:NewRatio=1 -XX:MaxGCPauseMillis=850). You can also profile memory usage (make sure you're not stressing the GC, as that can steal away CPU resources) and for that I believe you can use Java profilers (like YourKit which is pretty good). Also, try to do more stuff async, like in another thread, process or server. Use caching where it's easy, but don't over do it, as dealing with complex cache invalidation policies is a PITA.

nirvdrum 13y ago

I just wanted to add -- since you're not the first to point out the Rails part -- that I've also run a 42 node Cassandra cluster on a m1.xlarges and did a fair bit of CPU-bound operations (encryption and compression) on hundreds of TB of data on cc2.8xlarge. I just used the Rails one as an example. In the case of Cassandra, disk I/O was a constant issue. So, we grew the cluster much larger than would be necessary on another provider. We also lost instances pretty regularly. If we were lucky, Amazon would notify us about degraded hardware, but usually the instance would stay up but do things like drop 20% of its packets. Replacing a node in Cassandra is easy enough, but you quickly learn how much their I/O levels impact network performance as well. Nowadays Cassandra has the ability to compress data to reduce network load, but you then run into EC2's fairly low CPU performance. The CPU-bound application I mentioned wasn't so bad, but we paid heftily for that ($2.40 / hour - some volume discount). At the high end the hardware tends not to be over-subscribed. Performance, price, and reliability were all issues in all cases. Those are not EC2's strong suits and haven't been for a while.

bad_user 13y ago
Yeah, I feel you dude. I also experienced I/O issues. Unfortunately we also need auto-scaling capabilities and map-reducers and stuff. Maybe Google's new "Compute Engine" service will prove to be better.