4 ms·

What I'd love to see paired with this data is a cost comparison. At what scale does performance of ruby/python/php become cost prohibitive? Twitter made the mov

by saym 14y ago

What I'd love to see paired with this data is a cost comparison. At what scale does performance of ruby/python/php become cost prohibitive? Twitter made the move from Ruby to Java some years back, did they ever post a comparison of their numbers before and after? Also, the difference between EC2 and local i7 hardware is glaringly obvious. At what scale does owning the server hardware become imperative? I know these questions are beyond the scope of a performance review, but inquiring minds would like to know.

kainsavage 14y ago
Actually, we had an early revision of the benchmark tests that had exactly that analysis, but we really felt it is a very interesting subject that deserves more focus.
bhauer 14y ago
As you might imagine, during this exercise, we've had a lot of conversations about the points you raise. We have our own opinions, but we ultimately removed most of that content from the blog post because we didn't want it to be too editorial. We will be posting follow ups with some of our opinions. Some things are really difficult to answer in a vacuum. If you already have a competent devops staff, hosting your own hardware is probably beneficial. The increased performance per "server" is substantial. But no devops staff? Then it's either very risky or cost-prohibitive to own hardware.
- ochoseis 14y ago
  I too was shocked at the deltas between Amazon and dedicated hardware. I think AWS runs on Xen, so I wonder if you could strike a more optimal performance/flexibility balance using lighter virtualization (LXC or OpenVZ) with either in-house hardware or another VPS provider.
  - spartango 14y ago
    While it is true that EC2 runs on a customized version of Xen, it's very unlikely that latency is being introduced by the hypervisor itself. With paravirtualized kernel and drivers, Xen introduces negligible overhead, and thus LXC would likely be no better. The reason that the virtualized setup performs more poorly than the dedicated setup is that you are fighting for CPU time with other AWS customers, so those other customers are introducing latency into your application. Any shared/virtualized host will have this problem. Interestingly, where Netflix really wants to squeeze CPU performance out of EC2 instances, they allocate the largest instance type so that they know that there's nobody else on the underlying machine.
    - e12e 14y ago
      Regarding performance of Amazon, this post from 2009 is (still) very interesting: http://uggedal.com/journal/vps-performance-comparison/ http://uggedal.com/journal/vps-performance-comparison/ Amazon performance is surprisingly low, but also surprisingly consistent. For some use cases, consistency (knowing what you get you for what you pay) might be worth a considerable hit in performance.
Joeri 14y ago
Since amazon always takes a cut it stands to reason that if you can fully load a server on a continuous basis, owning s cheaper than renting, regardless of how efficient EC2 is. You should only rent the peak load, not the base load.

by saym 14y ago

kainsavage 14y ago

Actually, we had an early revision of the benchmark tests that had exactly that analysis, but we really felt it is a very interesting subject that deserves more focus.

bhauer 14y ago

As you might imagine, during this exercise, we've had a lot of conversations about the points you raise. We have our own opinions, but we ultimately removed most of that content from the blog post because we didn't want it to be too editorial. We will be posting follow ups with some of our opinions. Some things are really difficult to answer in a vacuum. If you already have a competent devops staff, hosting your own hardware is probably beneficial. The increased performance per "server" is substantial. But no devops staff? Then it's either very risky or cost-prohibitive to own hardware.

ochoseis 14y ago
I too was shocked at the deltas between Amazon and dedicated hardware. I think AWS runs on Xen, so I wonder if you could strike a more optimal performance/flexibility balance using lighter virtualization (LXC or OpenVZ) with either in-house hardware or another VPS provider.
- spartango 14y ago
  While it is true that EC2 runs on a customized version of Xen, it's very unlikely that latency is being introduced by the hypervisor itself. With paravirtualized kernel and drivers, Xen introduces negligible overhead, and thus LXC would likely be no better. The reason that the virtualized setup performs more poorly than the dedicated setup is that you are fighting for CPU time with other AWS customers, so those other customers are introducing latency into your application. Any shared/virtualized host will have this problem. Interestingly, where Netflix really wants to squeeze CPU performance out of EC2 instances, they allocate the largest instance type so that they know that there's nobody else on the underlying machine.
  - e12e 14y ago
    Regarding performance of Amazon, this post from 2009 is (still) very interesting: http://uggedal.com/journal/vps-performance-comparison/ http://uggedal.com/journal/vps-performance-comparison/ Amazon performance is surprisingly low, but also surprisingly consistent. For some use cases, consistency (knowing what you get you for what you pay) might be worth a considerable hit in performance.

Joeri 14y ago

Since amazon always takes a cut it stands to reason that if you can fully load a server on a continuous basis, owning s cheaper than renting, regardless of how efficient EC2 is. You should only rent the peak load, not the base load.