4 ms·

Full disclosure: Judging by this man's twitter (@ckeck), he seems to be employed by Rackspace. That may or may not have something to do with the tests he ran an

Full disclosure: Judging by this man's twitter (@ckeck), he seems to be employed by Rackspace. That may or may not have something to do with the tests he ran and the settings he chose. (There are an awful lot of "Test failed" entries under competitor columns, hmm...)

rbranson 17y ago
I'm starting to become suspect as well. My own m1.large EC2 apachebench (-n 100000 -c 128) yields: Requests per second: 5162.92 [#/sec] (mean) Time per request: 24.792 [ms] (mean) Time per request: 0.194 [ms] (mean, across all concurrent requests) 100% 60 (longest request)
- justinsb 17y ago
  I think he's comparing an EC2 small (we can tell because he says the memory is between 1GB and 2GB); it looks like the headings on the table are wrong. I think he's comparing this to a Rackspace 1GB, which is actually cheaper than EC2's small instance; the more direct comparison is to the Rackspace 2GB. You m1 large is $0.34 per hour, vs $0.06 per hour on the Rackspace machine in the blog (I believe). You only got twice as many request per second out of it, so on the requests per second per dollar metric (the real metric, I think), ec2 is lagging.
  - ckeck 17y ago
    Yes, this was a typo on my part and has been corrected. In the "Medium Instances" section it was an Amazon EC2 Small instance that was used which is in line with the 1024MB Rackspace Cloud Server and the Linode 1080 server instance.
  - rbranson 17y ago
    With microbenches sucking in general, this test is especially CPU bound. I ran it on a c1.medium instance ($0.17/hr) with -c 160 -n 100000 and got 8602.92 [#/sec] (mean). That's more than 3x the performance at <3x the cost. EC2 gives you all that CPU continuously, not just in bursts.
ckeck 17y ago
I make no attempt to mask that I am a Rackspace employee and I even updated the post with a large disclaimer at the top just in case. The fact of the matter is that the small instances from Linode and Media Temple could not perform the tests as conducted with their out of box configurations. I made no enhancements to any platform. Some helpful readers have pointed out some settings I can change that will allow the benchmarks to complete successfully so I am going to do a follow-up post with all of these "tweaks" implemented across all the providers. I am being as objective as possible here, don't let the fact that I work for Rackspace fog the air. These tests are also very easy to duplicate on your own if you really want to test the data set and results. -Chad
- moe 17y ago
  don't let the fact that I work for Rackspace fog the air. If you don't want us to recognize this as the cheap PR-piece it is then perhaps next time refrain from conclusions like the following (verbatim quote): The tests show some significant performance benefits when running on the Rackspace Cloud Servers platform most likely due to (in my opinion) a more robust network infrastructure, lower hardware utilization, and better host server hardware. And from unverified claims like (quote): Even if different web server software is used, it is unlikely that the results and trends would differ. Otherwise, if that truly was supposed to be a honest benchmark then please hand in your engineer-badge at the door...

rbranson 17y ago

I'm starting to become suspect as well. My own m1.large EC2 apachebench (-n 100000 -c 128) yields: Requests per second: 5162.92 [#/sec] (mean) Time per request: 24.792 [ms] (mean) Time per request: 0.194 [ms] (mean, across all concurrent requests) 100% 60 (longest request)

justinsb 17y ago
I think he's comparing an EC2 small (we can tell because he says the memory is between 1GB and 2GB); it looks like the headings on the table are wrong. I think he's comparing this to a Rackspace 1GB, which is actually cheaper than EC2's small instance; the more direct comparison is to the Rackspace 2GB. You m1 large is $0.34 per hour, vs $0.06 per hour on the Rackspace machine in the blog (I believe). You only got twice as many request per second out of it, so on the requests per second per dollar metric (the real metric, I think), ec2 is lagging.
- ckeck 17y ago
  Yes, this was a typo on my part and has been corrected. In the "Medium Instances" section it was an Amazon EC2 Small instance that was used which is in line with the 1024MB Rackspace Cloud Server and the Linode 1080 server instance.
- rbranson 17y ago
  With microbenches sucking in general, this test is especially CPU bound. I ran it on a c1.medium instance ($0.17/hr) with -c 160 -n 100000 and got 8602.92 [#/sec] (mean). That's more than 3x the performance at <3x the cost. EC2 gives you all that CPU continuously, not just in bursts.

ckeck 17y ago

I make no attempt to mask that I am a Rackspace employee and I even updated the post with a large disclaimer at the top just in case. The fact of the matter is that the small instances from Linode and Media Temple could not perform the tests as conducted with their out of box configurations. I made no enhancements to any platform. Some helpful readers have pointed out some settings I can change that will allow the benchmarks to complete successfully so I am going to do a follow-up post with all of these "tweaks" implemented across all the providers. I am being as objective as possible here, don't let the fact that I work for Rackspace fog the air. These tests are also very easy to duplicate on your own if you really want to test the data set and results. -Chad

moe 17y ago
don't let the fact that I work for Rackspace fog the air. If you don't want us to recognize this as the cheap PR-piece it is then perhaps next time refrain from conclusions like the following (verbatim quote): The tests show some significant performance benefits when running on the Rackspace Cloud Servers platform most likely due to (in my opinion) a more robust network infrastructure, lower hardware utilization, and better host server hardware. And from unverified claims like (quote): Even if different web server software is used, it is unlikely that the results and trends would differ. Otherwise, if that truly was supposed to be a honest benchmark then please hand in your engineer-badge at the door...