The area that I would be the most critical of the benchmarks is that it's unclear whether frameworks that use process based concurrency are being set up well.

The area that I would be the most critical of the benchmarks is that it's unclear whether frameworks that use process based concurrency are being set up well. With frameworks that use process based concurrency, generally one wants to run as many processes as can fit in RAM with a nice amount of headroom. That way, when one app process is waiting on I/O, another will be handling another request (and the operating system handles the context switching). Right now, the setup.py for Django specifies that it will create 3 * NCPU worker processes. On the i7, that would probably be 24 processes and possibly using an eighth of the RAM in the box. Similarly, for Rails, the Unicorn configuration specifies 8 worker processes (one process per GB of RAM) regardless of whether it's on EC2 or the i7 (while Django goes from 6 to 24 processes depending on the box it's on since it's 3 * NCPU). For all we know, the CPU usage might be around 25% and the RAM usage even less. As such, it's hard to determine if the speed achieved is because of the limitations in the framework or whether the box is simply being under-utilized due to sub-optimal configuration. If you're going to do another around, more than any other changes, I think a view of the CPU and RAM usage would really help - if for no other reason than to show that the test is accurate. Right now, I have questions about whether the process concurrency frameworks are well set-up. Why would Django benefit from getting 4x the number of worker processes on the i7, but not Rails? Is 24 processes utilizing the i7 box with Django? CPU and RAM numbers would help provide answers.