3 ms·

I disagree with others that have said that these tests are worthless for a real application. It is worth knowing the baseline speed of a framework as this affec

by lhnz 15y ago

I disagree with others that have said that these tests are worthless for a real application. It is worth knowing the baseline speed of a framework as this affects the maximum number of requests you will be able to achieve, without scaling out your application.

andrewvc 15y ago
I strongly disagree. There's no such thing as baseline speed, there's only baseline speed on a given hardware config. Rails does not have a max speed of X reqs/second. It has a max speed of X reqs/second on a given hardware config. If you're profiling your own app you'll have to retest it on your own hardware first to figure out how close to that baseline you are.
- moe 15y ago
  There's no such thing as baseline speed Of course there is. The "hello-world" microbenchmark tells you something about the core stack performance of the frameworks relative to one another. This tends to be surprisingly indicative of the relative real-world throughput per node that you may expect for a more complex application. I.e. Rails (ruby) scores 4 times lower than Python in that micro-benchmark. This is pretty close to the difference that I've observed between real applications on the respective platforms. If anything you'll see the difference magnified in a complex application, but it's unlikely to turn around. Whether that matters or not in the big picture is a different question (it usually doesn't).
  - andor 15y ago
    If anything you'll see the difference magnified in a complex application, but it's unlikely to turn around. I wouldn't say so. The bottleneck in a hello world benchmark might be a component of the framework, but in complex applications it's likely to be something else. Just look at the "Template Test with DB Query": There, Sqlite is the bottleneck, and the performance difference between Django and Rails fades into background completely.
    - moe 15y ago
      in complex applications it's likely to be something else Nope. Ruby/Python app-servers always end up CPU-bound, unless you're doing something very much out of the ordinary (such as blocking on an embedded SQLite...).
singingfish 15y ago
With all due resepct, bollocks. The dispatcher is not tested at all here for example. To do a proper test add in some complicated url dispatch rules, session handling, various kinds of nasty hard-to-abstract-out business logic and we might be talking. The trouble is there you're then talking about a week's work to write each app. On the other hand the "benchmarks" are an interesting ultra-simple rosetta stone.
- moe 15y ago
  The comparison is reasonable as long as you test against an equivalent feature-set on each platform. It might actually be an interesting exercise to gradually layer more features on top (such as the dispatcher chain and different templating/ORM layers) to discover inefficiencies in the various frameworks by comparing the relative performance impact. E.g. if you involve the dispatcher and Django suddenly gets 20% slower but Rails only 10% slower then that might hint at an inefficient dispatcher-impl in Django.

by lhnz 15y ago

andrewvc 15y ago

I strongly disagree. There's no such thing as baseline speed, there's only baseline speed on a given hardware config. Rails does not have a max speed of X reqs/second. It has a max speed of X reqs/second on a given hardware config. If you're profiling your own app you'll have to retest it on your own hardware first to figure out how close to that baseline you are.

moe 15y ago
There's no such thing as baseline speed Of course there is. The "hello-world" microbenchmark tells you something about the core stack performance of the frameworks relative to one another. This tends to be surprisingly indicative of the relative real-world throughput per node that you may expect for a more complex application. I.e. Rails (ruby) scores 4 times lower than Python in that micro-benchmark. This is pretty close to the difference that I've observed between real applications on the respective platforms. If anything you'll see the difference magnified in a complex application, but it's unlikely to turn around. Whether that matters or not in the big picture is a different question (it usually doesn't).
- andor 15y ago
  If anything you'll see the difference magnified in a complex application, but it's unlikely to turn around. I wouldn't say so. The bottleneck in a hello world benchmark might be a component of the framework, but in complex applications it's likely to be something else. Just look at the "Template Test with DB Query": There, Sqlite is the bottleneck, and the performance difference between Django and Rails fades into background completely.
  - moe 15y ago
    in complex applications it's likely to be something else Nope. Ruby/Python app-servers always end up CPU-bound, unless you're doing something very much out of the ordinary (such as blocking on an embedded SQLite...).

singingfish 15y ago

With all due resepct, bollocks. The dispatcher is not tested at all here for example. To do a proper test add in some complicated url dispatch rules, session handling, various kinds of nasty hard-to-abstract-out business logic and we might be talking. The trouble is there you're then talking about a week's work to write each app. On the other hand the "benchmarks" are an interesting ultra-simple rosetta stone.

moe 15y ago
The comparison is reasonable as long as you test against an equivalent feature-set on each platform. It might actually be an interesting exercise to gradually layer more features on top (such as the dispatcher chain and different templating/ORM layers) to discover inefficiencies in the various frameworks by comparing the relative performance impact. E.g. if you involve the dispatcher and Django suddenly gets 20% slower but Rails only 10% slower then that might hint at an inefficient dispatcher-impl in Django.