3 ms·
We use the word "framework" as a term of convenience covering the full spectrum from platforms, micro-frameworks, to full-stack frameworks. We are also liberal
by bhauer 8y ago
We use the word "framework" as a term of convenience covering the full spectrum from platforms, micro-frameworks, to full-stack frameworks. We are also liberal with accepting contributions from the community, which means we do indeed include several frameworks that are well outside the mainstream.
That said, in tests such as Fortunes, you will see many full-stack frameworks demonstrating their capability to deliver myriad web application fundamentals (such as request routing, database connection pooling, ORM, XSS countermeasures, character encoding, data structures, and server-side templates) at very high performance levels.
This project establishes a high-water mark of performance. Very few web applications process anything remotely close to the requests per second we're measuring. But using Fortunes as a proxy and applying a coefficient (e.g., 0.005) to adjust for real-world application sizing can give you a very rough but nevertheless potentially useful approximation of real-world expectations. For example, one can guess their application is 200 times more complex than our Fortunes test. Rough back-of-the-envelope math will then estimate a framework from the ultra-high performance tier (~200,000 fortunes/sec) might yield a generous ~1,000 real-world app requests per second on a modern Xeon server. Meanwhile, a framework in the middle tier (~10,000 fortunes/sec) would yield a more constraining ~50 real-world app requests per second. Again, this math is hand-waving and you can interpret the results in whatever way you prefer; you can dismiss the validity of approximating things so coarsely. But my experience is that real-world applications based on frameworks I've used—which are sprinkled among all tiers of our results—do align with this hand-waving approximation.
Edit: I invite you to read the last section of the blog entry about this round [2] where I argue the same point in another way and share one of the tweets from an application developer whose real world application benefited from the performance improvements made to his favorite framework.
[1] https://www.techempower.com/benchmarks/#section=motivation https://www.techempower.com/benchmarks/#section=motivation
[2] https://www.techempower.com/blog/2018/06/06/framework-benchmarks-round-16/ https://www.techempower.com/blog/2018/06/06/framework-benchm...
- blunte 8y agoThanks for the explanation. I see your reasoning. If there were a (asterisk) in the title: Web Framework Benchmarks (asterisk link), with the asterisk taking you to a page with something like your explanation above, that would be lovely.