3 ms·
This is actually my biggest complaint about the commonly cited benchmarks at TechEmpower. [1] Their "multiple queries" and "data updates" benchmarks seem to be
by SomeCallMeTim 10y ago
This is actually my biggest complaint about the commonly cited benchmarks at TechEmpower. [1]
Their "multiple queries" and "data updates" benchmarks seem to be more of a test of the database and the specific database bindings than anything like production differences in database access. If you look at the data, the database is clearly being pushed to its limits, and is the constraining factor for all of those tests.
It's fine to test the bindings for speed, but they should be clear that's what they're doing. Instead they claim to be testing the speed of the frameworks ("Web Framework Benchmarks" is the topmost title on the page). I think the binding speed is rarely the constraining factor in production, and that if you had a properly scaled database backend you'd find the same languages on top of that comparison that you find on top of the "PlainText" and "Single Query" benchmarks.
In a production environment you typically have a database that is running on different hardware, sometimes in shards, and that is scaled to handle your access patterns. In that environment all queries have higher latency than a database on localhost, and you can push a single application server (where the "framework" runs) a lot farther when you're using, say, Go or Node.js, which can more efficiently pipeline requests, than you can when you're using a thread-per-connection language like Ruby or PHP.
Finally, the TechEmpower "Fortunes" benchmark is nearly 100% a benchmark of the default templating approach in each framework; Go has template libraries that are literally 10x faster than the one in the standard library and could be among the top performers on that page as well.
I guess the moral of the story is that I should contribute to the discussions around TechEmpower benchmarking... :)
[1] https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/