4 ms·

> You still have to profile and test carefully. I wish more software provided appropriate metrics for performance analysis in live systems. I've seen people bl

> You still have to profile and test carefully. I wish more software provided appropriate metrics for performance analysis in live systems. I've seen people blaming the CPU or the disk or the database or the network for issues, but when you actually look at the graphs, the CPU is mostly idle, IO is almost nonexistent and the database sees a few dozen selects per second at worst, but somehow the problem is not in the software... And it's really difficult to get people to believe otherwise when all you have is data from around the software itself, not from inside it.

flukus 10y ago
> or the network for issues I never see the network blamed, it's usually ignored. I've seen way to many instances of slow database queries being replaced by slow cache reads when the problem wasn't slow queries but 1000's of fast ones.
- SomeCallMeTim 10y ago
  This is actually my biggest complaint about the commonly cited benchmarks at TechEmpower. [1] Their "multiple queries" and "data updates" benchmarks seem to be more of a test of the database and the specific database bindings than anything like production differences in database access. If you look at the data, the database is clearly being pushed to its limits, and is the constraining factor for all of those tests. It's fine to test the bindings for speed, but they should be clear that's what they're doing. Instead they claim to be testing the speed of the frameworks ("Web Framework Benchmarks" is the topmost title on the page). I think the binding speed is rarely the constraining factor in production, and that if you had a properly scaled database backend you'd find the same languages on top of that comparison that you find on top of the "PlainText" and "Single Query" benchmarks. In a production environment you typically have a database that is running on different hardware, sometimes in shards, and that is scaled to handle your access patterns. In that environment all queries have higher latency than a database on localhost, and you can push a single application server (where the "framework" runs) a lot farther when you're using, say, Go or Node.js, which can more efficiently pipeline requests, than you can when you're using a thread-per-connection language like Ruby or PHP. Finally, the TechEmpower "Fortunes" benchmark is nearly 100% a benchmark of the default templating approach in each framework; Go has template libraries that are literally 10x faster than the one in the standard library and could be among the top performers on that page as well. I guess the moral of the story is that I should contribute to the discussions around TechEmpower benchmarking... :) [1] https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/

flukus 10y ago

> or the network for issues I never see the network blamed, it's usually ignored. I've seen way to many instances of slow database queries being replaced by slow cache reads when the problem wasn't slow queries but 1000's of fast ones.

SomeCallMeTim 10y ago
This is actually my biggest complaint about the commonly cited benchmarks at TechEmpower. [1] Their "multiple queries" and "data updates" benchmarks seem to be more of a test of the database and the specific database bindings than anything like production differences in database access. If you look at the data, the database is clearly being pushed to its limits, and is the constraining factor for all of those tests. It's fine to test the bindings for speed, but they should be clear that's what they're doing. Instead they claim to be testing the speed of the frameworks ("Web Framework Benchmarks" is the topmost title on the page). I think the binding speed is rarely the constraining factor in production, and that if you had a properly scaled database backend you'd find the same languages on top of that comparison that you find on top of the "PlainText" and "Single Query" benchmarks. In a production environment you typically have a database that is running on different hardware, sometimes in shards, and that is scaled to handle your access patterns. In that environment all queries have higher latency than a database on localhost, and you can push a single application server (where the "framework" runs) a lot farther when you're using, say, Go or Node.js, which can more efficiently pipeline requests, than you can when you're using a thread-per-connection language like Ruby or PHP. Finally, the TechEmpower "Fortunes" benchmark is nearly 100% a benchmark of the default templating approach in each framework; Go has template libraries that are literally 10x faster than the one in the standard library and could be among the top performers on that page as well. I guess the moral of the story is that I should contribute to the discussions around TechEmpower benchmarking... :) [1] https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/