4 ms·
How about testing sites themselves with those browsers and benchmarking that?
by thejohnconway 5y ago
How about testing sites themselves with those browsers and benchmarking that?
- jsnell 5y agoOk. So the first problem is that people would end up optimizing for entirely the wrong thing. Rather than optimize for the machines and networks that are actually in use, you'd optimize for high CPU Linux servers with no GPU connected over high bandwidth and low latency networking. Not low CPU mobile phones over LTE or dodgy WiFi. Second, you wouldn't complain that it's an unfair advantage to pages in the Google AMP cache, due to a better network location? An unfair advantage to pages hosted on GCP? An unfair disadvantage to iOS due to there being no iOS server hardware that could be used for testing?
- noobquestion81 5y agoAgreed that if you did a shitty job benchmarking you could get shitty results.
- thejohnconway 5y agoThen don’t go with the stupidest most naive way of benchmarking, and actually try to replicate some real-world scenarios? This would be utterly trivial for Google.
- jsnell 5y agoSynthetic benchmarking with actual end-user network conditions, or actual end-user browser+hardware combinations is in my opinion functionally impossible. (And you've excluded real world telemetry as an option). Just how do would you fairly benchmark Safari on iPhone performance on billions of web pages?
- eli 5y agoIf it's truly not possible to measure these performance metrics in a way that takes non-Google browsers into account then they are bad metrics and I'd prefer something else.