2 ms·
It's not just "very weird setups." Totally normal setups differ greatly. One example of variation is that the users might be writing code using different langu
by skybrian 6y ago
It's not just "very weird setups." Totally normal setups differ greatly.
One example of variation is that the users might be writing code using different languages. The parser and static analysis for a project written in Go will be very different than a project written in Java. All the language-specific stuff is different code that has different performance characteristics.
As an analogy, it's sort of like testing browser performance for loading an "average" web page, when each web site has different performance, depending on how it was written. Load times for Hacker News aren't all that relevant for Gmail or the New York Times.
Yes, you can make a benchmark and optimize it, but you might not even be running the same code that real users are running, due to plugins.