3 ms·
> compare a bike and a car. Is unfair? yes. But, if the GOAL is how fast is go to the next city, is totally worth to know that the car is better. The problem
by rumanator 6y ago
> compare a bike and a car. Is unfair? yes. But, if the GOAL is how fast is go to the next city, is totally worth to know that the car is better.
The problem with these benchmarks is they have a propensity to not make a fair comparison. For example, sometimes they try to compare underperforming naive implementations in one language with expertly tuned implementations in another. Sometimes a blatantly underperforming implementation is used. Sometimes results are turned upside down by flipping compiler flags.
And sometimes photofinish results that fall within the standard deviation are sold as this absolute proof that language A outperforms language B.
Let's look at these benchmarks to see how these things are spun to sound like objective assertions are derived from meaningless data. Does it really matter if someone could use language A to shave off about 1MB from build artifacts? Are we really expected to take a 1.5s/5% difference in a cherry picked synthetic benchmark as meaningful? Is anyone in their right mind expected to drive their decisions on their tech stack on two cherry-picked tricks posted on a blog?
This is why no one takes these stunts seriously.