Just being extremely pedantic here, but for the least statistician of us I believe is important. Usually just have lower numbers in your benchmark doesn't actu

Just being extremely pedantic here, but for the least statistician of us I believe is important. Usually just have lower numbers in your benchmark doesn't actually means that your software runs faster, you should run a statistical test and see. In this case, if we run a standard test the confidence for improved performance is basically 1, and it is definitely accepted. However the claim of performance improved of 8%, well the confidence for that claim is of around ~83%, which is quite big, but lower than 95% usually expected in peer-reviewed journals. Said so, I really enjoy the post :)