4 ms·
Unfortunately performance claims appear to be bogus. 1. ospan, performance claims seem to be based on, doesn't do any bound checks, so you can easily get buffe
by vitaut 6y ago
Unfortunately performance claims appear to be bogus.
1. ospan, performance claims seem to be based on, doesn't do any bound checks, so you can easily get buffer overflow.
2. fast_io generates a whopping 50kB of static data just to format an integer.
So if these benchmark results are correct (I was not able to verify because the author hasn't provided the benchmark source):
> format_int 7867424 ns 7866027 ns 89 items_per_second=127.129M/s
> fast_io_ospan_res 6871917 ns 6870708 ns 102 items_per_second=145.545M/s
fast_io gives 15% perf improvement by replacing a safe format_int API from https://github.com/fmtlib/fmt https://github.com/fmtlib/fmt with a similar but unsafe one + 50kB of extra data. Adding safety will likely bring perf down which the last line seems to confirm:
> fast_io_concat 7967591 ns 7966162 ns 88 items_per_second=125.531M/s
This shows that fast_io is slightly slower than the equivalent {fmt} code. Again this is from the fast_io's benchmark results that I hasn't been able to reproduce.
50kB may not seem like much but for comparison, after a recent binary size optimization, the whole {fmt} library is around 57kB when compiled with `-Os -flto`: http://www.zverovich.net/2020/05/21/reducing-library-size.html http://www.zverovich.net/2020/05/21/reducing-library-size.ht...
The floating-point benchmark results are even less meaningful. They appear to be based on a benchmark that I wrote to test the worst case Grisu (https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/p...) performance on unrealistic random data with maximum digit count. fast_io compares it to Ryu (https://dl.acm.org/doi/pdf/10.1145/3192366.3192369 https://dl.acm.org/doi/pdf/10.1145/3192366.3192369) where maximum digit count is actually the best case and the performance degrades as the number of digits goes down. A meaningful thing to do would be to use Milo Yip's benchmark instead: https://github.com/miloyip/dtoa-benchmark https://github.com/miloyip/dtoa-benchmark
- knoebber 6y agoThe author has made a rebuttal to this comment: https://github.com/expnkx/fast_io/commit/8cf7497593eb185bba13ad0154a6dfab5a534388 https://github.com/expnkx/fast_io/commit/8cf7497593eb185bba1...
- usefulcat 6y agoI wonder why not provide source for benchmarks?
- expnkx 6y agoSource is here: https://github.com/expnkx/fast_io/tree/master/benchmarks/0015.fast_io_vs_fmt/benchmark https://github.com/expnkx/fast_io/tree/master/benchmarks/001... Including video: https://www.youtube.com/watch?v=avyIFLuFUQs https://www.youtube.com/watch?v=avyIFLuFUQs
- vitaut 6y agoThe new benchmark is even less meaningful because 1. It now constructs unnecessary `std::string` penalizing `format_int`: value+=fmt::format_int(i).str().size(); 2. The input is consecutive numbers which makes branches well predicted which is not realistic but beneficial for fast_io integer formatter which has a lot of branches. The precomputed table you can find in include/fast_io_core_impl/integers/jiaendu/table_gen.h
- usefulcat 6y ago> 1. It now constructs unnecessary `std::string` penalizing `format_int`: > value+=fmt::format_int(i).str().size(); Author appears to be doing the equivalent in the other benchmark: value+=fast_io::to<std::string>(i).size(); Maybe you could argue that in both cases it would be better not to measure time spent creating and destroying strings, but I don't see how the two benchmarks are not comparable.
- vitaut 6y agoIf std::string construction was the goal then sure, but that's not what the original benchmark that I've written was about. The goal was to find the fastest way to format an integer with or without std::string construction: http://www.zverovich.net/2013/09/07/integer-to-string-conversion-in-cplusplus.html http://www.zverovich.net/2013/09/07/integer-to-string-conver.... Those methods that construct a string but not required to are explicitly marked. The OP made some claims based on the method that doesn't construct std::string (which is one of the reasons it's rather fast) and then when I pointed out that it's unsafe switched to a much slower one but at the same time unnecessarily penalized other methods. It seems like moving a goalpost just to prove that your method is the fastest (which might be true but we don't know because of a poor methodology).