5 ms·
This is a fun benchmark in C++, where you can see that GCC has a more restrictive small string optimization. On my desktop the main python example runs in 3.1s.
by stabbles 6y ago
This is a fun benchmark in C++, where you can see that GCC has a more restrictive small string optimization. On my desktop the main python example runs in 3.1s. Then this code
void example() {
for (int64_t i = 1; i <= 20; ++i) {
for (int64_t j = 1; j <= 1'000'000; ++j) {
std::to_string(j);
}
}
}
runs with GCC in 2.0s and with clang in 133ms, so 15x faster.
I've also benchmarked it in Julia:
function example()
for j = 1:20, i = 1:1_000_000
string(i)
end
end
which runs in 592ms. Julia has no small string optimization and does have proper unicode support by default.
None of the compilers can see that the loop can be optimized out.
- eggsnbacon1 6y agoThe Clang result is suspiciously fast. Way faster than GCC and my Java attempt. Maybe its dead code pruning the string conversion? Java was about the same as Julia for this, on my machine
- runevault 6y agoIf it's using string optimizations due to short strings, you're cutting allocations in almost half I believe, as well as putting said strings on the stack instead of the heap which will give... significant performance gains. Short strings in the C++ standard library can be crazy fast.
- plorkyeran 6y agoIt doesn't merely cut allocations in half; it cuts them to zero.
- runevault 6y agoDo you mean allocations as in heap? If so yeah, I guess I should have stated it better as it still needs to put space on the stack for the short string.
- josefx 6y agoThe stack space for the whole function call is allocated in one step. You only get individual allocations when you are messing around with variable length arrays or alloca.
- pjmlp 6y agoFor Java did you try a second run using CDS and PGO data?
- eggsnbacon1 6y agoI ran mine in JMH which does a bunch of warmup iterations, hopefully enough to trigger full JIT
- hpcjoe 6y agoI rewrote the benchmark in perl and julia as well. Kept it to its origins, and including startup interpreter time. Original python: #!/usr/bin/env python3 def main(): for j in range(20): for i in range(1000000): str(i) main() /usr/bin/time ./st.py 3.16user 0.02system 0:03.19elapsed 99%CPU Perl port #!/usr/bin/env perl use strict; sub main { my ($i,$j); foreach $j (0 .. 20) { foreach $i (1 .. 1000000) { sprintf "%i",$i; } } } /usr/bin/time ./st.pl 2.31user 0.00system 0:02.31elapsed 99%CPU And julia #!/usr/bin/env julia function main() for j in 0:20 for i in 1:1000000 string(i) end end end main() /usr/bin/time ./st.jl 1.43user 0.52system 0:01.30elapsed 149%CPU This isn't a very useful test of anything. The optimizations ceased being python after the use of the map with list. Then it delved into C. Which is precisely the point that Julia is attempting to solve by providing the power and performance, without the multi-language costs/overhead. Note: This is Julia 1.4.1, Perl 5.30.2, and Python 3.8.2 running on my 2 year old linux laptop with a kaby lake processor. [Edited to fix formatting]
- stabbles 6y agoNeat, the way I benchmarked was through BenchmarkTools.jl, which runs the functions sufficiently many times to get a statistically relevant result. It will not measure compilation time, but I did not include that for gcc / clang either.
- hpcjoe 6y agoI kept that in simply to keep the "playing field level", as I run Julia code as "scripts", knowing full well that it is far more powerful than that. I do use BenchmarkTools.jl as well, though I've found TimerOutputs.jl to be quite useful in my optimization/debugging.
- kbenson 6y agoFYI, the Perl equivalent is probably "$i" (which in my benchmarks results in ~41% of the runtime. It doesn't appear to be optimized away either, as an empty loop and one with a simple incrementing variable that's printed at the end result in 19% of the running time. In Perl the appropriate way to convert a number to a string is to interpolate it or use a string based operator ("." for concatenation, etc) on it and let it convert the type itself. sprintf is very heavy and does quite a bit more than simply convert a number to a string.
- moonchild 6y ago> This is a fun benchmark in C++, where you can see that GCC has a more restrictive small string optimization Wouldn't this be on the choice of stl (libstdc++ vs libc++) rather than compiler? Anyway, on my system, (llvm/clang 10.0, gcc 10.1), both take ~360ms. > None of the compilers can see that the loop can be optimized out I was interested in this, so decided to investigate. I'm actually not sure why that's happening. This is the definition of to_string on my system: inline string to_string(long __val) { const bool __neg = __val < 0; const unsigned long __uval = __neg ? (unsigned long)~__val + 1ul : __val; const auto __len = __detail::__to_chars_len(__uval); string __str(__neg + __len, '-'); __detail::__to_chars_10_impl(&__str[__neg], __len, __uval); return __str; } __to_chars_len() is marked as 'constexpr', but __to_chars_10_impl is not, and the latter modifies an allocated buffer; maybe that's why?
- stabbles 6y agoNice, maybe std::string has changed in GCC 10, I was using GCC 9. Shouldn't C++20 have constexpr containers?
- slavik81 6y agoThe result of std::to_string is defined to be the same as sprintf, so it depends on the current locale, which is global, mutable state, and initialized to the user's chosen value at runtime. That makes the optimization impossible to do safely without special knowledge about the locale functions. The awful way that C hacked on locales via implicit global state has been a disaster, leading to both slow and broken code. If you want fast, predictable conversions between numbers and strings, you need to use std::to_chars. Or, std::format for localized conversions.
- deleted 6y ago[deleted]
- slavik81 6y agoAfter a little more spelunking, it seems that this particular case was possible to optimize. While you can set the thousands separator for integers in C++ via locales, it only applies to iostreams and not to sprintf. You can also set it for sprintf in POSIX, but it doesn't apply to integers unless you opt-in via the format specifier. So, there is no applicable locale information needed for integer formatting with std::to_string, and GCC began to replace the naive sprintf-based implementation with one based on std::to_chars in 2017: https://gcc.gnu.org/legacy-ml/gcc-patches/2017-05/msg02103.html https://gcc.gnu.org/legacy-ml/gcc-patches/2017-05/msg02103.h... The problems I previously mentioned would apply if that were a float, though, and I would still recommend using std::to_chars just so you don't have to think about it.