3 ms·

I think we need more well-maintained "parts" that can be used as alternatives to libc. For instance, string operations can be 5x faster on AVX2, but not every d

by herf 5y ago

I think we need more well-maintained "parts" that can be used as alternatives to libc. For instance, string operations can be 5x faster on AVX2, but not every distro provides this code. I recently upgraded some Centos 6 machines (glibc 2.17) to Debian 10 (glibc 2.28) and system-provided strstr got about 5x faster, which was really important to my binary. I imagine similar differences exist between musl and glibc, due to lack of AVX2 or some other Intel optimizations in musl. But meanwhile, everyday functions like strstr or memcmp (that don't even allocate memory) are totally glued to the system libc (often with a slow version for your architecture). There is no broken-out "fast intel string ops" library that you can link with to ensure great+consistent performance on your target CPU (though I guess you could roll your own with nasm). Are big shared libc libraries good for security or other reasons? Maybe. But it does not make performance consistent across distros. It's fine for different distros to be 10% different in speed, but it's really not okay for them to be 500% apart.

nonameiguess 5y ago
Some things like this do exist. Some programs or even entire customized distros will LD_PRELOAD jemalloc for size or hardened malloc for security (I'm sure there's a performance-optimized version out there I don't know about). There's nothing at all stopping developers from providing optimized subsets of libc intended for more specific use cases than the fully-general system-wide libc. Note that when you're comparing Centos 6 to Debian 10, you're comparing a system released in 2011 that went end of life last November to a system released in 2019. I don't think it's that surprising that 8 years of development brought a 500% performance increase. Presumably, Centos 8 (released 17 days after Debian 10) would have had the newer glibc.
- herf 5y ago
  I benchmarked an Alpine's musl doing strstr with a 200-byte "haystack" with a 10-byte "needle" and it's 8x slower than Debian 10's glibc.
  - erk__ 5y ago
    It could be interesting to see how BSDs compare here
dralley 5y ago
> It's fine for different distros to be 10% different in speed, but it's really not okay for them to be 500% apart. Debian 10 release date: 6 July 2019 Centos 6 release date: 10 July 2011 This is hardly an apples-to-apples comparison.

by herf 5y ago

nonameiguess 5y ago

Some things like this do exist. Some programs or even entire customized distros will LD_PRELOAD jemalloc for size or hardened malloc for security (I'm sure there's a performance-optimized version out there I don't know about). There's nothing at all stopping developers from providing optimized subsets of libc intended for more specific use cases than the fully-general system-wide libc. Note that when you're comparing Centos 6 to Debian 10, you're comparing a system released in 2011 that went end of life last November to a system released in 2019. I don't think it's that surprising that 8 years of development brought a 500% performance increase. Presumably, Centos 8 (released 17 days after Debian 10) would have had the newer glibc.

herf 5y ago
I benchmarked an Alpine's musl doing strstr with a 200-byte "haystack" with a 10-byte "needle" and it's 8x slower than Debian 10's glibc.
- erk__ 5y ago
  It could be interesting to see how BSDs compare here

dralley 5y ago

> It's fine for different distros to be 10% different in speed, but it's really not okay for them to be 500% apart. Debian 10 release date: 6 July 2019 Centos 6 release date: 10 July 2011 This is hardly an apples-to-apples comparison.