3 ms·

I was wondering whether there was a significant performance improvement of the table lookup glibc does over the naive implementation musl has. The answer is a

by minus7 6y ago

I was wondering whether there was a significant performance improvement of the table lookup glibc does over the naive implementation musl has. The answer is a pretty big yes. Glibc's isalnum is about 6-7 times faster than musl's. Benchmark code: https://paste.sr.ht/~minus/18b44cfe58789bc1fb69494130e859a1189d1772 https://paste.sr.ht/~minus/18b44cfe58789bc1fb69494130e859a11...

saagarjha 6y ago
Color me surprised, since locale stuff usually makes all these functions really slow…
- JdeBP 6y ago
  Speaking as someone who wrote a Standard C library like this many years ago, I can confirm. All of the work is done inside setlocale(), switching (or constructing) tables. The character type functions devolve to a single table lookup. For those interested: The technique is used in P. J. Plauger's 1992 The Standard C Library and has been around a long time.
ddevault 6y ago
Here's the thing: isalnum has never been a bottleneck in my code. The sum amount of runtime that glibc's optimizations have saved my programs is almost certainly significantly less than the amount of time I had to spend untangling glibc's bullshit when it broke. Isn't "programmer time is more expensive than computer time" the lie that we've built this whole mess on anyway? And even if you want to use a LUT, other implementations have incorporated a LUT without this segfault issue, for instance plan 9: https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/include/ctype.h https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/... https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/src/libc/port/ctype.c https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/...
- minus7 6y ago
  Absolutely, isalnum et al are probably never going to be a bottleneck. I was curious whether it even made a difference at all, after all, the naive implementation doesn't really do a lot. And I fully agree with you that this glibc code is awful. As evidenced by the Plan 9 implementation, table lookups can be done simple as well, and probably perform just the same.
  - saagarjha 6y ago
    Parsing?
- cygx 6y ago
  At first glance, the plan9 code looks broken to me: EOF is commonly defined as -1, which gets converted to unsigned char (ie 0xFF) before lookup. If your encoding uses the full 8-bit codespace, that's the wrong thing to do...
  - theamk 6y ago
    That’s probably because they do not care about non-US encodings. I dearly wish entire world would run on us-ascii, it would make software so much easier... but unfortunately there are whole countries which disagree with me.
- cycloptic 6y ago
  So, why not submit a patch to fix the segfault? Or submit a bug report at least? Bashing other projects might be fun but it's rarely productive.

by minus7 6y ago

saagarjha 6y ago

Color me surprised, since locale stuff usually makes all these functions really slow…

JdeBP 6y ago
Speaking as someone who wrote a Standard C library like this many years ago, I can confirm. All of the work is done inside setlocale(), switching (or constructing) tables. The character type functions devolve to a single table lookup. For those interested: The technique is used in P. J. Plauger's 1992 The Standard C Library and has been around a long time.

ddevault 6y ago

Here's the thing: isalnum has never been a bottleneck in my code. The sum amount of runtime that glibc's optimizations have saved my programs is almost certainly significantly less than the amount of time I had to spend untangling glibc's bullshit when it broke. Isn't "programmer time is more expensive than computer time" the lie that we've built this whole mess on anyway? And even if you want to use a LUT, other implementations have incorporated a LUT without this segfault issue, for instance plan 9: https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/include/ctype.h https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/... https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/src/libc/port/ctype.c https://code.9front.org/hg/plan9front/file/00397858f3d8/sys/...

minus7 6y ago
Absolutely, isalnum et al are probably never going to be a bottleneck. I was curious whether it even made a difference at all, after all, the naive implementation doesn't really do a lot. And I fully agree with you that this glibc code is awful. As evidenced by the Plan 9 implementation, table lookups can be done simple as well, and probably perform just the same.
- saagarjha 6y ago
  Parsing?
cygx 6y ago
At first glance, the plan9 code looks broken to me: EOF is commonly defined as -1, which gets converted to unsigned char (ie 0xFF) before lookup. If your encoding uses the full 8-bit codespace, that's the wrong thing to do...
- theamk 6y ago
  That’s probably because they do not care about non-US encodings. I dearly wish entire world would run on us-ascii, it would make software so much easier... but unfortunately there are whole countries which disagree with me.
cycloptic 6y ago
So, why not submit a patch to fix the segfault? Or submit a bug report at least? Bashing other projects might be fun but it's rarely productive.