3 ms·

The definition of the ctype functions as working on unsigned char values and EOF + CHAR_BIT being 8 everywhere now basically means that there isn't much locale-

The definition of the ctype functions as working on unsigned char values and EOF + CHAR_BIT being 8 everywhere now basically means that there isn't much locale-specificity to the ctype functions: they can be made to work with ASCII, ISO-8859-*, and... EBCDIC, but not UTF-8 in general (just ASCII) or any Unicode encoding (idk, maybe they can be made to be locale-specific for Shift-JIS, but only for ASCII in Shift-JIS). And... yes, glibc does have support for EBCDIC, which is probably ultimately why it has these run-time indirections in its ctype. There's no other reason to have run-time indirections for ctype functions given the limitation of unsigned char values + EOF. That means this code can be simplified a great deal. Anyways, yes, Drew DeVault's rant misses glibc's need to support EBCDIC, but glibc is exactly like this for every little thing -- an unmaintainable mess. There has to be a better way to produce a fast C library w/o being such a mess on the inside.

kazinator 5y ago
They obviously can be made to work for Unicode; iswhatever(x) can just report the whatever property for code points U+0000 through U+00FF, its documented range. If you want to know about higher-valued code points, use iswwhatever, the wide version.
- Asooka 5y ago
  But that is ASCII. That part of Unicode is a literal copy of ASCII. In any case, just putting an "if out of range, return 0" clause wouldn't hurt performance noticeably given all the indirection already present. If used in a loop, the CPU will predict that branch perfectly every time if your data is correct. There is no reason to just crash.
  - kazinator 5y ago
    ASCII goes to 7F, not to FF; it is a 7 bit character code. Therefore, for instance, isspace(0xA0) might usefully report true if we are in a Unicode locale, otherwise not. The 0x80-0xFF values are also used in 8 bit extensions over ASCII, like ISO-8859 1 and ISO 8859-15 character sets. E.g. 0xE0 is à in ISO-8859 1 (which is, of course, the same as the Unicode U+00E0 but logically distinct). A totally different 8 bit extension is KOI-8. The point is valid that if you don't support any "weird" extensions to ASCII (just ISO Latin) or non-ASCII 8 bit, then there isn't much of a need for run-time table indirection. The cases that may arise can be handled ad hoc. Along the lines of "if we are in an ASCII locale, then report false above 7F, otherwise go through the combined Latin/Unicode combined table".
- cryptonector 5y ago
  No not really. wchar_t is not Unicode. It's whatever the current locale's codeset and encoding demand. No, just say no to all of this. Instead: - use UTF-8 locales - use a Unicode library for all things Unicode - make your own ctype for when it's Just ASCII
  - kazinator 5y ago
    I don't know who you think said wchar_t is Unicode, but it wasn't me. In a given locale, you can almost certainly regard wchar_t as being a continuation of the range of unsigned char. If you want to know whether the value UCHAR_MAX + 1 is alpha-numeric, you can't pass that to isalnum, but if that value is in the range of wint_t, you can pass it to iswalnum. For values 0 to UCHAR_MAX, it would be surprising if isalnum and iswalnum produced different results.

kazinator 5y ago

They obviously can be made to work for Unicode; iswhatever(x) can just report the whatever property for code points U+0000 through U+00FF, its documented range. If you want to know about higher-valued code points, use iswwhatever, the wide version.

Asooka 5y ago
But that is ASCII. That part of Unicode is a literal copy of ASCII. In any case, just putting an "if out of range, return 0" clause wouldn't hurt performance noticeably given all the indirection already present. If used in a loop, the CPU will predict that branch perfectly every time if your data is correct. There is no reason to just crash.
- kazinator 5y ago
  ASCII goes to 7F, not to FF; it is a 7 bit character code. Therefore, for instance, isspace(0xA0) might usefully report true if we are in a Unicode locale, otherwise not. The 0x80-0xFF values are also used in 8 bit extensions over ASCII, like ISO-8859 1 and ISO 8859-15 character sets. E.g. 0xE0 is à in ISO-8859 1 (which is, of course, the same as the Unicode U+00E0 but logically distinct). A totally different 8 bit extension is KOI-8. The point is valid that if you don't support any "weird" extensions to ASCII (just ISO Latin) or non-ASCII 8 bit, then there isn't much of a need for run-time table indirection. The cases that may arise can be handled ad hoc. Along the lines of "if we are in an ASCII locale, then report false above 7F, otherwise go through the combined Latin/Unicode combined table".
cryptonector 5y ago
No not really. wchar_t is not Unicode. It's whatever the current locale's codeset and encoding demand. No, just say no to all of this. Instead: - use UTF-8 locales - use a Unicode library for all things Unicode - make your own ctype for when it's Just ASCII
- kazinator 5y ago
  I don't know who you think said wchar_t is Unicode, but it wasn't me. In a given locale, you can almost certainly regard wchar_t as being a continuation of the range of unsigned char. If you want to know whether the value UCHAR_MAX + 1 is alpha-numeric, you can't pass that to isalnum, but if that value is in the range of wint_t, you can pass it to iswalnum. For values 0 to UCHAR_MAX, it would be surprising if isalnum and iswalnum produced different results.