5 ms·

No, this is a case of glibc trying to support localization of ctype in spite of the fact that it can't be localized to anything other than English in UTF-8 loca

No, this is a case of glibc trying to support localization of ctype in spite of the fact that it can't be localized to anything other than English in UTF-8 locales, anything other than Latin scripts in ISO-8859-* locales, or English in C/POSIX or EBCDIC locales. And then on top of that trying to be fast. I'd give up on supporting localization for ctype. This makes me think, too, "never use ctype, just hardcode my own that assumes ASCII".

kazinator 5y ago
I don't know about never, but try hard not to use anything whose behavior is influenced by a call to setlocale.
- matheusmoreira 5y ago
  Is that even possible? This setlocale stuff is everywhere. Even basic functions like printf are affected. Reminds me of this incredible commit message: https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f027338b0fab0f5078971fbe https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02... I found it easier to get rid of libc and write freestanding C instead. Linux system calls have none of these problems. This locale bullshit is nowhere to be found. No global state anywhere. No thread-local errno. No stupid stuff like EOF. Writing C became fun again.
  - kazinator 5y ago
    Although printf is affected, not every aspect of printf is affected. IIRC, one nasty area is floating-point numbers, when you're in the locale that doesn't have . as the decimal point, but your program's requirements are such that . is the decimal point. Like, oh, you're writing a programming language which specifies that. Like, you know, C does for floating-point constants.
    - matheusmoreira 5y ago
      > floating-point numbers, when you're in the locale that doesn't have . as the decimal point I live in such a country. It's subtle but constant pain even for average users. Even the Windows calculator screws this up: it only accepts either . or , as input instead of both. > Like, oh, you're writing a programming language which specifies that. Yeah. The commit message I linked has one such example. > This is still less bad than that time when libquivi fucked up OpenGL rendering > calling a libquvi function would load some proxy abstraction library, which in turn loaded a KDE plugin > which in turn called setlocale() because Qt does this > made the mpv GLSL shader generation code emit "," instead of "." for numbers > and of course only for users who had that KDE plugin installed, and lived in a part of the world where "." is not used as decimal separator Just imagine debbuging this insanity.
      - jcelerier 5y ago
        Yeah, as a french it bit me a lot of times. Now I just LC_ALL=C.UTF-8, french translations of english software are shitty most of the time anyways and in particular for dev tools.
      - ygra 5y ago
        > Even the Windows calculator screws this up: it only accepts either . or , as input instead of both. It does, however, always accept the . key on the numeric keypad, so many users won't really notice the discrepancy.
  - gpderetta 5y ago
    If you had to takeaway just one thing from this glorious rant it would be: globals are evil.
  - Someone 5y ago
    OpenBSD man page (https://man.openbsd.org/isalnum.3 https://man.openbsd.org/isalnum.3): “OpenBSD always uses the C locale for these functions, ignoring the global locale, the thread-specific locale, and the locale argument.“ and https://man.openbsd.org/setlocale.3 https://man.openbsd.org/setlocale.3: “On OpenBSD, the only useful value for the category is LC_CTYPE. It sets the locale used for character encoding, character classification, and case conversion. For compatibility with natural language support in packages(7), all other categories — LC_COLLATE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC, and LC_TIME — can be set and retrieved, too, but their values are ignored by the OpenBSD C library. A category of LC_ALL sets the entire locale generically, which is strongly discouraged for security reasons in portable programs.” Probably non-standard, but from what I read here, also a defensible choice.
    - matheusmoreira 5y ago
      That's interesting! Non-standard but it doesn't matter. That's the right thing to do no matter what some piece of paper says.
emmelaich 5y ago
The man pages used to warn that isalnum, ispunct etc were only defined if isascii was true. That seems to have disappeared - which supports your view.

kazinator 5y ago

I don't know about never, but try hard not to use anything whose behavior is influenced by a call to setlocale.

matheusmoreira 5y ago
Is that even possible? This setlocale stuff is everywhere. Even basic functions like printf are affected. Reminds me of this incredible commit message: https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f027338b0fab0f5078971fbe https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02... I found it easier to get rid of libc and write freestanding C instead. Linux system calls have none of these problems. This locale bullshit is nowhere to be found. No global state anywhere. No thread-local errno. No stupid stuff like EOF. Writing C became fun again.
- kazinator 5y ago
  Although printf is affected, not every aspect of printf is affected. IIRC, one nasty area is floating-point numbers, when you're in the locale that doesn't have . as the decimal point, but your program's requirements are such that . is the decimal point. Like, oh, you're writing a programming language which specifies that. Like, you know, C does for floating-point constants.
  - matheusmoreira 5y ago
    > floating-point numbers, when you're in the locale that doesn't have . as the decimal point I live in such a country. It's subtle but constant pain even for average users. Even the Windows calculator screws this up: it only accepts either . or , as input instead of both. > Like, oh, you're writing a programming language which specifies that. Yeah. The commit message I linked has one such example. > This is still less bad than that time when libquivi fucked up OpenGL rendering > calling a libquvi function would load some proxy abstraction library, which in turn loaded a KDE plugin > which in turn called setlocale() because Qt does this > made the mpv GLSL shader generation code emit "," instead of "." for numbers > and of course only for users who had that KDE plugin installed, and lived in a part of the world where "." is not used as decimal separator Just imagine debbuging this insanity.
    - jcelerier 5y ago
      Yeah, as a french it bit me a lot of times. Now I just LC_ALL=C.UTF-8, french translations of english software are shitty most of the time anyways and in particular for dev tools.
    - ygra 5y ago
      > Even the Windows calculator screws this up: it only accepts either . or , as input instead of both. It does, however, always accept the . key on the numeric keypad, so many users won't really notice the discrepancy.
- gpderetta 5y ago
  If you had to takeaway just one thing from this glorious rant it would be: globals are evil.
- Someone 5y ago
  OpenBSD man page (https://man.openbsd.org/isalnum.3 https://man.openbsd.org/isalnum.3): “OpenBSD always uses the C locale for these functions, ignoring the global locale, the thread-specific locale, and the locale argument.“ and https://man.openbsd.org/setlocale.3 https://man.openbsd.org/setlocale.3: “On OpenBSD, the only useful value for the category is LC_CTYPE. It sets the locale used for character encoding, character classification, and case conversion. For compatibility with natural language support in packages(7), all other categories — LC_COLLATE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC, and LC_TIME — can be set and retrieved, too, but their values are ignored by the OpenBSD C library. A category of LC_ALL sets the entire locale generically, which is strongly discouraged for security reasons in portable programs.” Probably non-standard, but from what I read here, also a defensible choice.
  - matheusmoreira 5y ago
    That's interesting! Non-standard but it doesn't matter. That's the right thing to do no matter what some piece of paper says.

emmelaich 5y ago

The man pages used to warn that isalnum, ispunct etc were only defined if isascii was true. That seems to have disappeared - which supports your view.