4 ms·

As pointed out here; http://www.metafilter.com/138858/Artists-Notebook-Ramsey-Nasser#5531489 http://www.metafilter.com/138858/Artists-Notebook-Ramsey-Nas... H

by ldite 12y ago

As pointed out here; http://www.metafilter.com/138858/Artists-Notebook-Ramsey-Nasser#5531489 http://www.metafilter.com/138858/Artists-Notebook-Ramsey-Nas... He's probably using the wrong value for LANG in his terminal.

scott_karana 12y ago
Of course, but shouldn't we have gotten to the point where xx.UTF8 should be the default locale on modern operating systems?
- Crito 12y ago
  Well, except for Windows (which insists on UTF-16/UCS2), as far as I can see we are at that point. Modern Linux distros have come configured with everything set up with UTF-8 for years. Fedora has been that way since Fedora Core 1. I don't know what OSX is doing.. I would assume that normally it is pre-configured with UTF-8, but somehow that was lost in his setup. I don't have an OSX machine to confirm this though.
  - acdha 12y ago
    It is, but all POSIX operating systems are brain-damaged when it comes to things other than interactive terminal windows – they default to LANG=C so e.g. the same script which runs interactively crashes when you run it in cron or some other launcher.
    - Crito 12y ago
      Hmm, is OXS still actually POSIX compliant? I thought they haven't bothered with certification for the past few versions. Anyway, if a changed LANG breaks your script, I'd primarily place the blame on the script. You should be able to shuffle around UTF-8 bytes non-interactively without paying much mind to that.
      - acdha 12y ago
        The reason the Linux distributions don't set UTF-8 in the default environment is for backwards (bugwards?) compatibility with legacy code where e.g. something like Python/Perl/etc. might start actually throwing exceptions if it thinks you want UTF-8 and you gave it either a different encoding or just unvalidated garbage. There's no solution to this which will make everyone happy. I prefer to set UTF-8 for everything so I can find things which break and fix them but a lot of legacy shops choose not to spend the time fixing things which are “working”.
  - scott_karana 12y ago
    I decided to see what OS X did, so I created a new account and pre-configured nothing. Terminal.app, on 10.9.2: Scotts-MacBook-Air:~ test$ locale LANG="en_CA.UTF-8" LC_COLLATE="en_CA.UTF-8" LC_CTYPE="en_CA.UTF-8" LC_MESSAGES="en_CA.UTF-8" LC_MONETARY="en_CA.UTF-8" LC_NUMERIC="en_CA.UTF-8" LC_TIME="en_CA.UTF-8" LC_ALL= Scotts-MacBook-Air:~ test$ echo $TERM xterm-256color Arabic text copy/pastes into the terminal and appears to be displayed correctly. (I don't read it, so I wouldn't know if there were subtle errors)

by ldite 12y ago

scott_karana 12y ago

Of course, but shouldn't we have gotten to the point where xx.UTF8 should be the default locale on modern operating systems?

Crito 12y ago
Well, except for Windows (which insists on UTF-16/UCS2), as far as I can see we are at that point. Modern Linux distros have come configured with everything set up with UTF-8 for years. Fedora has been that way since Fedora Core 1. I don't know what OSX is doing.. I would assume that normally it is pre-configured with UTF-8, but somehow that was lost in his setup. I don't have an OSX machine to confirm this though.
- acdha 12y ago
  It is, but all POSIX operating systems are brain-damaged when it comes to things other than interactive terminal windows – they default to LANG=C so e.g. the same script which runs interactively crashes when you run it in cron or some other launcher.
  - Crito 12y ago
    Hmm, is OXS still actually POSIX compliant? I thought they haven't bothered with certification for the past few versions. Anyway, if a changed LANG breaks your script, I'd primarily place the blame on the script. You should be able to shuffle around UTF-8 bytes non-interactively without paying much mind to that.
    - acdha 12y ago
      The reason the Linux distributions don't set UTF-8 in the default environment is for backwards (bugwards?) compatibility with legacy code where e.g. something like Python/Perl/etc. might start actually throwing exceptions if it thinks you want UTF-8 and you gave it either a different encoding or just unvalidated garbage. There's no solution to this which will make everyone happy. I prefer to set UTF-8 for everything so I can find things which break and fix them but a lot of legacy shops choose not to spend the time fixing things which are “working”.
- scott_karana 12y ago
  I decided to see what OS X did, so I created a new account and pre-configured nothing. Terminal.app, on 10.9.2: Scotts-MacBook-Air:~ test$ locale LANG="en_CA.UTF-8" LC_COLLATE="en_CA.UTF-8" LC_CTYPE="en_CA.UTF-8" LC_MESSAGES="en_CA.UTF-8" LC_MONETARY="en_CA.UTF-8" LC_NUMERIC="en_CA.UTF-8" LC_TIME="en_CA.UTF-8" LC_ALL= Scotts-MacBook-Air:~ test$ echo $TERM xterm-256color Arabic text copy/pastes into the terminal and appears to be displayed correctly. (I don't read it, so I wouldn't know if there were subtle errors)