5 ms·

That reminds me of how I wanted to start a project with a name containing the German 'ö' character. Of course, as customary, I save my projects into folders wit

by Kenji 11y ago

That reminds me of how I wanted to start a project with a name containing the German 'ö' character. Of course, as customary, I save my projects into folders with the name of the project. And, also of course, GCC issues instantly ensued when I tried to compile the C++ source of that project. What did I learn? Stay with ASCII for your code and names. It might be 2015 but apparently it's more important to put poop into unicode than to actually implement the thing in all the important software.

gchpaco 11y ago
Don't idly dismiss emoji. They're the first thing from outside the Basic Multilingual Plane that people actually want to use, and as a result the first thing to find all the bugs and hidden assumptions going from a 16 bit wide character set to a 20 bit wide character set. This helps people who actually needed access to the various CJK Ideograph Extensions, most of which are in Plane 2.
- gsnedders 11y ago
  Nit-pick: 21-bit wide character set.
  - jhallenworld 11y ago
    Thanks to UTF-16, ugh.
  - gchpaco 11y ago
    Truth, I dunno why I have 20 bits stuck in my head.
    - Dylan16807 11y ago
      Probably because it barely goes over 20 bits. 20.09ish If you want to be cheeky you can argue that "hidden assumptions going from a 16 bit wide character set to a 20 bit wide character set" is still correct, because those hidden assumptions are part of UTF-16, and UTF-16 uses exactly 20 bits to express the astral planes, encoding them in a different way from the BMP.
jepler 11y ago
Have you tried this lately? On a Debian Jessie machine using the en_US.UTF-8 locale, I encountered no problems doing this, though I picked œ instead of ö. I did have a little bit of pain from git (git status showed \305\223 for œ) until I did "git config core.quotepath false". Not sure how this'll come through on ycombinator, so it's also on a pastebin for 24 hours: https://paste.debian.net/311370/ https://paste.debian.net/311370/ ~$ cd src ~/src$ mkdir œuvre ~/src$ cd œuvre ~/src/œuvre$ printf '#include <stdio.h>\nint main() {puts("hello œuvre");}\n' > œuvre.c ~/src/œuvre$ g++ œuvre.c -o œuvre ~/src/œuvre$ ./œuvre hello œuvre
- simoncion 11y ago
  I can also confirm success when compiling a file with a path "ömlaut/asdf.cpp". $ g++ --version | head -n 1 g++ (Gentoo Hardened 4.9.3 p1.1, pie-0.6.2) 4.9.3 $ uname -r -m 4.1.6-hardened i686 $ echo $LANG en_US.UTF-8 $ bash --version | head -n 1 GNU bash, version 4.3.42(1)-release (i686-pc-linux-gnu)
- eridius 11y ago
  That's a bit different. Presumably you're running this in a terminal emulator. Terminal emulators (and programmers editors), if they're set to a monospace font, operate on a grid. For characters in the selected monospace font this is just like normal font rendering, but when fallback fonts are used, it still uses the layout from the original monospace font (i.e. the grid) instead of the layout from the fallback font. This means that the fallback font rendering doesn't screw with the columns. Naturally there's still the issue of properly identifying double-wide characters. But as long as you can correctly identify them (and for the ambiguous characters it generally treats them all the same, either as narrow or as wide depending on the software in question), you can simply render them with 2 cells instead of 1, and everything remains lined up. But the article here was showing monospaced text in a web browser, and a web browser doesn't do any of this (nor a rich text editor). Font fallback usually attempts to maintain properties like monospacing, but the fallback font may still have different metrics, meaning it won't line up with the monospaced text from the source font (and depending on the fonts available, if you have no monospaced font with asian glyphs it would have to fall back to a non-monospaced font). I suspect what's going on with the locale stuff here is simply that the locale affects the fallback list, and it ends up picking a different font that happens to have the right metrics to line up. But I can't say for certain that this is the explanation.
GFK_of_xmaspast 11y ago
" it's more important to put poop into unicode than to actually implement the thing in all the important software." I somehow suspect those aren't the same people.
geofft 11y ago
Were you using a UTF-8 locale? (Try running `locale`.)

by Kenji 11y ago

gchpaco 11y ago

Don't idly dismiss emoji. They're the first thing from outside the Basic Multilingual Plane that people actually want to use, and as a result the first thing to find all the bugs and hidden assumptions going from a 16 bit wide character set to a 20 bit wide character set. This helps people who actually needed access to the various CJK Ideograph Extensions, most of which are in Plane 2.

gsnedders 11y ago
Nit-pick: 21-bit wide character set.
- jhallenworld 11y ago
  Thanks to UTF-16, ugh.
- gchpaco 11y ago
  Truth, I dunno why I have 20 bits stuck in my head.
  - Dylan16807 11y ago
    Probably because it barely goes over 20 bits. 20.09ish If you want to be cheeky you can argue that "hidden assumptions going from a 16 bit wide character set to a 20 bit wide character set" is still correct, because those hidden assumptions are part of UTF-16, and UTF-16 uses exactly 20 bits to express the astral planes, encoding them in a different way from the BMP.

jepler 11y ago

Have you tried this lately? On a Debian Jessie machine using the en_US.UTF-8 locale, I encountered no problems doing this, though I picked œ instead of ö. I did have a little bit of pain from git (git status showed \305\223 for œ) until I did "git config core.quotepath false". Not sure how this'll come through on ycombinator, so it's also on a pastebin for 24 hours: https://paste.debian.net/311370/ https://paste.debian.net/311370/ ~$ cd src ~/src$ mkdir œuvre ~/src$ cd œuvre ~/src/œuvre$ printf '#include <stdio.h>\nint main() {puts("hello œuvre");}\n' > œuvre.c ~/src/œuvre$ g++ œuvre.c -o œuvre ~/src/œuvre$ ./œuvre hello œuvre

simoncion 11y ago
I can also confirm success when compiling a file with a path "ömlaut/asdf.cpp". $ g++ --version | head -n 1 g++ (Gentoo Hardened 4.9.3 p1.1, pie-0.6.2) 4.9.3 $ uname -r -m 4.1.6-hardened i686 $ echo $LANG en_US.UTF-8 $ bash --version | head -n 1 GNU bash, version 4.3.42(1)-release (i686-pc-linux-gnu)
eridius 11y ago
That's a bit different. Presumably you're running this in a terminal emulator. Terminal emulators (and programmers editors), if they're set to a monospace font, operate on a grid. For characters in the selected monospace font this is just like normal font rendering, but when fallback fonts are used, it still uses the layout from the original monospace font (i.e. the grid) instead of the layout from the fallback font. This means that the fallback font rendering doesn't screw with the columns. Naturally there's still the issue of properly identifying double-wide characters. But as long as you can correctly identify them (and for the ambiguous characters it generally treats them all the same, either as narrow or as wide depending on the software in question), you can simply render them with 2 cells instead of 1, and everything remains lined up. But the article here was showing monospaced text in a web browser, and a web browser doesn't do any of this (nor a rich text editor). Font fallback usually attempts to maintain properties like monospacing, but the fallback font may still have different metrics, meaning it won't line up with the monospaced text from the source font (and depending on the fonts available, if you have no monospaced font with asian glyphs it would have to fall back to a non-monospaced font). I suspect what's going on with the locale stuff here is simply that the locale affects the fallback list, and it ends up picking a different font that happens to have the right metrics to line up. But I can't say for certain that this is the explanation.

GFK_of_xmaspast 11y ago

" it's more important to put poop into unicode than to actually implement the thing in all the important software." I somehow suspect those aren't the same people.

geofft 11y ago

Were you using a UTF-8 locale? (Try running `locale`.)