6 ms·

> It is often best to avoid non-ASCII characters in source code. Indeed, in some cases, there is no standard way to tell the compiler about your character encod

> It is often best to avoid non-ASCII characters in source code. Indeed, in some cases, there is no standard way to tell the compiler about your character encoding, so non-ASCII might trigger problems. To check all non-ASCII characters, you may do [^\x00-\x7F]. Depends of the language. In Python 3, files are expected to be utf8 by default, and you can change that by adding a "# coding: <charset>" header. In fact, it's one of the reasons it was a breaking release in the first place, and being able to put non-ASCII characters in strings and comments in my source code are a huge plus. I mentioning this because in another article, the author said being used to code in "C++, C, Go, Java, JavaScript, Python, R, Swift, Rust, C#" so I find it kinda weird.

kazinator 6y ago
I wouldn't say that everything in \x00 to \x1f is OK in source code.
- chrismorgan 6y ago
  Eh, I reckon they’re fine. In shell scripts especially (and sometimes elsewhere), if I want an ANSI escape code, I’ll type the actual bytes I want, because it’s more reliable and easier to type <C-V><Esc> and see ^[ in Vim than it is to worry about whether I’m using \u001b, \u{1b}, \x1b, \033 or something else, and whether I have '', "", $'', $"", printf, echo -e, &c. &c. Speaking generally, if you use the actual control characters, the code will either fail to run (parse or compile error) or work, whereas encoding your control codes with escape sequences is more hit and miss. I’ve seen quite a few others using control characters in this way too. Truth to tell, I’m quite liberal in using all of Unicode in my source code. About the only places I wouldn’t use the actual characters are when (a) the environment requires me to escape the thing; (b) the numeric values of the characters are more meaningful than the characaters themselves; and (c) when dealing with combining characters and modifiers, because otherwise they’ll do things like combine with the opening ' from the string/character literal.
  - kazinator 6y ago
    I mean, in non-strawman examples of programming languages. The code will work, but how will it edit outside of your personal environment? Or print? If you quote it in a PDF paper, will it be copy-pastable? Basically I would worry about all that sort of stuff. If I have echo '^L' to clear the screen, and that program is sent to a line printer, will I get: echo ' [ paper ejected ] ' If you so much as "cat" your code to a terminal, all the codes get interpreted.
    - morelisp 6y ago
      In Emacs Lisp it's common to to include real ^L specifically to trigger page breaks from when printing code was more common (and it has page-aware navigation commands accordingly).
speedplane 6y ago
> It is often best to avoid non-ASCII characters in source code. ... >> Depends of the language. In Python 3, files are expected to be utf8 by default, and you can change that by adding a "# coding: <charset>" header. It's interesting that many languages avoid unicode and non-ASCII text, yet they make assumptions about file and directory structures about the underlying system. It's as if interpreting directory and file system structures is "okay", but interpreting file formats is not. > In fact, it's one of the reasons it was a breaking release in the first place, and being able to put non-ASCII characters in strings and comments in my source code are a huge plus. Sorry, but as a Python dev that went from 2 to 3, yes native unicode features are nice, but no, it was not worth breaking two decades of existing code.
- BiteCode_dev 6y ago
  As somebody living in Europe, I think it's a perspective you can have only if you live mostly in an English speaking world. Up to 2015, my laptop was plagued with Python software (and others) crashing because they didn't handle unicode properly. Reading from my "Vidéos" directory, entering my first name, dealing with the "février" month in date parsing... For the USA, things just work most of the time. For us, it's a nightmare. It's even worse because most software is produced by English speaking people that have your attitude, and are completely oblivious about the crashing bugs they introduce on a daily basis. In Asia it's even more terrible. And I've heard the people saying you can perfectly code unicode aware software in Python 2. Yes, you can, just like you can code memory safe code in C. In fact, just teaching Python 2 is painful in Europe. The student write their name in a comment ? Crashes if you forget the encoding header. Using raw_input() with unicode to ask a question? Crashes. Reading bytes from an image file ? Get back a string object. Got garbage bytes in string object ? Silently concatenate with a valid string and produce garbage.
  - qsort 6y ago
    > As somebody living in Europe, I think it's a perspective you can have only if you live mostly in an English speaking world. I live in Europe and I (mostly) agree that (most) code shouldn't (usually) contain any codepoint greater than 127. It's a simple matter of reducing the surface area of possible problems. Code needs to work across machines and platforms, and it's basically guaranteed that someone, somewhere is going to screw up the encoding. I know it shouldn't happen, but it will happen, and ASCII mostly works around that problem. Another issue is readability. I know ASCII by heart, but I can't memorize Unicode. If identifiers were allowed to contain random characters, it would make my job harder for basically no reason. Furthermore, the entire ASCII charset (or at least the subset of ASCII that's relevant for source code) can easily be typed from a keyboard. Almost everyone I know uses the US layout for programming because European ones frankly just suck, and that means typing accents, diacritics, emoji or what have you is harder, for no real discernible reason. String handling is a different issue, and I agree that native functions for a modern language should be able to handle utf8 without crashing. The above only applies to the literal content of source file.
O_H_E 6y ago
> It is often best to avoid non-ASCII characters in source code. Scientists writing Julia are laughing on the floor.
- kps 6y ago
  Sometimes I wish COBOL style hadn't won over mathematical style as the convention for every general purpose language except possibly Go.
  - marcosdumay 6y ago
    There are many languages pushing for mathematical style formulas, but until they get easy to type, I don't think any of them will succeed. It's not COBOL that won. It's qwerty.
    - eigenspace 6y ago
      Julia makes typing unicode characters very easy; so easy that I keep a Julia repl open a lot of the time just for typing various mathematical symbols easily. For example, to type α², you just type \alpha, hit the tab button to turn it into α, and then type \^2 and hit tab again to turn that into ². If someone gives you a unicode character that you don't know how to type, you just hit the ? button to enter the repl-help mode and then paste the character in. help?> α² "α²" can be typed by \alpha<tab>\^2<tab> search: Couldn't find α² No documentation found. Binding α² does not exist.
      - snicker7 6y ago
        The amazing thing about this UI is that it just works across many of different editors and REPLs (VSCode, Emacs, Pluto, &etc). It reminds me of GNU TeXMacs, which has a similar interface for equations. That is, typing "\int" followed by tab will render an integral sign.

kazinator 6y ago

I wouldn't say that everything in \x00 to \x1f is OK in source code.

chrismorgan 6y ago
Eh, I reckon they’re fine. In shell scripts especially (and sometimes elsewhere), if I want an ANSI escape code, I’ll type the actual bytes I want, because it’s more reliable and easier to type <C-V><Esc> and see ^[ in Vim than it is to worry about whether I’m using \u001b, \u{1b}, \x1b, \033 or something else, and whether I have '', "", $'', $"", printf, echo -e, &c. &c. Speaking generally, if you use the actual control characters, the code will either fail to run (parse or compile error) or work, whereas encoding your control codes with escape sequences is more hit and miss. I’ve seen quite a few others using control characters in this way too. Truth to tell, I’m quite liberal in using all of Unicode in my source code. About the only places I wouldn’t use the actual characters are when (a) the environment requires me to escape the thing; (b) the numeric values of the characters are more meaningful than the characaters themselves; and (c) when dealing with combining characters and modifiers, because otherwise they’ll do things like combine with the opening ' from the string/character literal.
- kazinator 6y ago
  I mean, in non-strawman examples of programming languages. The code will work, but how will it edit outside of your personal environment? Or print? If you quote it in a PDF paper, will it be copy-pastable? Basically I would worry about all that sort of stuff. If I have echo '^L' to clear the screen, and that program is sent to a line printer, will I get: echo ' [ paper ejected ] ' If you so much as "cat" your code to a terminal, all the codes get interpreted.
  - morelisp 6y ago
    In Emacs Lisp it's common to to include real ^L specifically to trigger page breaks from when printing code was more common (and it has page-aware navigation commands accordingly).

speedplane 6y ago

> It is often best to avoid non-ASCII characters in source code. ... >> Depends of the language. In Python 3, files are expected to be utf8 by default, and you can change that by adding a "# coding: <charset>" header. It's interesting that many languages avoid unicode and non-ASCII text, yet they make assumptions about file and directory structures about the underlying system. It's as if interpreting directory and file system structures is "okay", but interpreting file formats is not. > In fact, it's one of the reasons it was a breaking release in the first place, and being able to put non-ASCII characters in strings and comments in my source code are a huge plus. Sorry, but as a Python dev that went from 2 to 3, yes native unicode features are nice, but no, it was not worth breaking two decades of existing code.

BiteCode_dev 6y ago
As somebody living in Europe, I think it's a perspective you can have only if you live mostly in an English speaking world. Up to 2015, my laptop was plagued with Python software (and others) crashing because they didn't handle unicode properly. Reading from my "Vidéos" directory, entering my first name, dealing with the "février" month in date parsing... For the USA, things just work most of the time. For us, it's a nightmare. It's even worse because most software is produced by English speaking people that have your attitude, and are completely oblivious about the crashing bugs they introduce on a daily basis. In Asia it's even more terrible. And I've heard the people saying you can perfectly code unicode aware software in Python 2. Yes, you can, just like you can code memory safe code in C. In fact, just teaching Python 2 is painful in Europe. The student write their name in a comment ? Crashes if you forget the encoding header. Using raw_input() with unicode to ask a question? Crashes. Reading bytes from an image file ? Get back a string object. Got garbage bytes in string object ? Silently concatenate with a valid string and produce garbage.
- qsort 6y ago
  > As somebody living in Europe, I think it's a perspective you can have only if you live mostly in an English speaking world. I live in Europe and I (mostly) agree that (most) code shouldn't (usually) contain any codepoint greater than 127. It's a simple matter of reducing the surface area of possible problems. Code needs to work across machines and platforms, and it's basically guaranteed that someone, somewhere is going to screw up the encoding. I know it shouldn't happen, but it will happen, and ASCII mostly works around that problem. Another issue is readability. I know ASCII by heart, but I can't memorize Unicode. If identifiers were allowed to contain random characters, it would make my job harder for basically no reason. Furthermore, the entire ASCII charset (or at least the subset of ASCII that's relevant for source code) can easily be typed from a keyboard. Almost everyone I know uses the US layout for programming because European ones frankly just suck, and that means typing accents, diacritics, emoji or what have you is harder, for no real discernible reason. String handling is a different issue, and I agree that native functions for a modern language should be able to handle utf8 without crashing. The above only applies to the literal content of source file.

O_H_E 6y ago

> It is often best to avoid non-ASCII characters in source code. Scientists writing Julia are laughing on the floor.

kps 6y ago
Sometimes I wish COBOL style hadn't won over mathematical style as the convention for every general purpose language except possibly Go.
- marcosdumay 6y ago
  There are many languages pushing for mathematical style formulas, but until they get easy to type, I don't think any of them will succeed. It's not COBOL that won. It's qwerty.
  - eigenspace 6y ago
    Julia makes typing unicode characters very easy; so easy that I keep a Julia repl open a lot of the time just for typing various mathematical symbols easily. For example, to type α², you just type \alpha, hit the tab button to turn it into α, and then type \^2 and hit tab again to turn that into ². If someone gives you a unicode character that you don't know how to type, you just hit the ? button to enter the repl-help mode and then paste the character in. help?> α² "α²" can be typed by \alpha<tab>\^2<tab> search: Couldn't find α² No documentation found. Binding α² does not exist.
    - snicker7 6y ago
      The amazing thing about this UI is that it just works across many of different editors and REPLs (VSCode, Emacs, Pluto, &etc). It reminds me of GNU TeXMacs, which has a similar interface for equations. That is, typing "\int" followed by tab will render an integral sign.