4 ms·
To avoid those kind of issues, non-ascii characters are forbidden in our code base. They are automatically verified in a pre-commit git hook.
by hleszek 7y ago
To avoid those kind of issues, non-ascii characters are forbidden in our code base. They are automatically verified in a pre-commit git hook.
- unfunco 7y agoIt should be checked as part of the CI too, if you're doing that, some people might never install the hooks, and some people might git commit --no-verify.
- hleszek 7y agoIndeed but in our case it is for a private repo with only 5 developers. The hooks are installed automatically when running make.
- rimliu 7y agoIt's like preventing the headache with a guillotine.
- deelly 7y agoWhy? Usually non-ascii characters belongs to translation, and translation usually not belongs to codebase.
- onei 7y agoMaybe it works in your english codebase, but I'd be very hesitant about rolling it out everywhere. For example, Perl 6 supports unicode in identifiers so it's perfectly valid to write your code in Japanese. Another increasingly common example I've seen in python is the use of emoji in command line tools.
- dragonwriter 7y ago> For example, Perl 6 supports unicode in identifiers so it's perfectly valid to write your code in Japanese. Raku/Perl6 also has non-ASCII Unicode operators (they all have multicharacter ASCII aliases, but the Unicode characters are usually more readable.)
- TwoHeadedBeast 7y agoJust off the top of my head, test cases would be a valid reason to have non-ASCII in your codebase.
- AstralStorm 7y agoThey definitely belong in comments though. Names, non-English languages and the like.
- Symbiote 7y agoThis (primarily American) attitude to encodings is why we're in this situation.
- michaelt 7y agoDocumentation stored in your repo, unit tests, and comments. If your README.md includes code bracketed with backticks, you've got non-ASCII characters in your repo. Things like Péter Rózsa or Kg.m² or ±0.5 PPM or C11 Standard §6.3.1.3/2 all work with my toolchain. Why mangle them into ASCII when there's no need to?
- BlueTemplar 7y agoThis issue would never have happened using modern software. And if your text (including source code) still isn't in UTF-8, you're doing it wrong. (I guess unless you're using some specific, very performance-conscious system that has to use 7/8-bit characters and is never going to connect to the Internet anyway.)
- Symbiote 7y agoI intentionally add non-ASCII characters to our code, so that an incorrectly configured IDE or bad tool fails. 75% of the development team has at least one non-ASCII character in their name, so it would be pretty rude otherwise. It's much better to knowingly reject a tool at the start, since it can't handle ordinary characters, than find out a year later with the first e.g. British customer that it can't handle "£".
- kortex 7y agoGmail ignores anything past the + in username+foo@gmail.com. I wonder how much bobby-tables-esque havoc I can wreak with name+£€ϵ@gmail.com. Edit: already found one. Leet is supposed to end with Compart › unicode “” U+2608 Thunderstorm Unicode Character. Doesn't show up on my comment.
- bloak 7y agoIf you don't allow non-ascii then you may have to mangle some people's names in the copyright header. It's good to check files for unexpected characters, though. Here's some Perl to do it: perl -e 'binmode(STDIN, ":utf8"); binmode(STDOUT, ":utf8"); foreach (split(//, join("", <STDIN>))) { ++$c{$_}; } foreach (sort(keys %c)) { printf "%8d %s\n", $c{$_}, $_; }' I tested that on a text file I was working on ... and I discovered that the file contained a BOM (U+FEFF), not at the start of the file, but at a random point in the middle of the file. I've deleted it. Who knows what problems it might have caused for me later? You could have a pre-commit git hook that refers to a whitelist of allowed non-ascii characters, or also allow all alphabetic characters, or something like that.