3 ms·
Ninja is mostly encoding agnostic, as long as the bytes Ninja cares about (like slashes in paths) are ASCII. This means e.g. UTF-8 or ISO-8859-1 input files oug
by julian37 14y ago
Ninja is mostly encoding agnostic, as long as the bytes Ninja cares about (like slashes in paths) are ASCII. This means e.g. UTF-8 or ISO-8859-1 input files ought to work.
I was going to say that this is brittle because UTF-8 multi-byte sequences might contain bytes such as 0x2F (forward slash) without actually encoding a slash... but it turns out that's wrong. All bytes in multi-byte sequences always have the high bit set, so you can look for ASCII-7 characters in UTF-8 strings without having to worry about getting false positives. That's a very useful property of UTF-8 I wasn't (consciously) aware of before.
http://en.wikipedia.org/wiki/UTF-8#Description http://en.wikipedia.org/wiki/UTF-8#Description
- groby_b 14y agoIt was actually one of the design goals of UTF-8. As much of a pain as dealing with it is, it certainly had a lot of forethought put in.
- grogs 14y agoUTF-8 is just... really good. The only disadvantage I can really think of is that the implementations must be fairly complex. That and determining character count can no longer be calculated from filesize.