3 ms·

There is no other Unicode code point that will be encoded in UTF8 with a zero byte anywhere within it. From what I understand, this means that no wild UTF-8 st

• arghwhat 9y ago
  It means that a UTF-8 string can contain the "NUL" character, which is a single null byte. While valid UTF-8, it is still an unprintable character, making it "binary" for most intents and purposes.
  • mikeash 9y ago
    It's worth noting that, while NUL is valid UTF-8, there's a lot of software which will fail strangely when presented with it, since UTF-8 is so often used to interoperate with NUL-terminated C strings. If you write such a string to a C string then it will be silently truncated. I ran into this once with an Objective-C program that created filenames from strings found in files. When presented with a string containing NUL, the code ran, but didn't really work. I'd get "foo" and then ask it to append ".txt" and the result would come back as "foo" still! And it depends on the context in which you use it. Using it as a filename truncated, because you ultimately go through the POSIX-level calls that use C strings. Printing it in the debugger truncated, as evidently it used C strings at some point in that process. Displaying it in a text field in the UI worked perfectly fine, though, as apparently that path never uses C strings and the NUL character is just an invisible zero-space character in the middle.
    • girvo 9y ago
      I ran into this when writing a Docker API client in Nim via its Unix socket! Was very confused until I remembered that.
• TazeTSchnitzel 9y ago
  It means that UTF-8 text will only contain NUL bytes in the same situations that ASCII text does. That is, usually only in binary files.