10 ms·
Interestingly, Python is one of those programs. You need to use the special "utf-8-sig" encoding for that, which is not prominently advertised anywhere in the
by nerdponx 3y ago
Interestingly, Python is one of those programs.
You need to use the special "utf-8-sig" encoding for that, which is not prominently advertised anywhere in the documentation (but it is stated deep inside the "Unicode HOWTO").
I never understood why ignoring this special character requires a totally separate encoding.
- duskwuff 3y ago> I never understood why ignoring this special character requires a totally separate encoding. Because the BOM is indistinguishable from the "real" UTF-8 encoding of U+FEFF (zero-width no-break space). Trimming that codepoint in the UTF-8 decoder means that some strings like "\uFEFF" can't be safely round-tripped; adding it in the encoder is invalid in many contexts.