> I never understood why ignoring this special character requires a totally separate encoding. Because the BOM is indistinguishable from the "real" UTF-8 encod

> I never understood why ignoring this special character requires a totally separate encoding. Because the BOM is indistinguishable from the "real" UTF-8 encoding of U+FEFF (zero-width no-break space). Trimming that codepoint in the UTF-8 decoder means that some strings like "\uFEFF" can't be safely round-tripped; adding it in the encoder is invalid in many contexts.