3 ms·
People say if the only tool you have is a hammer, everything looks like a nail - but when the only tool you have is a left-handed can opener life gets _really w
by tialaramex 7y ago
People say if the only tool you have is a hammer, everything looks like a nail - but when the only tool you have is a left-handed can opener life gets _really weird_ and that's how we got the UTF-8 BOM
UTF-8 BOM is largely a Microsoft idea, they've got a bunch of code that thinks in UCS-2 (now retrofitted to more or less pretend it knows UTF-16) and so it thinks about byte order when decoding text files, and from there a Byte Order Mark in files that don't have byte ordering seems like a reasonable idea.
If the files actually _mean_ something then a UTF-8 BOM just introduces confusion. Lots of code I'm responsible for processes UTF-8 just fine, but if it handles say files full of key = value pairs and your file begins with a BOM, well, OK then, that first key starts with U+FEFF, weird choice but no reason we should disallow that. And of course that isn't what you wanted and so now Windows users are complaining I'm not "compatible".
- mark-r 7y agoThe choice of character for the BOM isn't weird or random - it's a zero-width no-break space, which means it's supposed to be invisible even when your software is capable of displaying it. If you remove whitespace from the beginning and end of your keys you'll be fine. It seems arbitrary, but I don't think they could have made a better choice.
- marcosdumay 7y agoShouldn't U+FEFF be whitespace?
- tialaramex 7y agoSure. So what? Text file formats aren't magically obliged to ignore leading whitespace just because that suits Microsoft. If my format would consider U+0009 TAB or U+0020 SPACE to be part of the key if placed at the start of the key why not U+FEFF ?