5 ms·
You wouldn't prefix every string with it because presumably your API or program's state has already determined the string's encoding. I am not suggesting that
by makecheck 14y ago
You wouldn't prefix every string with it because presumably your API or program's state has already determined the string's encoding. I am not suggesting that every fragment of text has to be explicit (I agree that would be ridiculous). I am only stating facts: there is nothing incorrect about having the mark, a conformant reader must be able to handle the mark, and the mark has some value as a short-cut for avoiding elaborate decoding tricks.
- unconed 14y agoThe thing is, the BOM is metadata, it doesn't belong in content. It violates the contract of .txt files, which is: the entire file is a single string of content. Recognizing it at the edges of your program and stripping it out is not the end of the world, but it's annoying and no other (8-bit) encoding works that way. In fact, I find it hard to believe UTF-8 BOMs in MS programs were anything more than a programmer error. Once such files were out in the wild, everyone else had to deal with them.
- makecheck 14y agoThere are already plenty of cases that valid UTF-8 readers have to deal with (unused ranges of code points, invalid byte combinations, etc.). Ignoring a BOM is trivial by comparison. A UTF-8 reader honestly doesn't care about the "stringness" of a .txt file because of all the other crap that can be in a byte stream. Older programs do care, but as I've said elsewhere in the thread an ASCII file can remain ASCII (no BOM). There's no reason to BOM-ify an old ASCII file if it really is ASCII and only ASCII-expecting programs will ever use it. Over time these old programs will either be upgraded or go away and it will finally be safe to say that inputs must be UTF-8. At that time, the BOM has no reason to exist.