3 ms·
I've seen UTF-8 with a BOM while consuming data when integrating with strongly Windows-centric environments. Relatively uncommon, but does happen. And it is ver
by davidwtbuxton 7y ago
I've seen UTF-8 with a BOM while consuming data when integrating with strongly Windows-centric environments. Relatively uncommon, but does happen. And it is very annoying!
- C1sc0cat 7y agoIt used to and maybe still does cause problems with how google parsed robots.txt files! Which is why all my robots.txt files have a comment on the first line.
- Someone1234 7y ago> Which is why all my robots.txt files have a comment on the first line. That doesn't stop a BOM being generated or consumed.
- YSFEJ4SWJUVU6 7y agoBOM is only a problem with strict syntaxes, which robots.txt is not an example of. If the "consumer" simply ignores invalid or meaningless lines, you can avoid issues from invisible characters by not having anything meaningful on the first line of your file.