3 ms·
u'\ufeff' in Python string
- PaulHoule 5y agoThe #1 answer is factually correct but wrong from a practical standpoint. The trouble isn't that he came across a file that starts with a byte order mark but rather if you are going to inhale large amounts of text from the web and other low quality sources you are going to get wrongly encoded text. The BOM is allowed by some specs but it is really junk, it dates from an age when 16-bit encodings that might be big or endian encoding were hypothetically possible. In real life you never see those unless somebody is screwing around with Notepad in Windows. You occasionally see the meaningless BOM and it's not that different from the random iso-latin-1 character as a single you see from time to time even in a document that also uses proper utf-8 multibyte sequences. All the time you see multibyte sequences that are wrong according to some readings of the spec. What's unacceptable is crashing when you get text like this so you should be looking at the errors option of open.