4 ms·
Encoding issues in a spreadsheet received from digital collection donors
- sheetjs 4y agoUnfortunately the source spreadsheet was not shared, but the root cause was a misreported codepage (which you can actually fix with a hex editor!) "Â ", where the second character is a unicode non-breaking space characters (U+00A0), is the CP1252 encoding of the byte sequence [0xC2, 0xA0] which is the UTF-8 encoding of "\u00A0". So all that really needs to happen is to get Excel to think that the file is UTF-8 encoded. The fix? For XLS, there is a special Codepage record that indicates how strings are to be interpreted. The magic value to force UTF-8 interpretation is 65001, and it is possible to use a hex editor to find exactly where that record is located and change it.
- kgeist 4y agoAs a speaker of a Cyrillic-based language, this occurs quite frequently. It was once so common (before most sites adopted UTF8), they came up with a tool to detect the current encoding and convert it to readable Cyrillic automatically: https://www.artlebedev.ru/decoder/ https://www.artlebedev.ru/decoder/ (the tool exists since 1997)
- captn3m0 4y agohttps://en.wikipedia.org/wiki/Mojibake#English https://en.wikipedia.org/wiki/Mojibake#English
- hgazx 4y agoThat happened because the xls file was opened with something other than excel. Accept no imitations!
- netsharc 4y agoFrom my knowledge it's because Excel expects CSV files to be ANSI-encoded, but a lot of implementers just generate CSVs with UTF-8 -- which they're arguably entitled to do, it's Excel which is the one being stupid.