3 ms·
If you think you can detect only cp-1252 i have news for you. This is 1-byte encoding, it can't fit much. So, be prepared for whole zoo of other 1-byte encoding
by out_of_protocol 2y ago
If you think you can detect only cp-1252 i have news for you. This is 1-byte encoding, it can't fit much. So, be prepared for whole zoo of other 1-byte encodings from that era (e.g. cyrillic letters - welcome to cp-1251, where everything above first 128 chars have totally different meaning). Writing encoding detector is not easy at all. Chance of guessing wrong is high. I'm glad most of the world (but not excel) moved on away from that can of worms
> You are suggesting that if software developers willfully refuse to implement
it will happen anyway, sooner or later
> will result in lots of corruption
there are lots of corruption anyway, e.g euro sign, some european letters, whole other alphabets like cyrillic one). The most dangerous are subtle ones like german ẞ)
P.S. I don't have office installed to check, but online version exports non cp-1252 chars as "?" . So nice
P.P.S. Apple Numbers offers choice of encoding on export, with UTF-8 default. Google Sheets exports as UTF-8 without choice
- jrochkind1 2y agoOP is in fact about doing your best to detect encoding without proper tagging. It is not perfect and you won't have 100% accuracy. But it can get pretty decent (OP is literally about the techniques used to do so, statistically etc), and is necessary because of the actual world we live in. If just refusing to do it would get MS to export in UTF-8 by default that would of course have happened a long time ago!