3 ms·

Tell that to my coworkers. I still get emails encoded in SJIS every day, sometimes with attachments with the file name also encoded in SJIS, which results in fu

Tell that to my coworkers. I still get emails encoded in SJIS every day, sometimes with attachments with the file name also encoded in SJIS, which results in funny mojibake when saving them to disk. Not to mention the many web forms that insist you need to write your name in full-width characters or whatever funky shit. On the other I recently got some Python scripts to crash because someone in the European team decided to encode some texts in ISO-8859-1 and Python assumes everything is in UTF-8. I really, really wish one day all legacy encodings will disappear from the face of the Earth and only UTF-8 will stay.

anthk 6y ago
Iconv is your friend. On that issue, infsp6 (the Spanish library for Inform6, akin to the English inform6lib one) still uses iso8859-15 and it's a pain in the ass to convert the encoding to and from utf8 if you don't use neither emacs, joe or vim to edit the source code (I use nvi).
- pezezin 6y ago
  Mátalos a todos y que dios elija a los suyos. But at least it's not EBCDIC, the day I find that in the wild is the day I will retire from computers and become a farmer.
aYsY4dDQ2NrcNzA 6y ago
Out of curiosity, have you tried the UTF-8 decoder capability and stress test? https://www.w3.org/2001/06/utf-8-wrong/UTF-8-test.html https://www.w3.org/2001/06/utf-8-wrong/UTF-8-test.html
- pezezin 6y ago
  No, why? Thunderbird will display SJIS emails just fine. The problem with attachments is when some adds a ZIP with SJIS filenames, but then it's not Thunderbird's problem but whatever tool you use to decompress it. Regarding Python, the default behaviour when decoding and invalid UTF-8 strings is to raise an exception. But your comment made me research it and I just found that there is a way to replace invalid bytes with U+FFFD, so I will try it.
nonbirithm 6y ago
Not to mention that the Linux unzip utility doesn't have a way to handle Shift-JIS filenames, or really any filename encodings besides UTF-8. You have to use an entirely different program like unzip-jp just for those files, in order to not be left with dozens of unintelligible folder names. There's a reason the underground community calls it "shit-jizz."

anthk 6y ago

Iconv is your friend. On that issue, infsp6 (the Spanish library for Inform6, akin to the English inform6lib one) still uses iso8859-15 and it's a pain in the ass to convert the encoding to and from utf8 if you don't use neither emacs, joe or vim to edit the source code (I use nvi).

pezezin 6y ago
Mátalos a todos y que dios elija a los suyos. But at least it's not EBCDIC, the day I find that in the wild is the day I will retire from computers and become a farmer.

aYsY4dDQ2NrcNzA 6y ago

Out of curiosity, have you tried the UTF-8 decoder capability and stress test? https://www.w3.org/2001/06/utf-8-wrong/UTF-8-test.html https://www.w3.org/2001/06/utf-8-wrong/UTF-8-test.html

pezezin 6y ago
No, why? Thunderbird will display SJIS emails just fine. The problem with attachments is when some adds a ZIP with SJIS filenames, but then it's not Thunderbird's problem but whatever tool you use to decompress it. Regarding Python, the default behaviour when decoding and invalid UTF-8 strings is to raise an exception. But your comment made me research it and I just found that there is a way to replace invalid bytes with U+FFFD, so I will try it.

nonbirithm 6y ago

Not to mention that the Linux unzip utility doesn't have a way to handle Shift-JIS filenames, or really any filename encodings besides UTF-8. You have to use an entirely different program like unzip-jp just for those files, in order to not be left with dozens of unintelligible folder names. There's a reason the underground community calls it "shit-jizz."