4 ms·

It's not the names, it's the technology. Ask any non-US, non-UK, non MS-Windows computer user: any document not in the original ASC-127 subset is prone to be ma

It's not the names, it's the technology. Ask any non-US, non-UK, non MS-Windows computer user: any document not in the original ASC-127 subset is prone to be mangled for being shown with the wrong character set, wrong linefeed configuration, not having the right BOM, not allowing right-to-left... Open the document in the wrong application, and the text will be interspeded with ∆ and °, or ® and▯in the places of all the non-English characters, with no obvious way to fix the encoding. The most robust solution would be that each document-showing application had a dialog showing how the text is displayed in all the supported codesets at once, allowing the user to choose the right one. Instead, we get config options buried deep in the menu, showing endless lists of cryptic options, forcing the user to try then one by one. It's frustrating to know that the correct configuration is one click away, but having no idea which one is the right option. Mmmmmhhh... That gives me an idea for an app...

majewsky 8y ago
> any document not in the original ASC-127 subset is prone to be mangled for being shown with the wrong character set, wrong linefeed configuration, not having the right BOM, not allowing right-to-left... It got much better in the last years as UTF-8 gained wider adoption. The problems are mostly with legacy systems in government. (Cannot comment on that since my name fits in ASCII.)
- TuringTest 8y ago
  Maybe for registering to government systems... Using desktop applications is still a nightmare, where opening any document from the internet or navigating to a foreign page is hit or miss. Also, the main frustration is not that it happens often; it's that when you encounter it, fixing it may involve studying the whole software stack to find out where the setting is switched or at what point in the toolchain the format was mishandled.
  - TuringTest 8y ago
    And there are non-zero chances that changing to the wrong option and saving the document will permanently break the file as the combination of two wrongly configured codesets. I don't know, maybe it's me who's doing something wrong and turn to using only a small subset of software tools, properly configured? But I do need to process documents from many origins using a variety of different tools; I've never found a good programable multimedia editor that satisfies all my needs (something like emacs but with WYSIWIG capabilities).
  - Dylan16807 8y ago
    What kinds of documents are these? Because while sometimes I don't have the right fonts installed, never in my life have I seen a foreign web page fail to decode. The only mojibake I've seen is text that was corrupted during posting, not even showing up right to the author. Documents that people make have always worked as intended.
    - TuringTest 8y ago
      It depends... I've seen pages in Spanish where common specific Spanish characters are shown mangled. It tends to happen on websites with older platforms, but still... It also often happens with pages in Chinese or Japanese. But then you bring another problem: when a document is shown as a list full of ▯▯▯▯▯▯,▯▯-.▯▯▯, there's no way to tell if the problem is with the encoding or the fonts, and no obvious way to know which font should be installed and what part of the software stack is the one responsible. The whole field of displaying documents to end users could benefit from some user-centered design to build tools that helped users to fix these situations, based on some common heuristics to solve the most frequent problems.
TuringTest 8y ago
And apparently I'm not the only one having this problem [1], but I can't find software tools explicitly dedicated to solve this problem at the user layer (though I suppose there will be plenty in IDEs and programming languages). [1] https://softwareengineering.stackexchange.com/questions/187169/how-to-detect-the-encoding-of-a-file https://softwareengineering.stackexchange.com/questions/1871...

majewsky 8y ago

> any document not in the original ASC-127 subset is prone to be mangled for being shown with the wrong character set, wrong linefeed configuration, not having the right BOM, not allowing right-to-left... It got much better in the last years as UTF-8 gained wider adoption. The problems are mostly with legacy systems in government. (Cannot comment on that since my name fits in ASCII.)

TuringTest 8y ago
Maybe for registering to government systems... Using desktop applications is still a nightmare, where opening any document from the internet or navigating to a foreign page is hit or miss. Also, the main frustration is not that it happens often; it's that when you encounter it, fixing it may involve studying the whole software stack to find out where the setting is switched or at what point in the toolchain the format was mishandled.
- TuringTest 8y ago
  And there are non-zero chances that changing to the wrong option and saving the document will permanently break the file as the combination of two wrongly configured codesets. I don't know, maybe it's me who's doing something wrong and turn to using only a small subset of software tools, properly configured? But I do need to process documents from many origins using a variety of different tools; I've never found a good programable multimedia editor that satisfies all my needs (something like emacs but with WYSIWIG capabilities).
- Dylan16807 8y ago
  What kinds of documents are these? Because while sometimes I don't have the right fonts installed, never in my life have I seen a foreign web page fail to decode. The only mojibake I've seen is text that was corrupted during posting, not even showing up right to the author. Documents that people make have always worked as intended.
  - TuringTest 8y ago
    It depends... I've seen pages in Spanish where common specific Spanish characters are shown mangled. It tends to happen on websites with older platforms, but still... It also often happens with pages in Chinese or Japanese. But then you bring another problem: when a document is shown as a list full of ▯▯▯▯▯▯,▯▯-.▯▯▯, there's no way to tell if the problem is with the encoding or the fonts, and no obvious way to know which font should be installed and what part of the software stack is the one responsible. The whole field of displaying documents to end users could benefit from some user-centered design to build tools that helped users to fix these situations, based on some common heuristics to solve the most frequent problems.

TuringTest 8y ago

And apparently I'm not the only one having this problem [1], but I can't find software tools explicitly dedicated to solve this problem at the user layer (though I suppose there will be plenty in IDEs and programming languages). [1] https://softwareengineering.stackexchange.com/questions/187169/how-to-detect-the-encoding-of-a-file https://softwareengineering.stackexchange.com/questions/1871...