4 ms·

Here be dragons. Be afraid. Be very afraid.

stcredzero 17y ago
I should start an 'ocalypse of the week site.
patio11 17y ago
Everything you say and more. Internationalization, ick -- anyone who thinks this is easy has no clue how deep this rabbit hole goes. On Han unification: the Japanese reluctance to this is partly because they're being told "Some of your national literature needs to die so that our data standard can live. Deal." and partly because they're being told "What's with all the resistance, you xenophobic bastards, get with the effing program already.", generally by people who they perceive as not quite getting the issue. All the educated Americans in the room have read Romeo and Juliet, right? Remember the balcony scene? Remember the world in the balcony scene that you have never heard in any other context? O Romeo, Romeo, Wherefore art thou Romeo? Imagine being told "For technical reasons, we're standardizing computers away from being able to accept 'Wherefore' as input or output. As a workaround, we suggest using "why", or perhaps putting the word in an image file and pasting it in when it is required. Most people don't use "wherefore" anyhow and, if you routinely do, you can modify your editing software to accommodate it, as long as it doesn't have to interface with any other computer ever. Oh, by the way, some other words you know are also going to stop working. It's nothing major. Well, OK, 'Gertrudes' might find it somewhat annoying but we've got a nice selection of names from Aluicious to Xavier and, if all else fails, you can spell it phonetically because your language is capable of that, too, and don't pretend otherwise."
- jsonscripter 17y ago
  They should have an encoding standard for characters that are not in the standard character set. I've see base64 encoded images in CSS for a long time. Sure, it takes up a lot of space, but isn't this the best of both worlds? Of course, the images could be SVG or a sort of typeface definition. It doesn't matter because it's base64 encoded. Of course, I'm not trivializing the problems with internationalization. I'm simply pointing out that there are simple solutions to some of the problems, such as this one. I think the biggest problem with character encoding is getting people to agree on a standard.
  - patio11 17y ago
    I think the biggest problem with character encoding is getting people to agree on a standard. Prior to the Unicode standard springing into being, Japan had relatively little problem with agreeing on standards, by the simple expedient of agreeing on enough of them such that most stakeholders left the table with at least one they liked. People are sort of reluctant to leave these and the tangible benefits they offered, hence the lukewarm reception to Unicode. (If I hear tsk-tsking from any Americans on this point, just try to imagine how much adoption UTF-8 would have in the US if it didn't have the happy-and-totally-not-accidental property of working exactly like ASCII for the language American programmers and legacy systems/documents care about.) Sidenote: I was working on a translation of features of a particular software package today and had to think a little of how to explain mojibake, and why you'd want a servlet container that can detect and correct it, to an American audience. (It's when some program between the user and the server's application layer applies a heuristic incorrectly and results in one or the other getting complete gibberish.) The problem is every bit as fun to deal with as a developer and a user as you would expect it to be.
    - jrockway 17y ago
      To be fair, the Unicode folks totally screwed over the Japanese. Now there are many people who can't even write their name on a computer. Needless to say, these people would rather use the "legacy" character sets that didn't have this problem.
- ars 17y ago
  Do like other languages, and have a glyph represent a letter, and not a word. So take all those old words, and add a prefix character which means the next glyph is an old word. Unicode already has the concept of combining characters (for diacritics usually). Make a "diacritic" character that encodes the regional flavor, or variant of the character.
- ubernostrum 17y ago
  "On Han unification: the Japanese reluctance to this is partly because they're being told "Some of your national literature needs to die so that our data standard can live. Deal." and partly because they're being told "What's with all the resistance, you xenophobic bastards, get with the effing program already.", generally by people who they perceive as not quite getting the issue." As I understand it, the story of Han unification is quite a bit more complex than that, and not nearly so one-sided as you make it out to be. See this article (which seems rather balanced in its treatment) for an example: http://www.jbrowse.com/text/unij.html http://www.jbrowse.com/text/unij.html

I should start an 'ocalypse of the week site.

Everything you say and more. Internationalization, ick -- anyone who thinks this is easy has no clue how deep this rabbit hole goes. On Han unification: the Japanese reluctance to this is partly because they're being told "Some of your national literature needs to die so that our data standard can live. Deal." and partly because they're being told "What's with all the resistance, you xenophobic bastards, get with the effing program already.", generally by people who they perceive as not quite getting the issue. All the educated Americans in the room have read Romeo and Juliet, right? Remember the balcony scene? Remember the world in the balcony scene that you have never heard in any other context? O Romeo, Romeo, Wherefore art thou Romeo? Imagine being told "For technical reasons, we're standardizing computers away from being able to accept 'Wherefore' as input or output. As a workaround, we suggest using "why", or perhaps putting the word in an image file and pasting it in when it is required. Most people don't use "wherefore" anyhow and, if you routinely do, you can modify your editing software to accommodate it, as long as it doesn't have to interface with any other computer ever. Oh, by the way, some other words you know are also going to stop working. It's nothing major. Well, OK, 'Gertrudes' might find it somewhat annoying but we've got a nice selection of names from Aluicious to Xavier and, if all else fails, you can spell it phonetically because your language is capable of that, too, and don't pretend otherwise."

jsonscripter 17y ago
They should have an encoding standard for characters that are not in the standard character set. I've see base64 encoded images in CSS for a long time. Sure, it takes up a lot of space, but isn't this the best of both worlds? Of course, the images could be SVG or a sort of typeface definition. It doesn't matter because it's base64 encoded. Of course, I'm not trivializing the problems with internationalization. I'm simply pointing out that there are simple solutions to some of the problems, such as this one. I think the biggest problem with character encoding is getting people to agree on a standard.
- patio11 17y ago
  I think the biggest problem with character encoding is getting people to agree on a standard. Prior to the Unicode standard springing into being, Japan had relatively little problem with agreeing on standards, by the simple expedient of agreeing on enough of them such that most stakeholders left the table with at least one they liked. People are sort of reluctant to leave these and the tangible benefits they offered, hence the lukewarm reception to Unicode. (If I hear tsk-tsking from any Americans on this point, just try to imagine how much adoption UTF-8 would have in the US if it didn't have the happy-and-totally-not-accidental property of working exactly like ASCII for the language American programmers and legacy systems/documents care about.) Sidenote: I was working on a translation of features of a particular software package today and had to think a little of how to explain mojibake, and why you'd want a servlet container that can detect and correct it, to an American audience. (It's when some program between the user and the server's application layer applies a heuristic incorrectly and results in one or the other getting complete gibberish.) The problem is every bit as fun to deal with as a developer and a user as you would expect it to be.
  - jrockway 17y ago
    To be fair, the Unicode folks totally screwed over the Japanese. Now there are many people who can't even write their name on a computer. Needless to say, these people would rather use the "legacy" character sets that didn't have this problem.
ars 17y ago
Do like other languages, and have a glyph represent a letter, and not a word. So take all those old words, and add a prefix character which means the next glyph is an old word. Unicode already has the concept of combining characters (for diacritics usually). Make a "diacritic" character that encodes the regional flavor, or variant of the character.
ubernostrum 17y ago
"On Han unification: the Japanese reluctance to this is partly because they're being told "Some of your national literature needs to die so that our data standard can live. Deal." and partly because they're being told "What's with all the resistance, you xenophobic bastards, get with the effing program already.", generally by people who they perceive as not quite getting the issue." As I understand it, the story of Han unification is quite a bit more complex than that, and not nearly so one-sided as you make it out to be. See this article (which seems rather balanced in its treatment) for an example: http://www.jbrowse.com/text/unij.html http://www.jbrowse.com/text/unij.html