5 ms·

My companies product supports about 35 languages and 25 countries. I think 'just use Unicode' is fairly accurate. The problem may be how you go about doing it.

My companies product supports about 35 languages and 25 countries. I think 'just use Unicode' is fairly accurate. The problem may be how you go about doing it. I'm reminded of a professor that used to tell his students to use libraries and code written by people with an expertise in the relevant area whenever possible. We use the IBM ICU library for our product and it handles the vast majority of the issues presented in the article. There is no reason for me to be writing the code to handle the thousands of permutations of language, country, and culture when someone else has already done the heavy lifting here. ICU solves problems such as: sorting, number representation, etc. The other problems should be things unrelated to Unicode - things your product / business already need. 1)Pick a target market - If you haven't picked a target market worrying about using Unicode to support some languages is putting the cart before the horse. 2)Get a translator(s) that know what they are doing - You already needed a translator - now you need to pick one competent enough to know the difference between Mexican Spanish and the Spanish spoken in Spain. 3) Write software that can switch look and feel on the fly during run time - This has already be done and should be a good philosophy for companies writing GUI intensive applications anyway. Really - The correct sentence should be, "Just use Unicode, with your pre-existing good business practices."

philfreo 17y ago
This may work well for 35 languages. Not 350 or 3500.
- tseabrooks 17y ago
  Why doesn't this scale to any number of languages? Assuming you are using the existing practice of not hard coding strings in your software but rather referring to them with some sort of IDs then I see no reason not to use 3000 languages with this approach.
  - seabee 17y ago
    There's more to it than language replacement - there are cultural differences and representation of numbers, quantities etc., RTL support, allowing users to input their data in more than one language and have it behave in a reasonable manner... But honestly, out of all the problems you can have, that's a good one indicative of success.
    - psranga 17y ago
      I did a quick scan of the ICU library mentioned above (thanks for that!!) and it does handle these things. http://userguide.icu-project.org/i18n http://userguide.icu-project.org/i18n
  - lucumo 17y ago
    We are using IDs on our website instead of hard-coded variables. It causes no end of problems. A non-exhaustive list: 1. Using variables to put numbers in the string. Singular and plural are not always only two options. Rules in this regard can get quite complex. gettext has this solved by using rules: n%2 == 0? Use A. n==0? Use B. n%2 == 1? Use C. etc. That should give you an idea about complexity. 2. Replacing words into the string is nearly impossible due to declensions. 3. Names. When using a different script, will the name be transliterated? Do the names need to be inflected? Will they be handled correctly when changing writing direction? 3a. Names have a whole load of problems anyway. The "given name" isn't always the first name. The surname isn't always the most important one. Some people don't have family names, some people have two or more. Some people have double-barrelled given names. Etc. 4. Ordinals differ greatly from language to language. 5. String length can mess up your layout. A good translator can help by using different idiom, but sometimes it's just not possible. 6. Layout expectations are different when the writing system is right-to-left. You also need to learn the details of Unicode's direction algorithm, because you will encounter situations where it doesn't work properly without guidance. These are just some problems we've actually encountered when using multiple languages.
    - tseabrooks 17y ago
      Disclaimer: I'm don't do any web software at all, I work on embedded devices. We use IDs and I've not seen too many of the problems you mention. I will, however, agree that the string length can cause serious problems for a layout. We tend towards using very flexible UI designs where the UI elements resize and reposition themselves a long with the strings. However, UI design isn't as stylish for embedded devices so we get let off easy here. One of the biggest problems we have with our newer developers is that they don't know how to write their UIs so that they can easily re-arrange themselves for our various languages. Also, the Left and Right layout changes are all part of our look and feel engine. I wonder if some of these problems are specific to web applications and some of the 'fancier' UI designs that we don't use? I don't actually know enough about the differences between the two to say.
    - oikujhgfvg 17y ago
      You don't even need to go outside English for some of these to be a problem, What letter does the name Van der Waals sort under? What about D'Arcy Does McKay come before MacKay ? Is 7-4-09 an American holiday or a week after April fools?
  - philfreo 17y ago
    Besides several of the reasons mentioned in the original article (cultural differences of color and symbols, left to right / top to bottom writing systems, etc.) In many non-Roman scripts the letters change form drastically based on if they are at the beginning, middle, or end. In some, the letters literally switch positions with other letters visually. Current operating system font support, Unicode standards, and fonts simply don't exist to support all of this. Graphite (http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&cat_id=RenderingGraphite http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&#38...) is one great initiative to get the ball moving here and is gaining some support (being included as part of OpenOffice 3), however there are HUGE steps that have not yet been taken before applications can even think about scaling to 3000 languages. Unicode is great, but it's not a simple solution/answer/cure-all. Hopefully we'll be able to scale that large one day, but luckily for now we only need to target < 1% of the world's languages.
- Hoff 17y ago
  As is typical in the software universe, you must target which subset of folks are central to the profits of your business. How many customers will you have past your particular and local top 35 languages? Enough to warrant the translation and support and testing work inherent in adding another language? And yes, the target market for and the list of languages used by your customer base(s) may or will differ from mine, too.
psranga 17y ago
Great information; thanks!!! The ICU library looks very comprehensive, but I hope it's not too bureaucratic to use. :)

philfreo 17y ago

This may work well for 35 languages. Not 350 or 3500.

tseabrooks 17y ago
Why doesn't this scale to any number of languages? Assuming you are using the existing practice of not hard coding strings in your software but rather referring to them with some sort of IDs then I see no reason not to use 3000 languages with this approach.
- seabee 17y ago
  There's more to it than language replacement - there are cultural differences and representation of numbers, quantities etc., RTL support, allowing users to input their data in more than one language and have it behave in a reasonable manner... But honestly, out of all the problems you can have, that's a good one indicative of success.
  - psranga 17y ago
    I did a quick scan of the ICU library mentioned above (thanks for that!!) and it does handle these things. http://userguide.icu-project.org/i18n http://userguide.icu-project.org/i18n
- lucumo 17y ago
  We are using IDs on our website instead of hard-coded variables. It causes no end of problems. A non-exhaustive list: 1. Using variables to put numbers in the string. Singular and plural are not always only two options. Rules in this regard can get quite complex. gettext has this solved by using rules: n%2 == 0? Use A. n==0? Use B. n%2 == 1? Use C. etc. That should give you an idea about complexity. 2. Replacing words into the string is nearly impossible due to declensions. 3. Names. When using a different script, will the name be transliterated? Do the names need to be inflected? Will they be handled correctly when changing writing direction? 3a. Names have a whole load of problems anyway. The "given name" isn't always the first name. The surname isn't always the most important one. Some people don't have family names, some people have two or more. Some people have double-barrelled given names. Etc. 4. Ordinals differ greatly from language to language. 5. String length can mess up your layout. A good translator can help by using different idiom, but sometimes it's just not possible. 6. Layout expectations are different when the writing system is right-to-left. You also need to learn the details of Unicode's direction algorithm, because you will encounter situations where it doesn't work properly without guidance. These are just some problems we've actually encountered when using multiple languages.
  - tseabrooks 17y ago
    Disclaimer: I'm don't do any web software at all, I work on embedded devices. We use IDs and I've not seen too many of the problems you mention. I will, however, agree that the string length can cause serious problems for a layout. We tend towards using very flexible UI designs where the UI elements resize and reposition themselves a long with the strings. However, UI design isn't as stylish for embedded devices so we get let off easy here. One of the biggest problems we have with our newer developers is that they don't know how to write their UIs so that they can easily re-arrange themselves for our various languages. Also, the Left and Right layout changes are all part of our look and feel engine. I wonder if some of these problems are specific to web applications and some of the 'fancier' UI designs that we don't use? I don't actually know enough about the differences between the two to say.
  - oikujhgfvg 17y ago
    You don't even need to go outside English for some of these to be a problem, What letter does the name Van der Waals sort under? What about D'Arcy Does McKay come before MacKay ? Is 7-4-09 an American holiday or a week after April fools?
- philfreo 17y ago
  Besides several of the reasons mentioned in the original article (cultural differences of color and symbols, left to right / top to bottom writing systems, etc.) In many non-Roman scripts the letters change form drastically based on if they are at the beginning, middle, or end. In some, the letters literally switch positions with other letters visually. Current operating system font support, Unicode standards, and fonts simply don't exist to support all of this. Graphite (http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&cat_id=RenderingGraphite http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&#38...) is one great initiative to get the ball moving here and is gaining some support (being included as part of OpenOffice 3), however there are HUGE steps that have not yet been taken before applications can even think about scaling to 3000 languages. Unicode is great, but it's not a simple solution/answer/cure-all. Hopefully we'll be able to scale that large one day, but luckily for now we only need to target < 1% of the world's languages.
Hoff 17y ago
As is typical in the software universe, you must target which subset of folks are central to the profits of your business. How many customers will you have past your particular and local top 35 languages? Enough to warrant the translation and support and testing work inherent in adding another language? And yes, the target market for and the list of languages used by your customer base(s) may or will differ from mine, too.

psranga 17y ago

Great information; thanks!!! The ICU library looks very comprehensive, but I hope it's not too bureaucratic to use. :)