5 ms·

Show HN: a18n – Automated I18n JavaScript Solution

39 points by FallenMax 6y ago

taphangum 6y ago
Great work.
- FallenMax 6y ago
  Thanks!
falbala 6y ago
Nice work! What's the difference between this and .po/.mo files and the ugettext standard?
- FallenMax 6y ago
  Thank you! I'm not familiar with .po/.mo/ugettext, will take a look and answer later.
- e12e 6y ago
  Thank you for asking this question - it lead me to find this library (and I think the quickstart highlights some differences to how the gnu gettext standard could be leveraged in js/ts vs a "propiatary" solution based on json) : https://ttag.js.org/ https://ttag.js.org/ https://ttag.js.org/docs/quickstart.html https://ttag.js.org/docs/quickstart.html
  - FallenMax 6y ago
    Thanks for feedback! It looks like a decent lib and I would recommend anyone who need richer i18n support (like plural forms) take a look. As for a18n, it's a deliberate choice to use current format (JSON with extracted key/value pair) instead of standard like ICU messageformat. Here are reasons: 1. This lib is built to enable automated i18n workflow for existing projects (~200kloc), with minimal human interfere and impose minimal complexity for build configuration, in this regard current format is working great. 2. This lib is used by both projects, plugins and components in our company, so to keep its footprint down (in case it's not deduped by bundler), it means fewer features (like plural forms, date, list) are supported, and... 3. ...we originally planned to add support for ICU messageformat if these features are desired. But after two years in production, we found the demand is not too strong to justify the cost.
- FallenMax 6y ago
  Update: As far as I know, this should be a subset of the PO format (http://pology.nedohodnik.net/doc/user/en_US/ch-poformat.html#:~:text=The%20PO%20format%20is%20a,of%20what%20is%20being%20translated http://pology.nedohodnik.net/doc/user/en_US/ch-poformat.html....). The file format used by a18n is JSON, where its keys are extracted from code (using `a18n extract`) into one of these formats: - "static text" - "dynamic text with one variable: %s" - "dynamic text with multiple variables: %1, %2, %3" - "#some.text.id" or "xxx#some.text.id" -- user manually assign text an ID with: a18n("xxx", {_: "some.text.id"}) (It worth noting that my company further converts this JSON to Excel before sending to translators)
haydenkshaw 6y ago
Not sure on the name a18n, because it looks like a numeronym, like i18n is, but doesn't seem to be, had me trying to guess what the 18 characters are between a and n. I get it's a18n like Automated18n like AutomatedI18n.
- inopinatus 6y ago
  I naturally assumed, and by naturally I mean egrep '^a.{18}n$' /usr/share/dict/words, that a18n abbreviated anthropomorphization, and so in context that perhaps we'd be helping a recently translated program feel more settled in its new repository.
  - FallenMax 6y ago
    Not an English speaker but this is still awkward: I always think "i18n" expands to 18 characters until I see your comment and checked `"internationalization".length` in console to be 20
    - 1-more 6y ago
      yeah 18 is replacing the middle 18 characters in the word, not including that i and n at the ends. I personally don't like it because it's just an English speaking programmer in-group shibboleth that has bled over to k8s for kubernetes and a16z for Andreessen-Horowitz
- FallenMax 6y ago
  Thanks! Yes, Automation is what actually I thought of the 'a' in a18n :) But it's named as `a18n` for some complicated reasons: 1. It aligns well with translation function name and CLI command I want the translation function to be something: - looks like `i18n` (so user can tell its purpose at a glance) - but NOT `i18n` (so it's unique enough that we can safely assume its role when we parse/manipulate user's code) - also NOT too short like '_', or 'i' (for the same reason above) By these rules, I think `${someChar}18n` is a great fit as the function name, and to simplify things, I just use it as the CLI name and the package name. 2. (the real reason) It's really hard to name a JavaScript i18n lib today If you search for "i18n" in npm, more that 7000 libs pops up. Here is a list of lib names I searched and found taken: - simple-i18n - i18n-simple - s18n - v18n - next-i18n - i18n-next - react-i18n - i18n-react - ...(maybe 20 more?) So when I finally found that `a18n` is not taken? -- I know instantly it's a digital treasure that I have to grab :)
onei 6y ago
From what I can tell, this scans code for strings that can be translated, and then you supply translations from the initial language resulting in a mapping of Chinese to other languages (using Chinese as an example). What happens if you update that initial translation that every other language hooks off? Do you end up with orphans? At the same time, you may not speak all your languages so you may end up with no -equivalent messages too.
- FallenMax 6y ago
  Thank you! This concern is valid. If we change original (let's say) Chinese texts in code, and re-run the CLI (wrap+extract). These happens (in my company's workflow): 1. CLI removes (oldKey -> oldValue) entry from en.json (a18n don't keep stale keys) 2. CLI adds (newKey -> null) entry to en.json 3. Developer commits updated en.json (yeah it's versioned) 4. A script (not in a18n yet) scans en.json, found untranslated text and warns 5. It's up to the developer to decide what to do with missing translation Not too perfect, but I think keeping original text in code instead of "some.text.id" also has its benefit: 1. Devs understand the texts, this brings more context and makes code easier to understand 2. Some huge projects in my company exists long before they need to support i18n, and it is just too expensive to manually migrate from `text` to `code`
yboris 6y ago
Shameless self-promotion hijack: https://github.com/whyboris/JSON-i18n-Editor https://github.com/whyboris/JSON-i18n-Editor A web interface for letting your team translate JSON i18n files - in case you store your i18n in JSON. See the live demo: link on the GitHub page.
- FallenMax 6y ago
  Great job! A multi-user translate platform is essential to the entire workflow!
- FrontAid 6y ago
  Our CMS for JSON files goes into a similar direction: https://frontaid.io/ https://frontaid.io/
- pintxo 6y ago
  Nice. Being a happy customer of https://poeditor.com https://poeditor.com I'd like to point out that there's also a service for this out there.
  - yboris 6y ago
    I tried POEditor -- it has nice features that my tool does not. I'd like to think my tool is a good-enough free alternative for many cases.
zbraniecki 6y ago
Is it doing anything besides of localization? If not, why did you name i18n (internationalization is a superset of localization domain).
lucisferre 6y ago
This is great. As someone who spent a great deal of time working on i18n with JS for my own software the current state of the art is not amazing. A lot of half finished and abandoned projects, many of which don't entirely work or conform to pre-existing standards. Most don't play well with newer web frameworks and those that do tend to be very limited. Finding a good solution for VueJS, which has it's own .vue files was particularly hard. I ended up building on top an existing i18n extract tool and integrating it with the VueJS compiler to extract both HTML and JS translations. What I'd like to see, that I have not seen anywhere (other than a broken and not currently maintained Webpack module) is a plugin for Babel or Typescript compilers that allows the compiler to compile translations into the final output packages and produce one for each language. I think this is preferable to loading translations at runtime and swapping them in.
- FallenMax 6y ago
  Will definitely look into .vue support! But for compile translations (per-language), I'm not so sure if added performance can justify introduced complexity (for users' build/serve workflow).

39 points by FallenMax 6y ago

taphangum 6y ago

Great work.

FallenMax 6y ago
Thanks!

falbala 6y ago

Nice work! What's the difference between this and .po/.mo files and the ugettext standard?

FallenMax 6y ago
Thank you! I'm not familiar with .po/.mo/ugettext, will take a look and answer later.
e12e 6y ago
Thank you for asking this question - it lead me to find this library (and I think the quickstart highlights some differences to how the gnu gettext standard could be leveraged in js/ts vs a "propiatary" solution based on json) : https://ttag.js.org/ https://ttag.js.org/ https://ttag.js.org/docs/quickstart.html https://ttag.js.org/docs/quickstart.html
- FallenMax 6y ago
  Thanks for feedback! It looks like a decent lib and I would recommend anyone who need richer i18n support (like plural forms) take a look. As for a18n, it's a deliberate choice to use current format (JSON with extracted key/value pair) instead of standard like ICU messageformat. Here are reasons: 1. This lib is built to enable automated i18n workflow for existing projects (~200kloc), with minimal human interfere and impose minimal complexity for build configuration, in this regard current format is working great. 2. This lib is used by both projects, plugins and components in our company, so to keep its footprint down (in case it's not deduped by bundler), it means fewer features (like plural forms, date, list) are supported, and... 3. ...we originally planned to add support for ICU messageformat if these features are desired. But after two years in production, we found the demand is not too strong to justify the cost.
FallenMax 6y ago
Update: As far as I know, this should be a subset of the PO format (http://pology.nedohodnik.net/doc/user/en_US/ch-poformat.html#:~:text=The%20PO%20format%20is%20a,of%20what%20is%20being%20translated http://pology.nedohodnik.net/doc/user/en_US/ch-poformat.html....). The file format used by a18n is JSON, where its keys are extracted from code (using `a18n extract`) into one of these formats: - "static text" - "dynamic text with one variable: %s" - "dynamic text with multiple variables: %1, %2, %3" - "#some.text.id" or "xxx#some.text.id" -- user manually assign text an ID with: a18n("xxx", {_: "some.text.id"}) (It worth noting that my company further converts this JSON to Excel before sending to translators)

haydenkshaw 6y ago

Not sure on the name a18n, because it looks like a numeronym, like i18n is, but doesn't seem to be, had me trying to guess what the 18 characters are between a and n. I get it's a18n like Automated18n like AutomatedI18n.

inopinatus 6y ago
I naturally assumed, and by naturally I mean egrep '^a.{18}n$' /usr/share/dict/words, that a18n abbreviated anthropomorphization, and so in context that perhaps we'd be helping a recently translated program feel more settled in its new repository.
- FallenMax 6y ago
  Not an English speaker but this is still awkward: I always think "i18n" expands to 18 characters until I see your comment and checked `"internationalization".length` in console to be 20
  - 1-more 6y ago
    yeah 18 is replacing the middle 18 characters in the word, not including that i and n at the ends. I personally don't like it because it's just an English speaking programmer in-group shibboleth that has bled over to k8s for kubernetes and a16z for Andreessen-Horowitz
FallenMax 6y ago
Thanks! Yes, Automation is what actually I thought of the 'a' in a18n :) But it's named as `a18n` for some complicated reasons: 1. It aligns well with translation function name and CLI command I want the translation function to be something: - looks like `i18n` (so user can tell its purpose at a glance) - but NOT `i18n` (so it's unique enough that we can safely assume its role when we parse/manipulate user's code) - also NOT too short like '_', or 'i' (for the same reason above) By these rules, I think `${someChar}18n` is a great fit as the function name, and to simplify things, I just use it as the CLI name and the package name. 2. (the real reason) It's really hard to name a JavaScript i18n lib today If you search for "i18n" in npm, more that 7000 libs pops up. Here is a list of lib names I searched and found taken: - simple-i18n - i18n-simple - s18n - v18n - next-i18n - i18n-next - react-i18n - i18n-react - ...(maybe 20 more?) So when I finally found that `a18n` is not taken? -- I know instantly it's a digital treasure that I have to grab :)

onei 6y ago

From what I can tell, this scans code for strings that can be translated, and then you supply translations from the initial language resulting in a mapping of Chinese to other languages (using Chinese as an example). What happens if you update that initial translation that every other language hooks off? Do you end up with orphans? At the same time, you may not speak all your languages so you may end up with no -equivalent messages too.

FallenMax 6y ago
Thank you! This concern is valid. If we change original (let's say) Chinese texts in code, and re-run the CLI (wrap+extract). These happens (in my company's workflow): 1. CLI removes (oldKey -> oldValue) entry from en.json (a18n don't keep stale keys) 2. CLI adds (newKey -> null) entry to en.json 3. Developer commits updated en.json (yeah it's versioned) 4. A script (not in a18n yet) scans en.json, found untranslated text and warns 5. It's up to the developer to decide what to do with missing translation Not too perfect, but I think keeping original text in code instead of "some.text.id" also has its benefit: 1. Devs understand the texts, this brings more context and makes code easier to understand 2. Some huge projects in my company exists long before they need to support i18n, and it is just too expensive to manually migrate from `text` to `code`

yboris 6y ago

Shameless self-promotion hijack: https://github.com/whyboris/JSON-i18n-Editor https://github.com/whyboris/JSON-i18n-Editor A web interface for letting your team translate JSON i18n files - in case you store your i18n in JSON. See the live demo: link on the GitHub page.

FallenMax 6y ago
Great job! A multi-user translate platform is essential to the entire workflow!
FrontAid 6y ago
Our CMS for JSON files goes into a similar direction: https://frontaid.io/ https://frontaid.io/
pintxo 6y ago
Nice. Being a happy customer of https://poeditor.com https://poeditor.com I'd like to point out that there's also a service for this out there.
- yboris 6y ago
  I tried POEditor -- it has nice features that my tool does not. I'd like to think my tool is a good-enough free alternative for many cases.

zbraniecki 6y ago

Is it doing anything besides of localization? If not, why did you name i18n (internationalization is a superset of localization domain).

lucisferre 6y ago

This is great. As someone who spent a great deal of time working on i18n with JS for my own software the current state of the art is not amazing. A lot of half finished and abandoned projects, many of which don't entirely work or conform to pre-existing standards. Most don't play well with newer web frameworks and those that do tend to be very limited. Finding a good solution for VueJS, which has it's own .vue files was particularly hard. I ended up building on top an existing i18n extract tool and integrating it with the VueJS compiler to extract both HTML and JS translations. What I'd like to see, that I have not seen anywhere (other than a broken and not currently maintained Webpack module) is a plugin for Babel or Typescript compilers that allows the compiler to compile translations into the final output packages and produce one for each language. I think this is preferable to loading translations at runtime and swapping them in.

FallenMax 6y ago
Will definitely look into .vue support! But for compile translations (per-language), I'm not so sure if added performance can justify introduced complexity (for users' build/serve workflow).