6 ms·
MessageFormat: Unicode standard for localizable message strings
- rocqua 8mo agoThis seems great in concept, and totally infeasible. But if anyone can do it, unicode seems like a great candidate. Does anyone have reason for more optimism?
- hobofan 8mo agoCare to explain why you think it's infeasible? Then one could provide targeted counter-optimism ;) I don't see what's infeasible about it. It doesn't seem too different from .po files (gettext catalogs) meshed with hooks for post-processing as would see in e.g. a handlebars, both of which have individually found great adoption.
- bmn__ 8mo ago> why you think it's infeasible? GP based his opinion on the assumption that this spec new and no implementations for it exist.
- zbraniecki 8mo agoICU4C and ICU4J have implementations. We also have a JS polyfill and will be working on ICU4X impl this quarter.
- junon 8mo agoUnicode consortium already manages a ton of language specs. If there's any group of folks I'd trust to understand languages (natural or otherwise), it's them.
- Cthulhu_ 8mo agoThis is the one. Think of all the "misconceptions developer have about X" lists, I trust Unicode to have encountered (if not written) all of them. The people behind unicode are thorough. I mean they have hieroglyphs, some of which have plurals: https://www.unicode.org/charts/nameslist/n_13000.html https://www.unicode.org/charts/nameslist/n_13000.html
- tuyiown 8mo agoI've been using this format for almost 10 years, and I only see increasing adoption. Why would I be pessimistic?
- BoppreH 8mo agoThe meeting notes in the repo was a nice surprise. Overall looked great, striking a good balance. .input {$var :number maximumFractionDigits=0} .local $var2 = {$var :number maximumFractionDigits=2} .match $var2 0 {{The selector can apply a different function to {$var} for the purposes of selection}} * {{A placeholder in a pattern can apply a different function to {$var :number maximumFractionDigits=3}}} Oof, that's a programming language already. And new syntax to be inevitably iterated on. I feel like we have too many of those already, from Python f-strings to template engines. I wish it'll at least stay small: no nesting, no plugins, no looping, no operators, no side effects or calls to external functions (see Log4J).
- silvestrov 8mo agoEnglish has just singular and plural: one car, two cars, three cars (and zero cars). Some languages have more variations. E.g. Czech, Slovene and Russian has 1, 2-4 and 5 as different cases. Personally I think the syntax is too brittle. It looks too much like TeX code and it has the lisp like deal with lines ending with too many } braces. I would separate it into two cases: simple strings with just simple interpolation and then a more fuller markup language, more like a simplified xml. There are more example code at https://github.com/unicode-org/message-format-wg/blob/main/docs/tech-preview-blog-post.md https://github.com/unicode-org/message-format-wg/blob/main/d...
- BoppreH 8mo agoOh, the language aspect gets a lot worse than that. They explicitly have a non-goal of "all grammatical features of all languages", but the "common" cases are hard enough. From https://github.com/unicode-org/message-format-wg/blob/main/spec/syntax.md https://github.com/unicode-org/message-format-wg/blob/main/s... : .local $hasCase = {$userName :ns:hasCase} .match $hasCase vocative {{Hello, {$userName :ns:person case=vocative}!}} accusative {{Please welcome {$userName :ns:person case=accusative}!}} * {{Hello!}} But if anyone can find a good compromise, it's the Unicode team.
- Cthulhu_ 8mo agoIt looks more like a DSL than configuration, but then given what I've learned about localization that's probably necessary in some cases! However, ideally / in most cases it isn't.
- 1116574 8mo agoLooks alot like mozilla's project fluent, atleast in the basic use case. https://projectfluent.org/ https://projectfluent.org/ I wonder why it hasn't been adopted more widely.
- xnorswap 8mo agoI often wonder this myself, this really should be a standard by now.
- hobofan 8mo agoI can't speak for the status quo, but for at least the first ~5 years (so until 3 years ago when I last attempted to use it), the JS implementation of Fluent was a mess. Constant issues with incomplete API, wrong TS typings (which at that point were external) and build/bundling issues to the point where we opted for a homebrew solution. I imagine that I probably wasn't the only one driven away by that (and I gave it many attempts!).
- creshal 8mo agoThe standard is, for better or worse, gettext; it's good enough that any attempt to replace it runs into the problem that people can't agree on how much better an alternative needs to be to be worth migrating to; so you get a constant churn that so far hasn't seen any clear winner.
- Cthulhu_ 8mo agoFeels like it's That XKCD page; there were standards like gettext, then web development came along and a load of people (...present company included) rediscovered localization and pluralization through trial, error, half-building one's own localization library, then the JS world reinvented it, etc etc etc.
- zbraniecki 8mo agoWe are targeting MF2.0 for inclusion in JavaEcript stdlib (ECMA-402). And later maybe with its own format into DOM for DOM L10n.
- 8mo ago
- bmn__ 8mo agoLooking for an expert who knows both libintl/Gettext and MessageFormat. What is the equivalent of xgettext.pl, the file extension for the main catalog file `.po`, the __ function? How does gender work (small example)? How does layering pt_BR on pt_PT work? What is a compelling reason to switch?
- zbraniecki 8mo agohttps://messageformat.unicode.org/ https://messageformat.unicode.org/ Lmk if you have further questions!
- bmn__ 8mo agoThe site behind that link gives answers to only 2 out of 6 question. If your goal was to promote and teach, then you have failed. If your goal was to demoralise the HN readers and grind the conversation to a stop, then you have succeeded.
- zbraniecki 8mo agoDefinitely the former, apologies for making it confusing. > What is the equivalent of xgettext.pl There is no standard one, although people build their own. The general consensus is that source strings should not be inlined into code. The closest analogy is to "style" vs "class" in HTML/CSS - the clean separation of concerns comes from the "id" being the contract. You can read more about it here: https://github.com/projectfluent/fluent/wiki/Fluent-vs-gettext https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette... There are attempts to "merge" those two philosophies, by extracting and "generating" slugs as ids. Examples: - https://formatjs.github.io/docs/getting-started/message-extraction/ https://formatjs.github.io/docs/getting-started/message-extr... - https://lingui.dev/guides/message-extraction https://lingui.dev/guides/message-extraction - https://app.studyraid.com/en/read/15768/550728/setting-up-the-message-extraction-pipeline https://app.studyraid.com/en/read/15768/550728/setting-up-th... I'm fairly skeptical of this approach. > the file extension for the main catalog file `.po` In MF1.0 world, the file format is JSON or XML. You encode id=>Message pairs. In Fluent world there is a Fluent (FTL) file format. In MF2.0 the format itself is, again, message scoped. On top of it there's a proposal by Mozilla to create MessageResource - https://github.com/w3c/i18n-discuss/blob/gh-pages/explainers/message-resources.md https://github.com/w3c/i18n-discuss/blob/gh-pages/explainers... and that may feed into DOM L10n - https://github.com/mozilla/explainers/blob/main/dom-localization.md https://github.com/mozilla/explainers/blob/main/dom-localiza... > the __ function? see the (1) and links to "generated ids". > How does gender work (small example)? MF 1.0: ``` {GENDER, select, male {He answered} female {She answered} other {They answered} } ``` Fluent: ``` user-answered = { $gender -> [male] He answered. [female] She answered. *[other] They answered. } ``` > How does layering pt_BR on pt_PT work? MF does not prescribe fallback behavior. It also more popular to treat each locale as "complete" and fill "gaps" at build time. So at runtime you have `pt-BR` which has pt-BR strings and missing ones "completed" from `pt` (parent locale). Fluent has a "resource manager" (simple one like this: https://github.com/projectfluent/fluent-rs/tree/main/fluent-resmgr https://github.com/projectfluent/fluent-rs/tree/main/fluent-... or more complex like Mozilla L10nRegistry), which can fallback at runtime, allowing for what we call "partial locales" which can roll out to production with gaps and the resource manager will fetch the fallback strings from the parent locale. > What is a compelling reason to switch? If you and your users are happy with gettext, none! If either of those groups complain, there may be many: - https://github.com/projectfluent/fluent/wiki/Fluent-vs-gettext https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette... Hope that helps!
- jp1016 8mo agoOne practical thing I appreciated about MessageFormat is how it eliminates a bunch of conditional UI logic. I used to write switch/if blocks for: • 0 rows → “No results” • 1 row → “1 result” • n rows → “{n} results” Which seems trivial in English, but gets messy once you support languages with multiple plural categories. I wasn’t really aware of how nuanced plural rules are until I dug into ICU. The syntax looked intimidating at first, but it actually removes a lot of branching from application code. I’ve been using an online ICU message editor (https://intlpull.com/tools/icu-message-editor https://intlpull.com/tools/icu-message-editor) to experiment with plural/select cases and different locales helped me understand edge cases much faster than reading the spec alone.
- pferde 8mo agoDid not gettext have this for decades? https://www.gnu.org/software/gettext/manual/html_node/Plural-forms.html https://www.gnu.org/software/gettext/manual/html_node/Plural...
- Muromec 8mo agoGettext has everything, it just takes knowing five languages to understand what to use for
- Sharlin 8mo agoYeah, some sort of pluralization support is pretty much the second most important feature in any message localization tool, right after the ability to substitute externally-defined strings in the first place. Even in a monolingual application, spamming plural formatting logic in application code isn't exactly the best practice.
- iririririr 8mo agogettext have everything, plus a huge ecosystem like tools to coordinate collaboration from thousand of contributors etc. if alternatives don't start with a very strong case why gettext wasn't a good option, it's already a good indicator of not-invented-here syndrome.
- strogonoff 8mo agoDoes anyone know the ETA of MessageFormat 2.0? I am aware of the effort since pre-COVID times. I recall that some of the developers behind Mozilla Fluent have been among the people working on MF 2.0, and it’d be great to know whether Fluent and ICU MF are going to be interoperable in foreseeable future.
- Vinnl 8mo agoIIRC, the goal was for Fluent to have a convertor or something to be able to work with MessageFormat 2.0, but I don't quite remember where I heard that. My approach has just been to stick to Fluent for now.
- zbraniecki 8mo agoYep. Mozilla is planning an auto converter from Fluent to MF2.0 once we stabilize it.
- strogonoff 8mo agoIt is great to hear a confirmation, though the core of the question was more about when is that roughly forecast to happen rather than if. :)
- revetkn 8mo agoMy project Lokalized attempts to solve many of these complex plural/gender/ordinal/etc. rules with a tiny expression language: https://lokalized.com https://lokalized.com
- frizlab 8mo agoSame here (linked to a test because I don’t have a (meaningful) readme…) That being said your project looks very cool! https://github.com/Frizlab/XibLoc/blob/e85a5179bdd93e017473182d76b33af1f176e6bd/Tests/XibLocTests/XibLocTestsSwiftStr.swift#L286 https://github.com/Frizlab/XibLoc/blob/e85a5179bdd93e0174731...
- Cthulhu_ 8mo agoAre there any formal test suites to check and compare the various localization libraries with each other? There's a lot of languages and language specific rules and exceptions to consider, after all.
- deleted 8mo ago[deleted]
- Brosper 8mo agoI discovered it working in https://tolgee.io https://tolgee.io but I am kind of surprised it boomed today :D What I can say that it's a well-maintained format but also kinda hard to learn.
- alexchamberlain 8mo agoApologies if this is obvious and I missed it. Does this define a way to store the strings in various languages?
- Cthulhu_ 8mo agoI think this is just the format and specification itself, language selection and file storage and the like will depend on an implementing library. The i18next version for example (bizarrely) puts the whole string in a JSON key, but to be honest I think this is a bad example: https://github.com/i18next/i18next-icu?tab=readme-ov-file#more-complete-sample https://github.com/i18next/i18next-icu?tab=readme-ov-file#mo...
- zbraniecki 8mo agoHere is a proposal for a message resource format on top of MF2.0 - https://github.com/eemeli/message-resource-wg https://github.com/eemeli/message-resource-wg
- deleted 8mo ago[deleted]
- hackingonempty 8mo agoI know these libs are primarily for devs to localize their apps but can they be used also with untrusted inputs, both message strings and vars?
- ddevnyc 8mo agoOne thing I would really appreciate in this repository (and many like it) would be a simple, short, snippet of code that shows a typical use case of whatever the repo is selling me. Life's too short to dig around in the guts of the repository to find stuff like this out, it should be front and center. I want to know about the ergonomics and hackability of what I'm about to delve into.
- kccqzy 8mo agoYou are looking for the marketing page not the GitHub page then: https://messageformat.unicode.org/ https://messageformat.unicode.org/