3 ms·

How do Rust projects localize their format strings?

by ridiculous_fish 4y ago

sedatk 4y ago
There's no built-in support for localization AFAIK.
- deleted 4y ago
  [deleted]
tialaramex 4y ago
For a lot of Rust projects they just don't. Unlike a lot of stuff from the 1990s or so, the Rust core itself doesn't care about localization. It speaks Unicode fluently, but it doesn't try to think about locale problems at all. For projects that are user facing and so need to do localization, you do have a few options, including both gettext and ICU at opposite ends of the scale of "How difficult is your problem/ how much work do you want to do?" Gettext is just going to let you substitute translations like "X {1} Y {2}" in one language versus "YorX: {2}or{1}" in another language, whereas ICU understands how to render the Japanese form of today's date and which is the correct plural form for 38 of this thing in Russian (but it's still on you to prepare all the plural variants for each type of thing you want to quantify, ICU just tells you which of them to use)
- ridiculous_fish 4y ago
  That's a depressing answer, that nobody bothers. gettext seems pretty difficult to use in Rust - there's no obvious way to apply a runtime format string "My name is {1}". https://blog.hackeriet.no/rust-and-translation-files/ https://blog.hackeriet.no/rust-and-translation-files/ describes using gettext from Rust but not how to handle format strings.
  - haimez 4y ago
    It seems reasonable to me that this is not a compile time constraint, but rather one of tooling and the deployed environment- ie: is the Japanese translation available here at runtime? How? Is there a runtime configuration flag? Is the translations file available at runtime? Is the date formatter configured appropriately? Maybe if you bake in assumptions about the deployed environment at compile time you can catch errors earlier for your specific environment, but all environments? No. That’s not the responsibility of the standard library and given the language lacks a single specific runtime target- definitely not the responsibility of the language/compiler. This is instead a classic use case for tests and opinionated (optional/third party) libraries instead.
    - ridiculous_fish 4y ago
      I agree that e.g. locating .mo files is out of scope for a stdlib, but runtime format strings seems like something the stdlib could do. Because it does not, std::fmt is borderline useless for software which wants to support multiple languages.
      - haimez 4y ago
        My basic suggestion is that the standard library _should_ be useless at supporting multiple languages, because it’s not a goal that can be achieved for almost anyone’s actual use case. You’re going to have to actually design internationalization into your own codebase in a way that makes sense for your application, and a half-measured implementation just doesn’t have a place at the language core. Format strings, for example- are essentially compile time macros in rust in order to avoid all the fun kinds of string handling bugs that C’s implementation allowed- but that means no runtime customization without also defining were alternate formats live (and how they’re verified, etc). Not supporting multiple languages is a feature, not a bug. If you need multi language support, you need to a library to define those semantics in a way that fits or use case.
        noyoudumbdolt 4y ago
        [dead]
  - tialaramex 4y ago
    Rust's format! macro only wants actual literals, because you get a compile error if the format is wrong, which would not be possible for a non-literal. C++ does this for a constant even if it's not literal, which is fancier, but Rust doesn't have a way to achieve that today. However, you don't have to use format! to format stuff, you would presumably want an API which better reflects the runtime errors you can now encounter, such as wrong number of arguments, wrong order of arguments, incompatible formats.
    - estebank 4y ago
      On top of that, you could provide a `format` macro that shadows the language item and does support gettext or similar (although that might not be true with regards to the f-string support).
      - est31 4y ago
        The f-string support (capturing "{hello}") doesn't need anything more than a macro can do. macro_rules macros can't dream up literals from stuff they computed, but proc macros can. You are probably aware, but there is a currently ongoing project to move the format_args macro further into the compiler (it's a builtin macro right now, but not doing much that a proc macro can't do), to do its work during AST->HIR lowering. On top of that, some optimizations are proposed that would be impossible to implement on macros today. For some of those optimizations, a primitive to allow proc macros to expand macros of their own would make it possible to have with a pure-macro solution. Even just the ability for macros to say that they want their input to be expanded instead of receiving pre-expanded input would be enough. These primitives are not available today, but are possible future extensions.
  - merb 4y ago
    well there is: https://crates.io/crates/gettext-macros https://crates.io/crates/gettext-macros besides the problem, that it's gpl-v3 which is unsuitable for most projects, it's still a working example that it isn't so hard to use.
- est31 4y ago
  > Unlike a lot of stuff from the 1990s or so, the Rust core itself doesn't care about localization. It speaks Unicode fluently, but it doesn't try to think about locale problems at all. The design of Rust's standard library is to be as minimal as possible. Full localization support would be too much for a tiny standard library such as Rust's. Contrast this to Java or Go. In the library ecosystem, there are full implementations of fluent available, and the compiler is being translated. It's not ergonomic to use yet, meaning you have to put the strings into a separate .ftl file (maybe it will never be), but it's quite powerful and developed by a lot of experts on the topic.
- yencabulator 4y ago
  And for anyone who thinks gettext and ICU are just old/brittle/clunky/unsafe things, you may enjoy https://projectfluent.org/ https://projectfluent.org/ https://github.com/projectfluent/fluent-rs https://github.com/projectfluent/fluent-rs
LegionMammal978 4y ago
Personally, I haven't had any issues using the i18n-embed-fl crate [0], with a wrapper over the fl!() macro that passes a lazily-initialized global loader. (I did have to add `loader.set_use_isolating(false)` to my initialization function to keep it from outputting invisible Unicode characters; you only really need it to be `true` if you're planning on outputting RTL text in an LTR context, or vice versa.) [0] https://crates.io/crates/i18n-embed-fl https://crates.io/crates/i18n-embed-fl
- ridiculous_fish 4y ago
  Great tip, I wasn't aware of that crate!
npigrounet 4y ago
[dead]

How do Rust projects localize their format strings?

sedatk 4y ago

There's no built-in support for localization AFAIK.

deleted 4y ago
[deleted]

tialaramex 4y ago

For a lot of Rust projects they just don't. Unlike a lot of stuff from the 1990s or so, the Rust core itself doesn't care about localization. It speaks Unicode fluently, but it doesn't try to think about locale problems at all. For projects that are user facing and so need to do localization, you do have a few options, including both gettext and ICU at opposite ends of the scale of "How difficult is your problem/ how much work do you want to do?" Gettext is just going to let you substitute translations like "X {1} Y {2}" in one language versus "YorX: {2}or{1}" in another language, whereas ICU understands how to render the Japanese form of today's date and which is the correct plural form for 38 of this thing in Russian (but it's still on you to prepare all the plural variants for each type of thing you want to quantify, ICU just tells you which of them to use)

ridiculous_fish 4y ago
That's a depressing answer, that nobody bothers. gettext seems pretty difficult to use in Rust - there's no obvious way to apply a runtime format string "My name is {1}". https://blog.hackeriet.no/rust-and-translation-files/ https://blog.hackeriet.no/rust-and-translation-files/ describes using gettext from Rust but not how to handle format strings.
- haimez 4y ago
  It seems reasonable to me that this is not a compile time constraint, but rather one of tooling and the deployed environment- ie: is the Japanese translation available here at runtime? How? Is there a runtime configuration flag? Is the translations file available at runtime? Is the date formatter configured appropriately? Maybe if you bake in assumptions about the deployed environment at compile time you can catch errors earlier for your specific environment, but all environments? No. That’s not the responsibility of the standard library and given the language lacks a single specific runtime target- definitely not the responsibility of the language/compiler. This is instead a classic use case for tests and opinionated (optional/third party) libraries instead.
  - ridiculous_fish 4y ago
    I agree that e.g. locating .mo files is out of scope for a stdlib, but runtime format strings seems like something the stdlib could do. Because it does not, std::fmt is borderline useless for software which wants to support multiple languages.
    - haimez 4y ago
      My basic suggestion is that the standard library _should_ be useless at supporting multiple languages, because it’s not a goal that can be achieved for almost anyone’s actual use case. You’re going to have to actually design internationalization into your own codebase in a way that makes sense for your application, and a half-measured implementation just doesn’t have a place at the language core. Format strings, for example- are essentially compile time macros in rust in order to avoid all the fun kinds of string handling bugs that C’s implementation allowed- but that means no runtime customization without also defining were alternate formats live (and how they’re verified, etc). Not supporting multiple languages is a feature, not a bug. If you need multi language support, you need to a library to define those semantics in a way that fits or use case.
      - noyoudumbdolt 4y ago
        [dead]
- tialaramex 4y ago
  Rust's format! macro only wants actual literals, because you get a compile error if the format is wrong, which would not be possible for a non-literal. C++ does this for a constant even if it's not literal, which is fancier, but Rust doesn't have a way to achieve that today. However, you don't have to use format! to format stuff, you would presumably want an API which better reflects the runtime errors you can now encounter, such as wrong number of arguments, wrong order of arguments, incompatible formats.
  - estebank 4y ago
    On top of that, you could provide a `format` macro that shadows the language item and does support gettext or similar (although that might not be true with regards to the f-string support).
    - est31 4y ago
      The f-string support (capturing "{hello}") doesn't need anything more than a macro can do. macro_rules macros can't dream up literals from stuff they computed, but proc macros can. You are probably aware, but there is a currently ongoing project to move the format_args macro further into the compiler (it's a builtin macro right now, but not doing much that a proc macro can't do), to do its work during AST->HIR lowering. On top of that, some optimizations are proposed that would be impossible to implement on macros today. For some of those optimizations, a primitive to allow proc macros to expand macros of their own would make it possible to have with a pure-macro solution. Even just the ability for macros to say that they want their input to be expanded instead of receiving pre-expanded input would be enough. These primitives are not available today, but are possible future extensions.
- merb 4y ago
  well there is: https://crates.io/crates/gettext-macros https://crates.io/crates/gettext-macros besides the problem, that it's gpl-v3 which is unsuitable for most projects, it's still a working example that it isn't so hard to use.
est31 4y ago
> Unlike a lot of stuff from the 1990s or so, the Rust core itself doesn't care about localization. It speaks Unicode fluently, but it doesn't try to think about locale problems at all. The design of Rust's standard library is to be as minimal as possible. Full localization support would be too much for a tiny standard library such as Rust's. Contrast this to Java or Go. In the library ecosystem, there are full implementations of fluent available, and the compiler is being translated. It's not ergonomic to use yet, meaning you have to put the strings into a separate .ftl file (maybe it will never be), but it's quite powerful and developed by a lot of experts on the topic.
yencabulator 4y ago
And for anyone who thinks gettext and ICU are just old/brittle/clunky/unsafe things, you may enjoy https://projectfluent.org/ https://projectfluent.org/ https://github.com/projectfluent/fluent-rs https://github.com/projectfluent/fluent-rs

LegionMammal978 4y ago

Personally, I haven't had any issues using the i18n-embed-fl crate [0], with a wrapper over the fl!() macro that passes a lazily-initialized global loader. (I did have to add `loader.set_use_isolating(false)` to my initialization function to keep it from outputting invisible Unicode characters; you only really need it to be `true` if you're planning on outputting RTL text in an LTR context, or vice versa.) [0] https://crates.io/crates/i18n-embed-fl https://crates.io/crates/i18n-embed-fl

ridiculous_fish 4y ago
Great tip, I wasn't aware of that crate!

npigrounet 4y ago

[dead]