3 ms·

Chrome embeds a small LLM (never stops being a funny thing) in the browser allowing them to do local translations. I assume every browser will do the same as o

Chrome embeds a small LLM (never stops being a funny thing) in the browser allowing them to do local translations. I assume every browser will do the same as on-device models start becoming more useful.

rhabarba 1y ago
While I appreciate the on-device approach for a couple of reasons, it is rather ironic that Mozilla needs to document that for them.
- its-summertime 1y ago
  Firefox also has on-device translations for what its worth.
Asraelite 1y ago
What's the easiest way to get this functionality outside of the browser, e.g. as a CLI tool? Last time I looked I wasn't able to find any easy to run models that supported more than a handful of languages.
- ukuina 1y ago
  ollama run gemma3:1b https://ollama.com/library/gemma3 https://ollama.com/library/gemma3 > support for over 140 languages
  - diggan 1y ago
    Try to translate a paragraph with 1b gemma and compare it to DeepL :) Still amazing it can understand anything at all at that scale, but can't really rely on it for much tbh
- _1 1y ago
  If you need to support several languages, you're going to have to have a zoo of models. Small ones just can't handle that many; and they especially aren't good enough for distribution, we only use them for understanding.
- JimDabell 1y ago
  That depends on what counts as “a handful of languages” for you. You can use llm for this fairly easily: uv tool install llm # Set up your model however you like. For instance: llm install llm-ollama ollama pull mistral-small3.2 llm --model mistral-small3.2 --system "Translate to English, no other output" --save english alias english="llm --template english" english "Bonjour" english "Hola" english "Γειά σου" english "你好" cat some_file.txt | english https://llm.datasette.io https://llm.datasette.io
  - usagisushi 1y ago
    Tip: You might want to use `uv tool install llm --with llm-ollama`. ref: https://github.com/simonw/llm/issues/575 https://github.com/simonw/llm/issues/575
    - JimDabell 1y ago
      Thanks!
  - jan_Sate 1y ago
    That's just the base/stock/instruct model for general use case. There gotta be a finetune specialized in translation, right? Any recommendations for that? Plus, mistral-small3.2 has too many parameters. Not all devices can run it fast. That probably isn't the exact translation model being used by Chrome.
    - JimDabell 1y ago
      I haven’t tried it myself, but NLLB-200 has various sizes going down to 600M params: https://github.com/facebookresearch/fairseq/tree/nllb/ https://github.com/facebookresearch/fairseq/tree/nllb/ If running locally is too difficult, you can use llm to access hosted models too.
- wittjeff 1y ago
  https://ai.meta.com/blog/nllb-200-high-quality-machine-translation/ https://ai.meta.com/blog/nllb-200-high-quality-machine-trans... https://www.youtube.com/watch?v=AGgzRE3TlvU https://www.youtube.com/watch?v=AGgzRE3TlvU
- deivid 1y ago
  You can use bergamot ( https://github.com/browsermt/bergamot-translator https://github.com/browsermt/bergamot-translator ) with Mozilla's models ( https://github.com/mozilla/firefox-translations-models https://github.com/mozilla/firefox-translations-models ). Not the easiest, but easy enough (requires building). I used these two projects to build an on-device translator for Android.
- mftrhu 1y ago
  Setting aside general-purpose LLMs, there exist a handful of models geared towards translation between hundred of language pairs: Meta's NLLB-200 [0] and M2M-100 [1] can be run using HuggingFace's transformers (plus numpy and sentencepieces), while Google's MADLAD-400 [2], in GGUF format [3], is also supported by llama.cpp. You could also look into Argos Translate, or just use the same models as Firefox through kotki [4]. [0] https://huggingface.co/facebook/nllb-200-distilled-600M https://huggingface.co/facebook/nllb-200-distilled-600M [1] https://huggingface.co/facebook/m2m100_418M https://huggingface.co/facebook/m2m100_418M [2] https://huggingface.co/google/madlad400-3b-mt https://huggingface.co/google/madlad400-3b-mt [3] https://huggingface.co/models?other=base_model:quantized:google/madlad400-3b-mt https://huggingface.co/models?other=base_model:quantized:goo... [4] https://github.com/kroketio/kotki https://github.com/kroketio/kotki

rhabarba 1y ago

While I appreciate the on-device approach for a couple of reasons, it is rather ironic that Mozilla needs to document that for them.

its-summertime 1y ago
Firefox also has on-device translations for what its worth.

Asraelite 1y ago

What's the easiest way to get this functionality outside of the browser, e.g. as a CLI tool? Last time I looked I wasn't able to find any easy to run models that supported more than a handful of languages.

ukuina 1y ago
ollama run gemma3:1b https://ollama.com/library/gemma3 https://ollama.com/library/gemma3 > support for over 140 languages
- diggan 1y ago
  Try to translate a paragraph with 1b gemma and compare it to DeepL :) Still amazing it can understand anything at all at that scale, but can't really rely on it for much tbh
_1 1y ago
If you need to support several languages, you're going to have to have a zoo of models. Small ones just can't handle that many; and they especially aren't good enough for distribution, we only use them for understanding.
JimDabell 1y ago
That depends on what counts as “a handful of languages” for you. You can use llm for this fairly easily: uv tool install llm # Set up your model however you like. For instance: llm install llm-ollama ollama pull mistral-small3.2 llm --model mistral-small3.2 --system "Translate to English, no other output" --save english alias english="llm --template english" english "Bonjour" english "Hola" english "Γειά σου" english "你好" cat some_file.txt | english https://llm.datasette.io https://llm.datasette.io
- usagisushi 1y ago
  Tip: You might want to use `uv tool install llm --with llm-ollama`. ref: https://github.com/simonw/llm/issues/575 https://github.com/simonw/llm/issues/575
  - JimDabell 1y ago
    Thanks!
- jan_Sate 1y ago
  That's just the base/stock/instruct model for general use case. There gotta be a finetune specialized in translation, right? Any recommendations for that? Plus, mistral-small3.2 has too many parameters. Not all devices can run it fast. That probably isn't the exact translation model being used by Chrome.
  - JimDabell 1y ago
    I haven’t tried it myself, but NLLB-200 has various sizes going down to 600M params: https://github.com/facebookresearch/fairseq/tree/nllb/ https://github.com/facebookresearch/fairseq/tree/nllb/ If running locally is too difficult, you can use llm to access hosted models too.
wittjeff 1y ago
https://ai.meta.com/blog/nllb-200-high-quality-machine-translation/ https://ai.meta.com/blog/nllb-200-high-quality-machine-trans... https://www.youtube.com/watch?v=AGgzRE3TlvU https://www.youtube.com/watch?v=AGgzRE3TlvU
deivid 1y ago
You can use bergamot ( https://github.com/browsermt/bergamot-translator https://github.com/browsermt/bergamot-translator ) with Mozilla's models ( https://github.com/mozilla/firefox-translations-models https://github.com/mozilla/firefox-translations-models ). Not the easiest, but easy enough (requires building). I used these two projects to build an on-device translator for Android.
mftrhu 1y ago
Setting aside general-purpose LLMs, there exist a handful of models geared towards translation between hundred of language pairs: Meta's NLLB-200 [0] and M2M-100 [1] can be run using HuggingFace's transformers (plus numpy and sentencepieces), while Google's MADLAD-400 [2], in GGUF format [3], is also supported by llama.cpp. You could also look into Argos Translate, or just use the same models as Firefox through kotki [4]. [0] https://huggingface.co/facebook/nllb-200-distilled-600M https://huggingface.co/facebook/nllb-200-distilled-600M [1] https://huggingface.co/facebook/m2m100_418M https://huggingface.co/facebook/m2m100_418M [2] https://huggingface.co/google/madlad400-3b-mt https://huggingface.co/google/madlad400-3b-mt [3] https://huggingface.co/models?other=base_model:quantized:google/madlad400-3b-mt https://huggingface.co/models?other=base_model:quantized:goo... [4] https://github.com/kroketio/kotki https://github.com/kroketio/kotki