jelmervdl

7 ms·

1.
by jelmervdl 4y ago
You're pretty much right on the money. For ParaCrawl[1] (which I worked on) we used fast machine translation systems that were "good enough" to translate one side of each pair to the language of the other, see whether they&#x
2.
by jelmervdl 4y ago
We can already. Almost. The translator works perfectly* as just a web page[1]. It is only a matter of combining that bit with the full-page translation code[2] and some UI to toggle it. It is a bit of a question whether this is the way to g
3.
by jelmervdl 4y ago
The datasets these models are trained on are sentence pairs. So even if just a couple of sentences between two wikipedia sites are direct translations of each other, they will have appeared in the training set. They don’t have to have appea
4.
by jelmervdl 4y ago
That must be a bug. There is a lot of code in the extension specifically to make it track DOM changes and push translations for those as they happen. React pages are especially tricky to translate as the framework does not take kindly on ex
5.
by jelmervdl 4y ago
It is running a full transformer type neural network on your cpu, it needs all the speed it can get. It’s because the translation engine requires at least SSSE3.1 instructions[1]. These are translated to wasm SIMD instructions[2] which are
6.
by jelmervdl 4y ago
I don’t know about Disney, but in (European centred) machine translation we often pick Icelandic and Maltese as first languages to do specifically because they’re small but familiar enough as tests for the pipeline.
7.
by jelmervdl 4y ago
I think doing machine translation in the cloud makes more sense commercially (easier to monetise) and is a lot less challenging (better hardware, model sizes less of an issue, fewer worries about exposing proprietary code or models). But I
8.
by jelmervdl 4y ago
https://translatelocally.com/ The website isn’t pretty but it’s the same tech as the browser extension. It came out of the same research project. But because its optimised native code, and not limited by single threaded was
9.
by jelmervdl 4y ago
It might be because Firefox Translate hooks into some private extension apis (for browser chrome and telemetry) that you can only use when your extension is signed by Mozilla itself. Maybe those apis have been removed from Librewolf, or it
10.
by jelmervdl 4y ago
To be fair, Wikipedia basically appears in its entirety in the training data. It’s a good test to see whether the translation model and all the plumbing works well, but not whether the model generalises well.
11.
by jelmervdl 4y ago
I'm not aware of any actively maintained projects that give you this out of the box, but these two could be starting points for such a project. Mozilla implemented a REST service based on (an earlier version of) bergamot-translator [1]
12.
by jelmervdl 4y ago
All of them are freely available. Most of them through mtdata [1]. The exact list of the datasets is in the firefox-translations-training pipeline configuration file [2]. [1] https://pypi.org/project/mtdata/ [2]
13.
by jelmervdl 4y ago
I think the Firefox extension might not work on mobile because it hooks into some undocumented addon apis to draw that translation bar UI. Those might not be available on mobile. The translation code itself should work on mobile. It's
14.
by jelmervdl 4y ago
The training pipeline is also on Github! [1] I was experimenting with running the wasm version of bergamot-translator (the translation engine used by the addon) in node [2]. However, if you want more performance, using the Python library [3
15.
by jelmervdl 4y ago
Extension Github page: https://github.com/mozilla/firefox-translations
16.
by jelmervdl 4y ago
The upside of using WASM is that the extension itself can be easily ported to other browsers and platforms. The UI uses Firefox specific APIs but the parts that take the HTML from a page and push it through the translation engine would also