4 ms·

This is the most 2023 answer possible but I wonder if LLM translation will help handle this sort of situation better. An LLM first translates to tokens in more

by mabbo 3y ago

This is the most 2023 answer possible but I wonder if LLM translation will help handle this sort of situation better. An LLM first translates to tokens in more of a "concept space", then translates to other languages. So it would translate "zip file" to something completely different than "zip code", and translate to UK English correctly, or at least better I would hope. They may not solve all problems, but maybe this one?

wizzwizz4 3y ago
This is, in fact, the task that transformers (the technology currently branded as "LLMs") do well. It was the task they were invented for: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762
chrsjxn 3y ago
There's like two main ways this kind of mistake happens. One is a programmer using the wrong string ID. Probably a mistake when searching a translation file/db. The second is missing context. The word "zip" has multiple meanings in software, so you need more information to disambiguate when translating. "Zip code" and "zip file" is probably sufficient, but you could imagine them also having a templated translation for file types like "{} file". I could see an unspecialized LLM translation working well for whole sentences or paragraphs, because that kind of text isn't really likely to run into either of these problems. And if you have the context you need to avoid the second problem for small snippets, then human translation is also pretty unlikely to fail.
numpad0 3y ago
So, such LLM still would have to be informed that this is referring to a zip compressed archive, not a postcode file, whatever such file might be. This is basically the same thing as LLM hallucination, in that it comes from lack of constraints in latent space, if I dare use that buzzword. Without context, a “zip” or “zip file” still can be anything. Could even be “zip [this] file” as a command.

by mabbo 3y ago

wizzwizz4 3y ago

This is, in fact, the task that transformers (the technology currently branded as "LLMs") do well. It was the task they were invented for: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762

chrsjxn 3y ago

There's like two main ways this kind of mistake happens. One is a programmer using the wrong string ID. Probably a mistake when searching a translation file/db. The second is missing context. The word "zip" has multiple meanings in software, so you need more information to disambiguate when translating. "Zip code" and "zip file" is probably sufficient, but you could imagine them also having a templated translation for file types like "{} file". I could see an unspecialized LLM translation working well for whole sentences or paragraphs, because that kind of text isn't really likely to run into either of these problems. And if you have the context you need to avoid the second problem for small snippets, then human translation is also pretty unlikely to fail.

numpad0 3y ago

So, such LLM still would have to be informed that this is referring to a zip compressed archive, not a postcode file, whatever such file might be. This is basically the same thing as LLM hallucination, in that it comes from lack of constraints in latent space, if I dare use that buzzword. Without context, a “zip” or “zip file” still can be anything. Could even be “zip [this] file” as a command.