4 ms·

Huh. This article is about the issue of picking which translation of a word to use according to what the word means in context... a task that LLMs should ace.

Huh. This article is about the issue of picking which translation of a word to use according to what the word means in context... a task that LLMs should ace. I was hoping this was about using an LLM to produce text in the house style. That's also a task they should be good at, but that clearly doesn't happen by default. Here's a two-page public announcement from Ironhide Games, who make Kingdom Rush. They're apologizing over a direction they chose for the upcoming 6th game in the series: https://cdn.discordapp.com/attachments/524968371494846484/1530199721832878110/POST_1.jpg?ex=6a74870b&is=6a73358b&hm=4b46db94d660e295ccccc83425bdc3bbc1e78df8281f17ea94321eacbc7c7638& https://cdn.discordapp.com/attachments/524968371494846484/15... https://cdn.discordapp.com/attachments/524968371494846484/1530199722185461842/POST_2.jpg?ex=6a74870b&is=6a73358b&hm=12f4b2ac2fb61b6096f7f73e6f25e48327705cd67882601573c011f42e13735a& https://cdn.discordapp.com/attachments/524968371494846484/15... To me the main message is twofold: - "People complained, and we're going in a different direction." - "We had this translated from Spanish by an LLM." The 11 paragraphs of text are superfluous, because... (1) anything you could learn from them you could also learn from the two-bullet-point summary on the second page; and (2) the text style clearly doesn't reflect anything that the spokesperson for Ironhide Games wrote. This is a document written specifically to prevent any information from coming through other than the two bullet points in the summary. And that's a style that companies often invoke on purpose. PR statements are vetted for this kind of thing, which is why everyone hates them. But it appears to be the only style that LLMs want to produce, whether they're making a PR statement or not, and I feel sure that almost none of the people using them for translations actually want this. I also feel sure that the "LLM style" is specifically intended by the LLM providers, and that it wouldn't exist if they didn't spend enormous effort causing it to.

stuess 3mo ago
This is interesting! Exactly our experience, you do have to guide LLMs on author the source text in the way we would write, with few-shot examples in the prompt. And then prompt them to transcreate so it doesn't calque (read as a literal translation rather than something a native speaker would have written for other native speakers). Both of this can be solved for reasonably well with the right prompt guidance IMHO. In my experience, there's two key incredients to get to a good results (with humans OR LLMs doing the work), I've worked on plenty of sites that had human l10n before LLMs came about. If you're interested in the actual prompt we're using for our voice that includes instructions on how to avoid sounding like bad LLM copy, I've popped that here in a Github Gist: https://gist.github.com/tobyurff/2b461c259c34dfc6758a39329822dab8 https://gist.github.com/tobyurff/2b461c259c34dfc6758a3932982... Hope it helps, I might turn that into a blog post one day as well, seems like this could be useful to others!

stuess 3mo ago

This is interesting! Exactly our experience, you do have to guide LLMs on author the source text in the way we would write, with few-shot examples in the prompt. And then prompt them to transcreate so it doesn't calque (read as a literal translation rather than something a native speaker would have written for other native speakers). Both of this can be solved for reasonably well with the right prompt guidance IMHO. In my experience, there's two key incredients to get to a good results (with humans OR LLMs doing the work), I've worked on plenty of sites that had human l10n before LLMs came about. If you're interested in the actual prompt we're using for our voice that includes instructions on how to avoid sounding like bad LLM copy, I've popped that here in a Github Gist: https://gist.github.com/tobyurff/2b461c259c34dfc6758a39329822dab8 https://gist.github.com/tobyurff/2b461c259c34dfc6758a3932982... Hope it helps, I might turn that into a blog post one day as well, seems like this could be useful to others!