2 ms·
> When LLMs came out, and they now produce text that sounded like it was written by a native speaker (in major languages). But they still often get things comp
by zahlman 1y ago
> When LLMs came out, and they now produce text that sounded like it was written by a native speaker (in major languages).
But they still often get things completely wrong, especially in high-context languages such as Japanese. There often isn't a way to convey the necessary context in text. For example, a Japanese live-streamer who says "配信来てくれてありがとう" means "Thank you for coming to (watch) the stream", but DeepL will give "Thanks for coming to the delivery." - because 配信 actually does mean "delivery" in ordinary circumstances, and that's just the word they idiomatically use to refer to a stream. No matter how much you add from the transcript before or after that, it doesn't communicate the essential fact that the text is a transcript of a livestream.
(And going the other way around, DeepL will insert a に which is grammatically correct but rarely actually uttered by livestreamers who are speaking informally and colloquially; and if you put "stream" in the English, it will be rendered as a loanword ストリーム which is just not what they actually say. Although I guess it should get credit for realizing that you don't mean a small river, which would be 小川.)
(Also, DeepL comes up with complete incomprehensible nonsense for 歌枠, where even basic dictionaries like Jisho will get it right.)
More obviously, they get pronouns and roles wrong all the time - they can't reliably tell whether someone is saying that "I did X" or "you did X" because that may depend on facts of the natural world outside of the actual text (which wouldn't include more information than the equivalent of "did X"). A human translator for, say, a video game cutscene may be able to fix these mistakes by observing what happened in the cutscene. The LLM cannot; no matter how good its model of how Japanese is spoken, it lacks this input channel.