4 ms·
FWIW even with machine translations (on GPT-4o) I do get occasional hallucinations. Like, it'll add an additional sentence or relative clause that wasn't in the
by BalinKing 2y ago
FWIW even with machine translations (on GPT-4o) I do get occasional hallucinations. Like, it'll add an additional sentence or relative clause that wasn't in the original. It'll still make sense in context, so the overall meaning is the same, but it's not what I'd expect from, say, a human translator. (For context, I know enough of both languages—English and Japanese—to be able to verify this myself.)
It doesn't happen very often—maybe 1 out of 50 times?—but it's definitely a somewhat regular occurrence. And ofc, it's not just a result of the fact that every language is different and so the same meaning will need to be expressed differently—it's actually hallucinating bona fide content.
- BalinKing 2y agoOh, and IME it doesn't ask clarifying questions that a human translator would know are necessary (at least without explicit prompting to do so, which I admittedly haven't tried). So when translating from a high-context language like Japanese, it'll come up with a translation that works in some contexts but is meaningless in others. I figure this is related to the overconfidence issue that's prevalent with (current) LLMs.