5 ms·

It’s not free, but I’ve had some success using ChatGPT’s Advanced Voice mode for sequential interpreting between English and Japanese. I found I had to first ex

It’s not free, but I’ve had some success using ChatGPT’s Advanced Voice mode for sequential interpreting between English and Japanese. I found I had to first explain the situation to the model and tell it what I wanted it to do. For example: “I am going to have a conversation with my friend Taro. I speak English, and he speaks Japanese. Translate what I say into Japanese and what he says into English. Only translate what we say. Do not add any explanations or commentary.” We had to be careful not to talk over each other or the model, and the interpreting didn’t work well in a noisy environment. But once we got things set up and had practiced a bit, the conversations went smoothly. The accuracy of the translations was very good. Such interpreting should get even better once the models have live visual input so that they can “see” the speakers’ gestures and facial expressions. Hosting on local devices, for less latency, will help as well. In business and government contexts, professional human interpreters are usually provided with background information in advance so that they understand what people are talking about and know how to translate specialized vocabulary. LLMs will need similar preparation for interpreting in serious contexts.

bemmu 2y ago
I tried this a few times since my family speaks Finnish and son speaks Japanese, but the issue is that it keeps forgetting the task. It’ll work at first for a sentence or two, then the other party asks something and instead of translating the question, it will attempt to answer the question. Even if you remind it of its task it quickly forgets again.
- tkgally 2y ago
  That happened once or twice for me, too. I wonder if an interpreting-specific system prompt would prevent that problem….
mrandish 2y ago
> In business and government contexts, professional human interpreters are usually provided with background information in advance so that they understand what people are talking about and know how to translate specialized vocabulary. LLMs will need similar preparation for interpreting in serious contexts. I've given some high-profile keynote speeches where top-notch (UN-level) real-time simultaneous interpreters were used. Even though my content wasn't technical or even highly specialized (more general business), spending an hour with them previewing my speech and answering their questions led to dramatically better audience response. I was often the only speaker on a given day who made the effort to show up for the prep session. The interpreters told me the typical improvement they could achieve from even basic advance prep was usually >50%. It gave me a deep appreciation for just how uniquely challenging and specialized the ability to do this at the highest level is. These folks get big bucks and from my experience, they're worth it. AI is super impressive but I suspect getting AI to replicate the last 10% of quality top humans can achieve is going to be >100% more work. An even more specialized and challenging use case is the linguists who spend literally years translating one high-profile book of literature (like Tolstoy). They painstakingly craft every phrase to balance conveying meaning, pace and flavor etc. If you read expert reviews of literature translation quality you get the sense it's a set of trade-offs for which no optimal solution exists, thus how these trade-offs are balanced is almost as artistic and expressive an endeavor as the original authorship. Current AI translators can do a fine job translating things like directions and restaurant menus in real-time, even on a low-end mobile device but the upper bound of quality achieved by top humans on the hardest translation tasks is really high. It may be a quite a while before AIs can reach these levels technically - and perhaps even longer practically because the hardest use cases are so challenging and the market at the high-end is far too small to justify investing the resources in attaining these levels.
- tkgally 2y ago
  I agree with everything you wrote. A few more comments from a long-time professional translator: The unmet demand for communication across language barriers is immense, and, as AI quality continues to improve, translators and interpreters working in more faceless and generic fields will gradually be replaced by the much cheaper AI. That is already happening in translation: Some experienced patent translators I know have seen their workflows dry up in the past couple of years and are trying to find new careers. This past August, I gave a talk on AI developments to a group of professional interpreters in Tokyo. While they hadn’t heard of any interpreters losing work to AI yet, they thought that some types of work—online interpreting for call centers, for example—would be soon replaced by AI. The safest type of human interpreting work, I suspect, is in-person interpreting at business meetings and the like. The interpreters’ physical presence and human identity should encourage the participants to value and trust them more than they would AI. Real-time simultaneous translators are an interesting case. On the one hand, they are the rarest and most highly skilled, and national governments and international organizations have a long track record of relying on and trusting them. On the other hand, they usually work in soundproof booths, barely visible (if at all) to the participants. The work is very demanding, so interpreters usually work in twenty- to thirty-minute shifts; the voice heard by meeting participants therefore changes periodically. The result is less awareness that the interpreting is being done by real people, so users might feel less hesitation about replacing them with AI. When I’ve organized or participated in conferences that had simultaneous human interpreting, the results were mixed. While sometimes it worked well, people often reported having a hard time following the interpretion on headphones. Question-and-answer sessions were often disjointed, with an audience member asking about some point that they seem to have misunderstood in the interpretation and then their interpreted question not being understood by the speaker. The interpreters were well-paid professionals, though not perhaps UN-level.

bemmu 2y ago

I tried this a few times since my family speaks Finnish and son speaks Japanese, but the issue is that it keeps forgetting the task. It’ll work at first for a sentence or two, then the other party asks something and instead of translating the question, it will attempt to answer the question. Even if you remind it of its task it quickly forgets again.

tkgally 2y ago
That happened once or twice for me, too. I wonder if an interpreting-specific system prompt would prevent that problem….

mrandish 2y ago

> In business and government contexts, professional human interpreters are usually provided with background information in advance so that they understand what people are talking about and know how to translate specialized vocabulary. LLMs will need similar preparation for interpreting in serious contexts. I've given some high-profile keynote speeches where top-notch (UN-level) real-time simultaneous interpreters were used. Even though my content wasn't technical or even highly specialized (more general business), spending an hour with them previewing my speech and answering their questions led to dramatically better audience response. I was often the only speaker on a given day who made the effort to show up for the prep session. The interpreters told me the typical improvement they could achieve from even basic advance prep was usually >50%. It gave me a deep appreciation for just how uniquely challenging and specialized the ability to do this at the highest level is. These folks get big bucks and from my experience, they're worth it. AI is super impressive but I suspect getting AI to replicate the last 10% of quality top humans can achieve is going to be >100% more work. An even more specialized and challenging use case is the linguists who spend literally years translating one high-profile book of literature (like Tolstoy). They painstakingly craft every phrase to balance conveying meaning, pace and flavor etc. If you read expert reviews of literature translation quality you get the sense it's a set of trade-offs for which no optimal solution exists, thus how these trade-offs are balanced is almost as artistic and expressive an endeavor as the original authorship. Current AI translators can do a fine job translating things like directions and restaurant menus in real-time, even on a low-end mobile device but the upper bound of quality achieved by top humans on the hardest translation tasks is really high. It may be a quite a while before AIs can reach these levels technically - and perhaps even longer practically because the hardest use cases are so challenging and the market at the high-end is far too small to justify investing the resources in attaining these levels.

tkgally 2y ago
I agree with everything you wrote. A few more comments from a long-time professional translator: The unmet demand for communication across language barriers is immense, and, as AI quality continues to improve, translators and interpreters working in more faceless and generic fields will gradually be replaced by the much cheaper AI. That is already happening in translation: Some experienced patent translators I know have seen their workflows dry up in the past couple of years and are trying to find new careers. This past August, I gave a talk on AI developments to a group of professional interpreters in Tokyo. While they hadn’t heard of any interpreters losing work to AI yet, they thought that some types of work—online interpreting for call centers, for example—would be soon replaced by AI. The safest type of human interpreting work, I suspect, is in-person interpreting at business meetings and the like. The interpreters’ physical presence and human identity should encourage the participants to value and trust them more than they would AI. Real-time simultaneous translators are an interesting case. On the one hand, they are the rarest and most highly skilled, and national governments and international organizations have a long track record of relying on and trusting them. On the other hand, they usually work in soundproof booths, barely visible (if at all) to the participants. The work is very demanding, so interpreters usually work in twenty- to thirty-minute shifts; the voice heard by meeting participants therefore changes periodically. The result is less awareness that the interpreting is being done by real people, so users might feel less hesitation about replacing them with AI. When I’ve organized or participated in conferences that had simultaneous human interpreting, the results were mixed. While sometimes it worked well, people often reported having a hard time following the interpretion on headphones. Question-and-answer sessions were often disjointed, with an audience member asking about some point that they seem to have misunderstood in the interpretation and then their interpreted question not being understood by the speaker. The interpreters were well-paid professionals, though not perhaps UN-level.