3 ms·

Communication is me taking an idea in my head, considering you the listener and our shared context, and deciding which outward signals will invoke that same ide

Communication is me taking an idea in my head, considering you the listener and our shared context, and deciding which outward signals will invoke that same idea in your head. The full meaning of a statement is NEVER completely contained in the words themselves; they are selected in empathetic prediction merely as triggers of hidden assumed shared state unique to various speaker/listener pairings. As children, we blindly associate phrases to situations in which those phrases were heard, and attempt to map them in our outward communication, often to comical effect. It's a cargo cult model of communication, since at that stage we don't understand how those phrases were originally constructed under specific intention & purpose. The meaning of words and phrases will change over time and location, as we have new experiences which text associates. Certainly as humans we can understand and continually update multiple simultaneous contexts in which things are said, and from which version an utterance is invoking its meaning. These issues are why I don't see the direct text-to-text model of machine translation getting beyond a certain ceiling of usefulness. Text fundamentally refers to out-of-band human experiences, in very inconsistent associations.

skybrian 8y ago
People do still write books targeted at a general audience, though. This generic shared context (what distant strangers can reasonably be expected to know) can plausibly be inferred from training data. So it's not going to get people's in-jokes or local slang, but it still seems like the ceiling could be pretty darn high?
- white-flame 8y ago
  The general audience verbiage still changes over the time axis, though. Are you going to have different training sets for different time periods, and were would you cut the threshold between different corpuses? There's a lot of continuous flux, as well as discrete moments when a phrase turns sour when it was positive or neutral in the past. However, these are still classification issues which NNs could potentially resolve, but are still issues. But it's certainly true that texts written to be specifically & clearly informative will always be easier to translate than fiction, opinion pieces, or informal conversations. It all depends on what you consider to be "high" on the result spectrum. The successes already achieved leave the focus on the existing shortcomings as the ceiling, not necessarily on adding refinement to what it already does respectably well.

skybrian 8y ago

People do still write books targeted at a general audience, though. This generic shared context (what distant strangers can reasonably be expected to know) can plausibly be inferred from training data. So it's not going to get people's in-jokes or local slang, but it still seems like the ceiling could be pretty darn high?

white-flame 8y ago
The general audience verbiage still changes over the time axis, though. Are you going to have different training sets for different time periods, and were would you cut the threshold between different corpuses? There's a lot of continuous flux, as well as discrete moments when a phrase turns sour when it was positive or neutral in the past. However, these are still classification issues which NNs could potentially resolve, but are still issues. But it's certainly true that texts written to be specifically & clearly informative will always be easier to translate than fiction, opinion pieces, or informal conversations. It all depends on what you consider to be "high" on the result spectrum. The successes already achieved leave the focus on the existing shortcomings as the ceiling, not necessarily on adding refinement to what it already does respectably well.