3 ms·

The author is Douglas Hofstadter, author of Gödel, Escher, Bach.[a] In this article, he shows with concrete examples how Google Translate falls short, and then

by cs702 9y ago

The author is Douglas Hofstadter, author of Gödel, Escher, Bach.[a] In this article, he shows with concrete examples how Google Translate falls short, and then offers two criticisms: * Feeding more data to current models won't bring them any closer to understanding, since understanding involves having ideas (including ideas about the state of the world), and this lack of ideas is the root of all the problems for machine translation today. His examples are powerful evidence of this limitation of current state-of-the-art machine translation systems. * Current machine translation systems make no attempt to go beyond the surface level of words and phrases. These systems merely discover statistical regularities that relate words to other words at multiple, hierarchical levels of composition. In Hofstadter's words, "there's no attempt to create internal structures that could be thought of as ideas, images, memories, or experiences. Such mental etherea are still far too elusive to deal with computationally." He is right. That said, AI researchers are aware of these limitations, and are exploring possible ways to overcome them. An early, crude example of such research is the multi-modal model of "One Model to Learn Them All" (https://arxiv.org/abs/1706.05137 https://arxiv.org/abs/1706.05137), which is trained to learn to perform multiple image-recognition, language-translation, image-captioning, speech-recognition, and language-parsing tasks at the same time, using representations that are shared by all tasks. While these early research efforts fall far short of the kind of "understanding of the world" Hofstadter shows is necessary for human-level language translation, it's encouraging to see AI researchers actively looking for ways to move beyond the mere discovery of 'hierarchical statistical regularities' that relate words to other words. This is an exciting time for AI research. [a] https://en.wikipedia.org/wiki/Douglas_Hofstadter https://en.wikipedia.org/wiki/Douglas_Hofstadter

brd 9y ago
I was under the impression that modern word embedding techniques have emergent properties that start to approach conceptual understanding. The trick is figuring out how to leverage that property properly. I believe Google Brain has a demo where they show translation between two languages without having any corpus specifically linking the two (i.e. the system learned English/Japanese translations and English/Greek translations and then translated Greek/Japanese) with a reasonable degree of accuracy. This seems to me to hint at a more conceptual understanding as opposed to a simple pattern matching activity. In either case, I 100% agree that the multi-modal model is the path forward. It feels like there's a lot of low hanging fruit in the area of model ensambles.

by cs702 9y ago

brd 9y ago

I was under the impression that modern word embedding techniques have emergent properties that start to approach conceptual understanding. The trick is figuring out how to leverage that property properly. I believe Google Brain has a demo where they show translation between two languages without having any corpus specifically linking the two (i.e. the system learned English/Japanese translations and English/Greek translations and then translated Greek/Japanese) with a reasonable degree of accuracy. This seems to me to hint at a more conceptual understanding as opposed to a simple pattern matching activity. In either case, I 100% agree that the multi-modal model is the path forward. It feels like there's a lot of low hanging fruit in the area of model ensambles.