4 ms·
Since GPT-3 transforms sentences into high-dimensional vectors with connections between all the words and of course its internal weights trained on both the Eng
by blixt 6y ago
Since GPT-3 transforms sentences into high-dimensional vectors with connections between all the words and of course its internal weights trained on both the English language and all languages in general, I find that it has a basically flawless ability to interpret normal language.
I tested a prompt where the input was English and the output was Swedish, and it did a really good (but not always perfect) job at translating Swedish. I would assume the amount of Swedish in its training set is relatively tiny, so I find its ability to do that kind of translation in addition to all this other stuff really impressive. This also shows in its ability to translate really well between English and bash shell commands, or JSON data structures, or XML.
- aspenmayer 6y agoIf this kind of thing interests you, look into the field of zero-shot learning. https://en.wikipedia.org/wiki/Zero-shot_learning https://en.wikipedia.org/wiki/Zero-shot_learning Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System https://ai.googleblog.com/2016/11/zero-shot-translation-with-googles.html https://ai.googleblog.com/2016/11/zero-shot-translation-with... Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly https://ieeexplore.ieee.org/document/8413121 https://ieeexplore.ieee.org/document/8413121 also available at: https://arxiv.org/abs/1703.04394 https://arxiv.org/abs/1703.04394 mbsariyildiz / zsl_eval > Python implementation of zero-shot/generalized zero-shot evaluation framework proposed by Xian et al in Zero-Shot Learning - The Good, The Bad, The Ugly. https://github.com/mbsariyildiz/zsl_eval https://github.com/mbsariyildiz/zsl_eval