4 ms·
This is much more impressive than I think it will get credit for. I mean really, having an always on, immediate access massive NLP DNN system translating every
by AndrewKemendo 9y ago
This is much more impressive than I think it will get credit for.
I mean really, having an always on, immediate access massive NLP DNN system translating every piece of text from any language on the worlds largest platform is a staggering feat.
Facebook has the most impressive applications of ML right now in my opinion. They have Yann Lecun to thank for that (and Mark for recruiting him).
- whatrusmoking 9y ago??? Google's been doing this at scale for a while.
- gwern 9y agoGoogle switched Google Translate over just last year, November 2016. Considering Facebook is developing all this internally and the scale they operate at and the presumable hardware disadvantages (does FB have any equivalent of Google's TPUs?), switching over in production so quickly is impressive.
- kuschku 9y agoOnly for very few language pairs, though. But for those translation massively improved, especially for German the old solution was basically unusable.
- zitterbewegung 9y agoGoogle was doing a statistical method of translation for a long time for Google Translate. Last year they switched over to Neural Machine Translation.
- dmcy22 9y agoGoogle only supports Neural Machine Translation for a subset of languages. Whereas as Facebook is doing it for 2,000 translation directions.
- Retric 9y ago45 languages * 44 other langues ~= 2000 translation directions. There are ~4,500 languages with more than 1,000 speakers ~4,500 * 4,499 ~= 20,254,500 language pairs so that's a very long way from every language.
- hood_syntax 9y agoFrankly though, I imagine the vast majority of those languages are unappealing targets in terms of return on investment.
- flamedoge 9y agoand much smaller training data..
- actuator 9y agoI guess they really don't need to cover all of them though as most of the people speaking a language with low number of speakers would be able to converse in other mainstream languages as well. I can speak four languages but I rarely converse in two of them as speakers of the two languages are usually comfortable with a mainstream language as well.
- jacquesm 9y agoEven Swedish is barely in the top 100 with 8.7 million speakers.
- rjeli 9y agoThe most obvious thing to me is to use an intermediate language - I assume this has already been discussed extensively, anyone have references? edit: I should have scrolled down https://techcrunch.com/2016/11/22/googles-ai-translation-tool-seems-to-have-invented-its-own-secret-internal-language/ https://techcrunch.com/2016/11/22/googles-ai-translation-too...
- riku_iki 9y agoGoogle may also have higher bar for translation quality..
- deleted 9y ago[deleted]
- tanilama 9y agoGoogle created GNMT and put it into production first, then improve it so single model can tranlsate multiple languages.
- amelius 9y ago> having an always on, immediate access massive NLP DNN system translating every piece of text from any language on the worlds largest platform is a staggering feat. Too bad that they also have an always on system that's tracking my doings and whereabouts. Seriously, we need to stop applauding these companies. Sad to see that LeCun and colleagues don't care to find an employer with more noble goals.
- tanilama 9y agoDo u have a list of such noble companies to offer?
- jawbone3 9y agoGapminder, amnesty, msf, save the children... wait did you ask for ad companies like fb and google that sell clicks but had a noble agenda? I think the mercenary nature of ads precludes most moral stances in deference to the client.
- visarga 9y agoI avoid FB as a social network, but I love their open source projects such as those in ML (PyTorch) and web design (React). Weirdly, I feel better working with FB products than the equivalent from Google (Angular and TensorFlow). It seems that approval for the behavior of a company and it's open tech stack don't need to be correlated.
- amelius 9y agoIs that a rational argument or an emotional argument?
- dmcy22 9y agoImpressive indeed. Would you happen to know if Facebook is able to translate without using English as an intermediate language? For example, from Dutch directly to French, without needing to go from Dutch to English to French.
- zlynx 9y agoReminds me reading about Google's new translator. It apparently uses a unique internal language it developed during training. All the languages it knows go through this internal language / neural pattern. From what I recall reading, Google has people researching this internal language to see if they can discover any new interesting things about human thought. https://techcrunch.com/2016/11/22/googles-ai-translation-tool-seems-to-have-invented-its-own-secret-internal-language/ https://techcrunch.com/2016/11/22/googles-ai-translation-too...
- landon32 9y agoI don't know Facebook's specifics, but typically in research they will train a wide variety of examples, e.g. English -> French French -> English English -> German French -> German
- etiam 9y agoIsn't that what they are doing? My impression of the pure DL approaches is that the preferred way to go about it is to map from each available language example, via a common latent representation, to every other available language example. The common currency is not English but a vector space representing something much more akin to the meaning underlying the statements.