3 ms·
No offense, but this person has no idea what they are talking about. Google published a paper that explains exactly what they are doing and provides metrics as
by ma2rten 10y ago
No offense, but this person has no idea what they are talking about. Google published a paper that explains exactly what they are doing and provides metrics as well [1,2].
It's true that neural networks can not be easily applied to natural languages, but there are less obvious ways of applying them (namely Embeddings, LSTMs, Attention) and provided you have enough data and computational resources they give so much better results than any other method that it doesn't even help anymore to combine them.
[1] https://arxiv.org/abs/1609.08144 https://arxiv.org/abs/1609.08144
[2] https://arxiv.org/abs/1611.04558 https://arxiv.org/abs/1611.04558
EDIT: I also don't want to give the impression that Deep Learning can solve every NLP problem. We are still far away from passing the Turing test. It's true as well in my opinion that Google's Machine Translation is oversold. It's best at what it's trained and evaluated for: translating individual sentences from news sources.
EDIT 2: There are some tasks where traditional methods can work better, e.g. text classification on long documents. That's mainly because deep learning methods are too expensive computationally.
- GFK_of_xmaspast 10y ago> No offense, but this person has no idea what they are talking about Honestly that seems to be the default for technical posts on linkedin.
- stared 10y agoSeconding it with: https://arxiv.org/abs/1610.03017 https://arxiv.org/abs/1610.03017 Fully Character-Level Neural Machine Translation without Explicit Segmentation. In general, RNNs are especially fit for discrete sequences. And their continuous representation is actually an advantage (so they can see that two words are similar or analogous). BTW: see my draft on word2vec: http://p.migdal.pl/2016/12/30/why-do-word2vec-analogies-work.html http://p.migdal.pl/2016/12/30/why-do-word2vec-analogies-work...
- lucidrains 10y agothis paper still consistently blows my mind whenever it is brought up. the future is clearly headed for complete end-to-end learning systems, maybe even with learning to learn methods. https://youtu.be/x1kf4Zojtb0?t=1h4m54s https://youtu.be/x1kf4Zojtb0?t=1h4m54s
- joe_the_user 10y agoNo offense, but this person has no idea what they are talking about. He says: "Why Deep Learning cannot be Applied to Natural Languages Easily" But I say: "It's true that neural networks can not be easily applied to natural languages," Take that!
- ma2rten 10y agoRead beyond the headline.
- joe_the_user 10y agoThe GP is pushing a particular solution they have for the problems of NN's applied to NLP. They might be right that this is a decent answer to the article's objections, for all I know. But at the same time they say "It's true as well in my opinion that Google's Machine Translation is oversold" so the "they don't know what they're talking about except for all the points that I agree with them on" tone seems a bit silly.
- nl 10y agoNeural Translation isn't oversold. It's a bigger breakthrough than AlphaGo.
- ma2rten 10y agoI agree. What I meant was that Google's blog post and the New York Times article make it sound like machine translation is a solved problem.
- nl 10y agoWhich Google blog post made it sound like that? "A Neural Network for Machine Translation, at Production Scale"[1] says: Machine translation is by no means solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page. There is still a lot of work we can do to serve our users better. However, GNMT represents a significant milestone. We would like to celebrate it with the many researchers and engineers—both within Google and the wider community—who have contributed to this direction of research in the past few years. That seems fair to me. "Found in translation: More accurate, fluent sentences in Google Translate"[2] says: Today’s step towards Neural Machine Translation is a significant milestone for Google Translate, but there’s always more work to do and we’ll continue to learn over time. We’ll also continue to rely on Translate Community, where language loving multilingual speakers can help share their language by contributing and reviewing translations. "Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System"[3] says: In September, we announced that Google Translate is switching to a new system called Google Neural Machine Translation (GNMT), an end-to-end learning framework that learns from millions of examples, and provided significant improvements in translation quality. [1] https://research.googleblog.com/2016/09/a-neural-network-for-machine.html https://research.googleblog.com/2016/09/a-neural-network-for... [2] https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/ https://blog.google/products/translate/found-translation-mor... [3] https://research.googleblog.com/2016/11/zero-shot-translation-with-googles.html https://research.googleblog.com/2016/11/zero-shot-translatio...