6 ms·
Deep Learning – Review by LeCun, Bengio, and Hinton
- cowpig 11y agoHurray for science behind paywalls!
- IanCal 11y agoHere's a link where you can read it: http://www.nature.com/articles/nature14539.epdf?referrer_access_token=K4awZz78b5Yn2_AoPV_4Y9RgN0jAjWel9jnR3ZoTv0PU8PImtLRceRBJ32CtadUBVOwHuxbf2QgphMCsA6eTOw64kccq9ihWSKdxZpGPn2fn3B_8bxaYh0svGFqgRLgaiyW6CBFAb3Fpm6GbL8a_TtQQDWKuhD1XKh_wxLReRpGbR_NdccoaiKP5xvzbV-x7b_7Y64ZSpqG6kmfwS6Q1rw%3D%3D&tracking_referrer=www.nature.com http://www.nature.com/articles/nature14539.epdf?referrer_acc...
- rfrey 11y ago$32 seems steep for one article. Does anyone know if Nature allows authors to publish on their websites? Nothing so far on Hinton's, LeCun's, or Bengio's page.
- cowpig 11y agoHaving to pay for this at all is problematic.
- robotresearcher 11y agowhy? Because the authors are not paid for their work, or some other reason?
- chriskanan 11y agoThis link allows you to access the entire document using ReadCube: http://www.nature.com/articles/nature14539.epdf?referrer_access_token=K4awZz78b5Yn2_AoPV_4Y9RgN0jAjWel9jnR3ZoTv0PU8PImtLRceRBJ32CtadUBVOwHuxbf2QgphMCsA6eTOw64kccq9ihWSKdxZpGPn2fn3B_8bxaYh0svGFqgRLgaiyW6CBFAb3Fpm6GbL8a_TtQQDWKuhD1XKh_wxLReRpGbR_NdccoaiKP5xvzbV-x7b_7Y64ZSpqG6kmfwS6Q1rw%3D%3D&tracking_referrer=www.nature.com http://www.nature.com/articles/nature14539.epdf?referrer_acc... It doesn't allow you to print or to save the article.
- deleted 11y ago[deleted]
- gwern 11y agohttps://www.dropbox.com/s/fmc3e4ackcf74lo/2015-lecun.pdf https://www.dropbox.com/s/fmc3e4ackcf74lo/2015-lecun.pdf / http://sci-hub.org/downloads/d397/lecun2015.pdf http://sci-hub.org/downloads/d397/lecun2015.pdf
- dmazin 11y agogwern saves the day once again.
- nabla9 11y agohttp://libgen.in/scimag/?s=10.1038/nature14539 http://libgen.in/scimag/?s=10.1038/nature14539
- foxbarrington 11y agoBut think of how much all the research costs Nature.
- IanCal 11y agoTheir editor-in-chief estimates their costs to be ~$30-40,000 per article. Much higher than other publishers, it seems, probably in part due to their high rejection rate (92% in 2011). http://www.nature.com/news/open-access-the-true-cost-of-science-publishing-1.12676 http://www.nature.com/news/open-access-the-true-cost-of-scie...
- joelthelion 11y agoThat could be reduced a lot if they had an incentive to do it. The work that counts, ie. the research and the peer-review, are free and not compensated at all by the publisher.
- IanCal 11y agoI expect prices for everything could be reduced with enough effort, but I think that all businesses have an incentive to reduce internal costs. They apparently publish roughly 1000 articles per year[0] so that suggests internal costs in the range of 30-40 million. > The work that counts, ie. the research and the peer-review, are free and not compensated at all by the publisher. The peer review is done for free, yes. I don't think I'd class the research as free though, unless you mean free to the publisher, the scientists are still generally paid. Again, I'm not arguing for paid journals, just pointing out that they do have costs to run. [0] http://lib.hzau.edu.cn/xxfw/SCIzx/Document/249/Image/20101012249_5.pdfa http://lib.hzau.edu.cn/xxfw/SCIzx/Document/249/Image/2010101...
- gwern 11y ago"_Nature_ says that it will not disclose information on margins."
- IanCal 11y ago
- thisisdave 11y agoHas LeCun changed his position on open access? He'd previously "pledged to no longer do any volunteer work, including reviewing, for non-open access publications." [1] A major part of his reasoning was, "Why should people pay $31 to read individual papers when they could get them for free if they were published by JMLR?" I had assumed that meant he wouldn't write for them either (and thus wouldn't enlist other people to volunteer as reviewers when the final product would cost $32 to read). [1] https://plus.google.com/+YannLeCunPhD/posts/WStiQ38Hioy https://plus.google.com/+YannLeCunPhD/posts/WStiQ38Hioy
- letitgo12345 11y agoTo be fair, it's a review article in nature -- so meant to advertise deep learning to scientists in other fields. His actual research is still publicly accessible.
- grayclhn 11y ago"Review" in the context of your link means "referee." I don't think that most researchers would view writing a review article for Nature as "volunteer work" because the career benefits of the publication are pretty high. So, going by the link, I don't think this is a change in his position on open access, but I also don't think that his position involved as much self-sacrifice as you'd assumed. edit: I don't know the field well, but these review articles usually recycle a lot of material from review articles the author wrote a year or two before. The contents of the article might be basically available for free already.
- thisisdave 11y ago> I don't think that most researchers would view writing a review article for Nature as "volunteer work" I was referring more specifically to the fact that Nature had to enlist volunteers to referee the paper he wrote. I was curious whether he was okay with that even though he wouldn't volunteer himself, if his position on non-open journals had changed, or if there was some other explanation.
- paulsutter 11y ago
- yzh 11y agoThere is a whole session on machine intelligence in this issue. Just curious, is there more CS-related research on Nature now than before?
- chriskanan 11y agoNature and Science both generally have only a few CS-focused papers per year. This issue has a special theme (Machine learning and robots), so it breaks that general pattern. I regularly read Nature and Science for general science news, commentary, etc.
- yzh 11y agoThanks.
- evc123 11y agoNature should just use targeted advertising to make their journals free, similar to the way google/facebook make their services free using targeted ads.
- grayclhn 11y agoI think you're overestimating the number of people that read Nature.
- itistoday2 11y agoWhy do these articles on RNNs and "Deep Learning" never mention Hierarchical Temporal Memory? https://en.wikipedia.org/wiki/Hierarchical_temporal_memory#Deep_Learning https://en.wikipedia.org/wiki/Hierarchical_temporal_memory#D...
- akyu 11y agoIt seems like theres a kind of taboo against HTM in the ML community. I guess it stems from their lack of impressive benchmarks, but I think that's kind of missing the point when it comes to HTM. Maybe HTM isn't the right solution, but I think there is a lot to be learned by using neocortex inspired models, and HTM is at least a solid step in that direction. And the work Numenta has contributed on Sparse Codings shouldn't be overlooked.
- davmre 11y agoI don't think most ML people are actively hostile to HTM, just indifferent until they show some results. For example, Yann Lecun in his Reddit AMA: http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_yann_lecun/chisjsc http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_...
- ajays 11y agoI've yet to find one significant benchmark dataset where HTM beats other methods. One.
- dwf 11y agoExtraordinary claims demand extraordinary evidence. Numenta has plenty of the former and nil as regards the latter. Even if they had invented some believably interesting task and done fair comparisons with other methods and shown that HTM succeeds where others fail, it would be considered worth a look by the wider machine learning community.
- Teodolfo 11y agoBecause it doesn't meet academic standards for publishing in NIPS and ICML, the most prestigious machine learning conferences. Edit: To clarify, research papers generally cite other peer-reviewed research papers in similar venues preferentially. ML papers should mostly be citing ML papers in high-quality, peer-reviewed venues. HTM doesn't have papers like this to cite.
- paulsutter 11y agoIf you only have time to read one paper on Deep Learning, read this paper. A few quotes: "This rather naive way of performing machine translation has quickly become competitive with the state-of-the-art, and raises serious doubts about whether understanding a sentence requires anything like the internal symbolic expressions that are manipulated by using inference rules. It is more compatible with the view that everyday reasoning involves many simultaneous analogies that each contribute plausibility to a conclusion" "The issue of representation lies at the heart of the debate between the logic-inspired and the neural-network-inspired paradigms for cognition. In the logic-inspired paradigm, an instance of a symbol is something for which the only property is that it is either identical or non-identical to other symbol instances. It has no internal structure that is relevant to its use; and to reason with symbols, they must be bound to the variables in judiciously chosen rules of inference. By contrast, neural networks just use big activity vectors, big weight matrices and scalar non-linearities to perform the type of fast ‘intuitive’ inference that underpins effortless commonsense reasoning." "Problems such as image and speech recognition require the input–output function to be insensitive to irrelevant variations of the input, such as variations in position, orientation or illumination of an object, or variations in the pitch or accent of speech, while being very sensitive to particular minute variations (for example, the difference between a white wolf and a breed of wolf-like white dog called a Samoyed). At the pixel level, images of two Samoyeds in different poses and in different environments may be very different from each other, whereas two images of a Samoyed and a wolf in the same position and on similar backgrounds may be very similar to each other. A linear classifier, or any other ‘shallow’ classifier operating on raw pixels could not possibly distinguish the latter two, while putting the former two in the same category.... The conventional option is to hand design good feature extractors, which requires a considerable amount of engineering skill and domain expertise. But this can all be avoided if good features can be learned automatically using a general-purpose learning procedure. This is the key advantage of deep learning." "Deep neural networks exploit the property that many natural signals are compositional hierarchies, in which higher-level features are obtained by composing lower-level ones. In images, local combinations of edges form motifs, motifs assemble into parts, and parts form objects. Similar hierarchies exist in speech and text from sounds to phones, phonemes, syllables, words and sentences. The pooling allows representations to vary very little when elements in the previous layer vary in position and appearance"
- jphilip147 11y agoVery Helpful review.