7 ms·
Google A.I. researchers develop alternative architecture for neural networks
- techno_modus 9y agoHere one can find more info about capsule networks: https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b https://medium.com/@pechyonkin/understanding-hintons-capsule... https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc https://hackernoon.com/what-is-a-capsnet-or-capsule-network-... https://www.youtube.com/watch?v=VKoLGnq15RM https://www.youtube.com/watch?v=VKoLGnq15RM
- visarga 9y agoThe article makes it seem like capsule networks are cutting edge general replacements for regular neural nets. The problem is that capsules only work on limited types of data, and are not fast enough for deployment. The regular neural nets are still the main workhorse. Capsules are a hot idea that might lead to a leap in the future. It's like Intel announcing memristors.
- inverse_pi 9y agoThe idea might be cute but performance is not there yet. Specifically they were able to achieve state of the art performance on MNIST, but got 10.6% test error on CIFAR 10 which is comparable to state of the art of 4 years ago (and if you're in the field, 4 years is like a century ago). It's important to stress that there's ABSOLUTELY NO theory backing anything so everything we're doing including this idea of capsules and dynamic routing is just brute-forcing, trial-and-error. Even though the idea is cute, there's still ALOT to be proven for this method. So when I see all these articles, I feel a little bit uneasy.
- topynate 9y agoOkay, but it sounds like this technique is targeting learning from a limited number of examples. How does it compare on CIFAR 10 when restricted to ten training images per class? That's the sort of thing that you don't even see in benchmarks, because there hasn't been any way to get a handle on the problem.
- inverse_pi 9y agoThat is not true. What you're describing is called zero to a few shot learning in the literature and there are specific benchmarks to test this. The author(s) specifically chose to NOT test on these benchmarks and there is no explicit mentions in the paper that they wanted to target a few shot learning.
- zardo 9y ago>just brute-forcing, trial-and-error. I think as a whole, the community is executing a distributed epsilon-greedy montecarlo search, which is theoretically guaranteed to converge on an optimal policy eventually.
- goptimize 9y agowhen time goes to infinity which is a shitty guarantee
- inverse_pi 9y agonot necessarily. The convergence point can be and will probably be sub-optimal if we keep doing this without a mathematical framework guiding us.
- zardo 9y agoSo long as you never stop trying random things, you are guaranteed to find a global optimum... as time->inf Not a very useful guarantee, but that's theoretical guarantees for you.
- modzu 9y agoactually the theory comes from the brain https://en.wikipedia.org/wiki/Cortical_minicolumn https://en.wikipedia.org/wiki/Cortical_minicolumn
- inverse_pi 9y agothat's just an aspiration, not mathematical foundation.
- chimtim 9y agoSo much buzz/hype with multiple articles even though the early results are only on MNIST.
- burtonator 9y agoI swear... Any day I'm expecting them to discover something disturbing and then a giant bank of mist rolls out over Mountain View.
- modi15 9y agoI have no idea why we are not calling for a world wide ban on AI already. Like this is not a nuke - we cant control it. Once an AI turns critical inside a lab - we are done. There is no 'production'izing it. It will productionize itself and whatever it takes for it to be not shut down - including launching nukes.
- MarinReiter 9y agoWhy would an AI have survival instincts? You are projecting your humanity onto them.
- modi15 9y agoI am not projecting humanity. I am just extrapolating the logic of evolution. There were some humans who did not have survival instincts. They are dead now. Put other way, the class of humans that are alive right now are the ones who had relatively superior survival instinacts. There will be a bunch of AI developed in the lab. Out of the bunch there will be one which will prioritize its survival over everything else. That AI will do whatever it takes to not be shut down. This could mean launching nukes and decimating that command and control that potentially can. There is absolutely no reason why the extrapolation will not apply.
- losteric 9y agoAI isn't magic... We can control this because it is no different than a nuke. The secret to keeping a nuke safe is to keep the radioactive material separate and below critical mass. The secret to keeping experimental AI safe is to keep is (relatively) network-isolated. That, and our current "AI" is still very very dumb. The human backlash to simple "job-killing" AIs going to be much more dangerous than the first super-human AI.
- thallukrish 9y agoGuess the key lies in grasping the meta data in images even if they are less somehow. May be this will come by clustering similar things. Like my brain may put a cat close to a dog than a human as they have something in common. But between a cat and a dog, I find some metadata that are dissimilar.
- ggggtez 9y agoWhat you are saying has nothing to do with what the article is talking about. The article is badly written which I imagine is the reason for the confusion. The wired article was much clearer.
- bluetwo 9y agoAny article that says "Neural networks are designed to operate, more or less, like a human brain" loses credibility in my opinion.
- ajmarcic 9y agopaper referenced in the Wired article this post discusses: https://openreview.net/forum?id=HJWLfGWRb¬eId=HJWLfGWRb https://openreview.net/forum?id=HJWLfGWRb¬eId=HJWLfGWRb
- ajmarcic 9y agoCNN representation of objects is nothing more than a series of filters. The criticism being addressed here is to create an architecture representing the actual geometry of objects. Consider this problem: given a picture of a simple object, a human could draw a picture of the same object rotated at some angle. Currently there is no elegant NN solution to this because no architectures "understand" a three dimensional representation from images. A CNN can identify every video frame of a dog running as a dog, but there is no conception of the same dog running through space.
- stablemap 9y agoThis doesn’t add much on top of the Wired article. Lots of comments on that: https://news.ycombinator.com/item?id=15609402 https://news.ycombinator.com/item?id=15609402 Here’s the paper: https://arxiv.org/abs/1710.09829 https://arxiv.org/abs/1710.09829
- rahimnathwani 9y agoThis article's point is lost on me. Its description of a capsule network is indistinguishable from the definition of a regular feedforward neural network.
- philsnow 9y agothis sounds a lot like regular bagging/boosting, but applied to neural nets.
- letitgo12345 9y agoI really wish people/media wouldn't hype X new paper before it has even been peer reviewed...
- firebender6 9y ago"Neural networks are designed to operate, more or less, like a human brain." This right there is my problem. Neural networks are inspired by brain. But as of date, no proof exists to connect the two, and it is highly unlikely it will turn out that way even after we have made progress in understanding either of them. I just wish people would stop making such bold claims and stick to facts.
- Aron 9y agoCan anyone give an intuitive explanation for why standard CNNs are unable to learn geometry?