7 ms·
Artificial Neural Nets Grow Brainlike Navigation Cells
- jimfleming 8y agoTo draw too many parallels here would be like comparing stick figures to still life paintings and proclaiming "They're both flowers!" While it might be true you won't learn much about still life paintings from stick figures. At best this research says something about the task of navigation and optimal representations for that task rather than anything profound about neural networks other than they can both optimize for some task—which should surprise no one.
- chiefalchemist 8y agoThings get over-stated because there would be nothing worthy of publishing otherwise. It's the same (f'd up) model that drives the MSM, etc. The Internet...access to all the information in the world; most of which isn't new or worth knowing about. Bring a shovel. You're gonna need it.
- dmix 8y agoJust because it's not some profound connection doesn't mean the analogy isn't interesting in itself. The author maybe should have toned it down but it was still interesting and blog post worthy IMO.
- chiefalchemist 8y agoPerhaps. But why overblow it? That makes me suspect. I confess, I've spent far too many cycles reading "interesting" things only to have close to no memory of them 2 or 3 days later. My conclusion? Novelty does not equal an increase in the quality of my life. That's not to discount (less impactful) entertainment; only to say (that for me) interesting isn't enough any moew, it's too often not worth the time suck.
- airstrike 8y agoThere are literally dozens of us who feel the same way, yet I for one am still looking for a way to cope.
- abakker 8y agoBack in the old days, people paid for this. I don’t mean to sound snarky, but in a small way, coping with the novelty is my job (as an industry analyst) and some people chose to pay for that. If you need to know, but can’t cope, then you pay someone else to do it who can specialize. This does not solve the problem of things that get published which you don’t need to know and which flood your everyday life. I can’t help but feel that that is an intrinsic part of the internet- Both a feature and a bug.
- chiefalchemist 8y agoIf you're still on FB, I use groups __a lot__. If there's a topic I enjoy, I put a bunch of my friens into it, and then hope for quality sharea. Mainly though, I'm just trying to be more mindful. I have to ask myself "is this __really__ worth my time?" Finally, more (print) books and print mags.
- dbasedweeb 8y agoI think the formula has to be interesting + relevant x challenging = memorable. Trivial novelty is empty, difficult novelty is empty, but difficult, relevant novelty is enriching. If I had to work for knowledge which relates to a broader schema, I find it never goes away.
- chiefalchemist 8y agoThat's how the brain operates: No pain. No gain. I sometimes wonder if the UX mantra "don't make me think" is best. It could be said another way as "make it unmemorable." Sure there are times that's good. But there are plenty where it's not. We all fall for "oh. I'll remember that." But we don't. Whwn we work to attain something the brain is wired to value it more.
- zitterbewegung 8y agoYea same thing happened with fractal patterns and nebula's in deep space 20 years ago. (Also neurons too)
- xaedes 8y agoAny source or keywords to look for this?
- zitterbewegung 8y agohttp://fractalfoundation.org/OFC/OFC-1-6.html http://fractalfoundation.org/OFC/OFC-1-6.html
- limsup 8y agoLink to pdf: https://sci-hub.tw/https://www.nature.com/articles/s41586-018-0102-6.epdf?author_access_token=BjM-5BdGxd14c17YFA6PsdRgN0jAjWel9jnR3ZoTv0OEfySMT4t78PpPpCS7uExW3njb8Q4UlgcwRM32WwBCKZs73SThwkfI42wHhFEtJM-Y7sQxDsR1cR7_C9Kq1GwuxGJn46kzRnujvrDMGzc4TQ%3D%3D# https://sci-hub.tw/https://www.nature.com/articles/s41586-01...
- TaylorAlexander 8y agoThanks. I was curious what it takes to get it from Nature. For the low low price of $199 you too can buy a subscription! As a roboticist just beginning to read ML papers (to help in this very field!), this information would otherwise just be out of reach.
- eboyjr 8y agoThis supports the idea that, increasingly, machine learning is coming back full circle to support neuroscience. Previously, AI researchers looked at the brain for inspiration and now more than ever neuroscientists are being inspired by advances in deep learning, etc.
- make3 8y agothis would be the dream, but biological neural networks don't do gradient decent, so have little to do with ANNs until we discover how they really train.
- jacquesm 8y agoIt is perfectly possible to build neural networks with other functions besides gradient descent.
- make3 8y agoI agree. I'm saying we don't know how biological neural networks are trained, which is a pretty important part of any learning construct, so we can't really know how actually similar artificial neural nets are.
- taneq 8y agoWhat other methods are there for training a neural network? Even blind evolutionary algorithms (assuming each generation is initialized to be somewhat 'near' or 'similar' to its progenitors) are effectively descending a gradient.
- luk32 8y agoTo my knowledge they explore solution space but not through a gradient descent but by probing. How can you perform a GD on non continuous function? Genetic algorithms and evolutionary methods can explore disjoint domains and functions hard from analytical pov. Just because they can be used for gradient descend doesn't mean they use it.
- jcims 8y agoI wonder what the coordinate plane is for the ML visualizations and how it relates to same for visualizations from a physical brain. Seems ripe for gaming.
- zamalek 8y agoAs it is dead reckoning, I assume it is relational/velocity.
- r-bit-rare-e 8y agoQuanta has quit a knack for making perfectly reasonable concepts sound like utter bullshit
- taliesinb 8y agoOn a train and don't have great wifi -- but there was a super-cool poster at ICLR which demonstrated that training an RNN to perform dead-reckoning naturally produced grid-like cells. Is this an extension of that work? Or an independent discovery of the same phenomenon? I'm trying to reproduce that original work, so far without success.
- jimfleming 8y agoI think you're referring to this paper: "Emergence of grid-like representations by training recurrent neural networks to perform spatial localization". https://arxiv.org/abs/1803.07770 https://arxiv.org/abs/1803.07770 It appears to be from Columbia vs DeepMind with different authors.
- gimagon 8y agoAre you referring to the Cueva paper? I also spent a considerable amount of time trying to replicate it without success. Do you know if the author has released any code yet?
- bra-ket 8y agowhy use "deep reinforcement learning" at all, is there any basis to believe it's a valid model of biological learning?
- bitL 8y agoNatural reinforcement learning [1] was known before the mathematical one. Of course, people often mistake Markov chains with reality, but they still can be useful even if in completely unexpected ways like with DRL. [1] Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning II. New York: Appleton-Century Crofts; 1972. pp. 64–99.
- bra-ket 8y agolooks like reinforcement learning people have been missing out on the last 70 years of cognitive psychology. Stimulus-response theories of classical conditioning have been a subject of controversy since at least the 50s[1], these are very poor cognitive models [1] https://en.wikipedia.org/wiki/Behaviorism#Criticisms_and_limitations_of_behaviorism https://en.wikipedia.org/wiki/Behaviorism#Criticisms_and_lim...
- wyattpeak 8y ago> The “grid units” that spontaneously emerged in the network were remarkably similar to what’s seen in animals’ brains, right down to the hexagonal grid. Could someone with more experience in ML explain what this means? In what sense do NN cells have positions or geometry? What are the NN heat maps below the quote showing?
- mr_toad 8y agoThe neurons themselves don’t form grids. A map of the points where firing of a grid neuron in happens the real world forms a triangular/ hexagonal grid. These neurons seem to have discovered what board gamers found out much later - hexagonal grids are better for calculating movement. https://en.m.wikipedia.org/wiki/Grid_cell https://en.m.wikipedia.org/wiki/Grid_cell
- wyattpeak 8y agoAh, thank you. That makes a lot more sense.
- bluetwo 8y agoGive me a valuable use for this and I'll give you an up-vote.