4 ms·

https://en.wikipedia.org/wiki/Bio-inspired_computing https://en.wikipedia.org/wiki/Bio-inspired_computing Present day Neuron models lack an incredible number o

https://en.wikipedia.org/wiki/Bio-inspired_computing https://en.wikipedia.org/wiki/Bio-inspired_computing Present day Neuron models lack an incredible number of functional features that are clearly present in the human brain. NTMs = representing memory that is stored in neurons https://en.wikipedia.org/wiki/Neuronal_memory_allocation https://en.wikipedia.org/wiki/Neuronal_memory_allocation Decoupled Neural Interfaces using Synthetic Gradients = https://en.wikipedia.org/wiki/Electrochemical_gradient https://en.wikipedia.org/wiki/Electrochemical_gradient Differentiable Neural Computers = Won't specify what natural aspect of the brain this derives from. Pick an aspect of a neuron or the brain that isn't modeled, write a model... Bleeding edge + Operating on another level The fact that someone is going out of there way to remove points from my posts so that this doesn't see tomorrow's foot traffic instead of replying and critiquing me just goes to show how truthful these statements are. Anyone can create such models. No one has a monopoly or patent on how the brain functions. Thus, expect many models and approaches.. Some better than others. You can down-vote all you want. The better model and architecture wins this game. It would help the community if people were honest about what's going on here but people instead want to believe in magic and subscribe to the idea that only a specific group of people are writing biologically inspired software and are capable authoring a model of what is clearly documented in the human brain. Interesting that this is the reception.

gjm11 10y ago
You're not getting downvoted for being mean about DeepMind, you're getting downvoted for making overconfident pronouncements about things you don't understand. "Neural Turing machines" are not the same thing as neuronal memory allocation: NTMs' memory is external and neuronal memory allocation is all about how memory is stored in neurons in the brain. The "synthetic gradients" in that paper have nothing to do with the electrochemical gradients you mention other than the name. No one is claiming that the DeepMind guys are "operating on another level" because they do bio-inspired things. They are claiming that because they are getting more impressive results than anyone else. Now: Are they really? If so, is that enough justification for such a grand-sounding claim. I don't know. That would be an interesting discussion to have. But "Boooo, these people are just copying things present in the brain, there's nothing impressive about that" is not, especially when the parallels between the brain-things and the DeepMind-things are as feeble as in your examples.
- outsideline 10y ago
  Overconfident pronouncement by indicating that they are making computational models of natural processes that no one can confidently state are correct or are the most efficient? Making statements that allow people to see behind the curtains and maybe go off and make their own competitive models... Yes, this is a disservice to the advancement of A.I and should be downvoted : Removing the prestigious veil and illusion from published works. NTMs memory is external in what sense? Please detail what this means in a 'functional' sense. It's biologically inspired. Neurons maintain memory beyond synaptic weights. The neuron models of present day A.I were basic. Someone comes along and sees the obvious : There is no computational model for how neurons utilize memory and suddenly they're thinking on another level? Give me a break.. Synthetic gradients have everything to do w/ electro-chemical gradients : http://www.nature.com/articles/srep14527 http://www.nature.com/articles/srep14527 http://www.pnas.org/content/110/30/12456.full.pdf http://www.pnas.org/content/110/30/12456.full.pdf So, where is your establishment that I am incorrect. It is nowhere to be found. Again, biologically inspired computational models. Oh look, someone published a paper back in June that is an implementation of Differentiable Neural Computers: https://arxiv.org/abs/1607.00036 https://arxiv.org/abs/1607.00036 It's hype and that is a disservice to the community of people completing similar work and taking similar approaches. It would be an interesting discussion to have. That discussion was terminated in favor of downvoting me. They're feeble to someone who isn't well informed on neuroscience. Thus, you'd rather be wow'd and believe in the fantasy that only a small segment of people can write computational models of biology. Continue believing the hype. Rarely will someone be truthful and honest about where they got their ideas when hype follows. An interesting conversation could have transpired. Enjoy the feels from the downvotes.
  - TeeWEE 10y ago
    An "Electro chemical gradient" is an ion. That works on small scale within cells. The gradient here is the "electrochemical potential" of the ion. An Synthetic Gradient is a way to allow learning Forward Propagated Neural Nets in a parallel way. The gradient here is referring to the 'error' backpropagation that is part of the training process of an neural net (im talking about computer science neural nets). They have nothing todo with each other. The papers that you are referring to have nothing todo with the process of training a neural net.
znah 10y ago
Ideas are cheap, making them work is hard.
- outsideline 10y ago
  The brain's architecture is laid bare for anyone to see. Making models of features is cheap. Anyone can do it. There are loads of white papers and benchmarks. Some approaches beat others depending on the benchmark. Claiming it 'works' by tweaking it until it fits a canned benchmark is not hard work. Not having any explanation as to why it works is not hard work. Coming up with a functional systems architecture that ties the bits and pieces together is hard work. Understanding what is really happening in the human brain, how/why it is performing various functions, and how this provides for an intelligent architecture is hard work. Creating an 'aware' platform is hard and elusive work which is why people chase the low hanging fruit of optimization algorithms. *Cheers
deleted 10y ago
[deleted]
akssri 10y ago
Do you know if CompNeuro models have been trained to do things interesting to CS folk ?

gjm11 10y ago

You're not getting downvoted for being mean about DeepMind, you're getting downvoted for making overconfident pronouncements about things you don't understand. "Neural Turing machines" are not the same thing as neuronal memory allocation: NTMs' memory is external and neuronal memory allocation is all about how memory is stored in neurons in the brain. The "synthetic gradients" in that paper have nothing to do with the electrochemical gradients you mention other than the name. No one is claiming that the DeepMind guys are "operating on another level" because they do bio-inspired things. They are claiming that because they are getting more impressive results than anyone else. Now: Are they really? If so, is that enough justification for such a grand-sounding claim. I don't know. That would be an interesting discussion to have. But "Boooo, these people are just copying things present in the brain, there's nothing impressive about that" is not, especially when the parallels between the brain-things and the DeepMind-things are as feeble as in your examples.

outsideline 10y ago
Overconfident pronouncement by indicating that they are making computational models of natural processes that no one can confidently state are correct or are the most efficient? Making statements that allow people to see behind the curtains and maybe go off and make their own competitive models... Yes, this is a disservice to the advancement of A.I and should be downvoted : Removing the prestigious veil and illusion from published works. NTMs memory is external in what sense? Please detail what this means in a 'functional' sense. It's biologically inspired. Neurons maintain memory beyond synaptic weights. The neuron models of present day A.I were basic. Someone comes along and sees the obvious : There is no computational model for how neurons utilize memory and suddenly they're thinking on another level? Give me a break.. Synthetic gradients have everything to do w/ electro-chemical gradients : http://www.nature.com/articles/srep14527 http://www.nature.com/articles/srep14527 http://www.pnas.org/content/110/30/12456.full.pdf http://www.pnas.org/content/110/30/12456.full.pdf So, where is your establishment that I am incorrect. It is nowhere to be found. Again, biologically inspired computational models. Oh look, someone published a paper back in June that is an implementation of Differentiable Neural Computers: https://arxiv.org/abs/1607.00036 https://arxiv.org/abs/1607.00036 It's hype and that is a disservice to the community of people completing similar work and taking similar approaches. It would be an interesting discussion to have. That discussion was terminated in favor of downvoting me. They're feeble to someone who isn't well informed on neuroscience. Thus, you'd rather be wow'd and believe in the fantasy that only a small segment of people can write computational models of biology. Continue believing the hype. Rarely will someone be truthful and honest about where they got their ideas when hype follows. An interesting conversation could have transpired. Enjoy the feels from the downvotes.
- TeeWEE 10y ago
  An "Electro chemical gradient" is an ion. That works on small scale within cells. The gradient here is the "electrochemical potential" of the ion. An Synthetic Gradient is a way to allow learning Forward Propagated Neural Nets in a parallel way. The gradient here is referring to the 'error' backpropagation that is part of the training process of an neural net (im talking about computer science neural nets). They have nothing todo with each other. The papers that you are referring to have nothing todo with the process of training a neural net.

znah 10y ago

Ideas are cheap, making them work is hard.

outsideline 10y ago
The brain's architecture is laid bare for anyone to see. Making models of features is cheap. Anyone can do it. There are loads of white papers and benchmarks. Some approaches beat others depending on the benchmark. Claiming it 'works' by tweaking it until it fits a canned benchmark is not hard work. Not having any explanation as to why it works is not hard work. Coming up with a functional systems architecture that ties the bits and pieces together is hard work. Understanding what is really happening in the human brain, how/why it is performing various functions, and how this provides for an intelligent architecture is hard work. Creating an 'aware' platform is hard and elusive work which is why people chase the low hanging fruit of optimization algorithms. *Cheers

deleted 10y ago

[deleted]

akssri 10y ago

Do you know if CompNeuro models have been trained to do things interesting to CS folk ?