3 ms·

(When does this thread get closed by Hacker News as being too old? If/when it does is there somewhere public you want to continue the discussion? If you'd like

(When does this thread get closed by Hacker News as being too old? If/when it does is there somewhere public you want to continue the discussion? If you'd like we can move it to the Nengo forums at forum.nengo.ai) 1. Dammit. I misread that paper. You're totally right that they do show it works for multiple layers. 2. By faster hardware, I mean neuromorphic hardware such as BrainScaleS and Spinnaker. The software we use, Nengo, is pretty dependent on the speed of a single GPU, since it's really hard to separate the networks across multiple GPUs. You're right that there's always the possibility that our newer algorithms don't map well onto the specialized hardware we've built. The reason why we think we're ready to at least implement a few hardware prototypes is: - The principles that underlie our algorithms, the Neural Engineering Framework, have been around for 15 years and are pretty mature. The software built to support these principles, Nengo, has been through six re-writes and if finally pretty stable. - Some of the hardware implementations are general enough that they can handle pretty drastic changes in algorithms. For example, Spinnaker is just a hexagonal grid of stripped-down ARM chips. - Even if the hardware ends up limiting what algorithms we can implement, we can probably re-use a lot of the design to implement whatever the new algorithms require. 3. I've been meaning to investigate the HTM theory of Numenta for years (they were actually the first people to get me excited about brain-motivated machine intelligence), but never got around to it. I'm also super unclear on the relation between HTM, the Neural Engineering Framework and the newer theory FORCE. I'll write a question on cogsci.stackexchange.com to motivate myself to dig in. Where is the internship happening? Will you be working with [Sughanda Sharma][1]? She's my lab-mate working on that using the Neural Engineering Framework. [1] http://compneuro.uwaterloo.ca/people/sugandha-sharma.html http://compneuro.uwaterloo.ca/people/sugandha-sharma.html

p1esk 10y ago
Sure, let's move to Nengo forums. Do you mind creating a topic there and sending me a link? I actually don't know much about spiking NNs (software or hardware), but Spaun seems like the only real competitor to HTM. Unlike Spaun, Numenta's algorithms are not currently limited by computational resources, because they focus on a rather small part of neocortex (2-3 layers of a single region, working with tiny input sizes), and they abstract spikes. Numenta claims that if we understand what spikes are doing (computation and information transfer), then there's not need to emulate them exactly, we can construct algorithms which do the same thing using traditional computations and data structures. Instead, HTM wants to understand what the layers do and how they interact. There have been an interesting discussion on Numenta mailing list, with James Smith, who is developing his own spikes-focused theory of the neural computation: http://lists.numenta.org/pipermail/nupic-theory_lists.numenta.org/2015-June/002846.html http://lists.numenta.org/pipermail/nupic-theory_lists.nument... I believe he has been working on it since, and plans to publish a book. The internship offer is from HRL, a small company in Malibu. It's quite possible that they are looking at the ideas of your lab-mate, and they might even ask me to implement them. I'm still deciding though.
- Seanny123 10y ago
  Here's the link to the new topic. I'll let you start the discussion there to make sure I've transferred between contexts correctly. https://forum.nengo.ai/t/the-usefulness-of-neuromorphic-hardware/257?u=seanny123 https://forum.nengo.ai/t/the-usefulness-of-neuromorphic-hard...

p1esk 10y ago

Sure, let's move to Nengo forums. Do you mind creating a topic there and sending me a link? I actually don't know much about spiking NNs (software or hardware), but Spaun seems like the only real competitor to HTM. Unlike Spaun, Numenta's algorithms are not currently limited by computational resources, because they focus on a rather small part of neocortex (2-3 layers of a single region, working with tiny input sizes), and they abstract spikes. Numenta claims that if we understand what spikes are doing (computation and information transfer), then there's not need to emulate them exactly, we can construct algorithms which do the same thing using traditional computations and data structures. Instead, HTM wants to understand what the layers do and how they interact. There have been an interesting discussion on Numenta mailing list, with James Smith, who is developing his own spikes-focused theory of the neural computation: http://lists.numenta.org/pipermail/nupic-theory_lists.numenta.org/2015-June/002846.html http://lists.numenta.org/pipermail/nupic-theory_lists.nument... I believe he has been working on it since, and plans to publish a book. The internship offer is from HRL, a small company in Malibu. It's quite possible that they are looking at the ideas of your lab-mate, and they might even ask me to implement them. I'm still deciding though.

Seanny123 10y ago
Here's the link to the new topic. I'll let you start the discussion there to make sure I've transferred between contexts correctly. https://forum.nengo.ai/t/the-usefulness-of-neuromorphic-hardware/257?u=seanny123 https://forum.nengo.ai/t/the-usefulness-of-neuromorphic-hard...