5 ms·

The Nengo Neural Simulator

96 points by indescions_2018 8y ago

kirillseva 8y ago
Nengo is a very exciting technology. Wonder if they are working with Intel to make nendo models run on the announced spiking neural network processor
- aurelian15 8y ago
  Student from the lab that originally developed Nengo here. Yes! As far as I know, the company that develops Nengo (ABR) [1] is working on this with Intel [2]. Also see the presentation "Building applications with next generation neuromorphic hardware" from the NICE 2018 workshop [2] (video is available, password is on the top of the page). [1] https://appliedbrainresearch.com/ https://appliedbrainresearch.com/ [2] https://appliedbrainresearch.com/pressreleases/02-nice-loihi/ https://appliedbrainresearch.com/pressreleases/02-nice-loihi... [3] http://niceworkshop.org/nice-2018-agenda/ http://niceworkshop.org/nice-2018-agenda/
  - Uberphallus 8y ago
    Are there any inner details in Loihi available? I'm currently working on something I can't speak much about, but it involves heterogeneous NN, different ICs for training and for operation. Operation is that way simpler, and hence significantly faster and more paralleliable, as removes useless logic and the complexity of the "too many clock domains" hell on backpropagation networks.
    - orbifold 8y ago
      There are two published articles about it one describing the hardware architecture, the other describing the software architecture.
BucketSort 8y ago
How does this differ from something like TensorFlow? "Nengo is used to implement networks for deep learning, vision.." is this true? Does this system even support things like automatic symbolic differentiation? This seems more like a neuro science research tool than a practical framework for building neural networks.
- aurelian15 8y ago
  Student from the lab that originally developed Nengo here. You're correct in the assumption that Nengo is ‒ among others ‒ a neuroscience research tool; primarily, because it allows you to build networks of spiking neurons. Apart from neuroscience you can also do things like mapping pre-trained deep neural networks onto spiking neurons. These networks can then be executed on neuromorphic hardware. However, you can also run your simulations with "regular" rate-based neurons. In this case, the interface that Nengo provides is significantly more low-level than that of TensorFlow. Still, the main difference to TensorFlow is that Nengo networks are dynamic, i.e. the network is not described in terms of discrete time steps, but as a fully dynamical system (though a numerical simulator will discretise time at some point). In conjunction with neuromorphic hardware this allows things like real-time motor control with (sub-)millisecond resolution.
  - psandersen 8y ago
    That sounds pretty interesting, would you happen to know if these can be used to simulate the worm connectome (OpenWorm?) linked up in a virtual environment?
    - Seanny123 8y ago
      Other student from the lab here. You could, but it might not be particularly useful. They already have a really good simulation environment and ways to connect it to virtual environments. It would be useful if you wanted to extend the model with further neural populations!
      - 8y ago
phonebucket 8y ago
Refreshing that they use an OpenCL backend instead of CUDA. Would love for OpenCL to grow more to challenge CUDA’s dominance in scientific computing.
- pixelHD 8y ago
  True! The fact that you'd be able to use pretty much any GPU - even integrated/AMD/NVidia would really help. Speaking of which, doesn't the new Vulkan API include compute too?
  - grovesNL 8y ago
    Yes, Vulkan includes compute capabilities. For example, here is a relevant section about compute pipelines in the Vulkan specification: https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#pipelines-compute https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspe...
internetman55 8y ago
Nengo is neat, fun stuff if you are interested in neurons or whatever

Nengo is a very exciting technology. Wonder if they are working with Intel to make nendo models run on the announced spiking neural network processor

aurelian15 8y ago
Student from the lab that originally developed Nengo here. Yes! As far as I know, the company that develops Nengo (ABR) [1] is working on this with Intel [2]. Also see the presentation "Building applications with next generation neuromorphic hardware" from the NICE 2018 workshop [2] (video is available, password is on the top of the page). [1] https://appliedbrainresearch.com/ https://appliedbrainresearch.com/ [2] https://appliedbrainresearch.com/pressreleases/02-nice-loihi/ https://appliedbrainresearch.com/pressreleases/02-nice-loihi... [3] http://niceworkshop.org/nice-2018-agenda/ http://niceworkshop.org/nice-2018-agenda/
- Uberphallus 8y ago
  Are there any inner details in Loihi available? I'm currently working on something I can't speak much about, but it involves heterogeneous NN, different ICs for training and for operation. Operation is that way simpler, and hence significantly faster and more paralleliable, as removes useless logic and the complexity of the "too many clock domains" hell on backpropagation networks.
  - orbifold 8y ago
    There are two published articles about it one describing the hardware architecture, the other describing the software architecture.

BucketSort 8y ago

How does this differ from something like TensorFlow? "Nengo is used to implement networks for deep learning, vision.." is this true? Does this system even support things like automatic symbolic differentiation? This seems more like a neuro science research tool than a practical framework for building neural networks.

aurelian15 8y ago
Student from the lab that originally developed Nengo here. You're correct in the assumption that Nengo is ‒ among others ‒ a neuroscience research tool; primarily, because it allows you to build networks of spiking neurons. Apart from neuroscience you can also do things like mapping pre-trained deep neural networks onto spiking neurons. These networks can then be executed on neuromorphic hardware. However, you can also run your simulations with "regular" rate-based neurons. In this case, the interface that Nengo provides is significantly more low-level than that of TensorFlow. Still, the main difference to TensorFlow is that Nengo networks are dynamic, i.e. the network is not described in terms of discrete time steps, but as a fully dynamical system (though a numerical simulator will discretise time at some point). In conjunction with neuromorphic hardware this allows things like real-time motor control with (sub-)millisecond resolution.
- psandersen 8y ago
  That sounds pretty interesting, would you happen to know if these can be used to simulate the worm connectome (OpenWorm?) linked up in a virtual environment?
  - Seanny123 8y ago
    Other student from the lab here. You could, but it might not be particularly useful. They already have a really good simulation environment and ways to connect it to virtual environments. It would be useful if you wanted to extend the model with further neural populations!
    - 8y ago

phonebucket 8y ago

Refreshing that they use an OpenCL backend instead of CUDA. Would love for OpenCL to grow more to challenge CUDA’s dominance in scientific computing.

pixelHD 8y ago
True! The fact that you'd be able to use pretty much any GPU - even integrated/AMD/NVidia would really help. Speaking of which, doesn't the new Vulkan API include compute too?
- grovesNL 8y ago
  Yes, Vulkan includes compute capabilities. For example, here is a relevant section about compute pipelines in the Vulkan specification: https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#pipelines-compute https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspe...

internetman55 8y ago

Nengo is neat, fun stuff if you are interested in neurons or whatever