4 ms·

This article was pretty low on details, so I went out and collected some links: This looks more or less like another stab at the Cray connection machine[0], bu

This article was pretty low on details, so I went out and collected some links: This looks more or less like another stab at the Cray connection machine[0], but with modern hardware and a better framework about how neural nets can and should work. > The SpiNNaker engine is a massively-parallel multi-core computing system. It will contain up to 1,036,800 ARM9 cores and 7Tbytes of RAM distributed throughout the system in 57K nodes, each node being a System-in-Package (SiP) containing 18 cores plus a 128Mbyte off-die SDRAM (Synchronous Dynamic Random Access Memory). Each core has associated with it 64Kbytes of data tightly-coupled memory (DTCM) and 32Kbytes of instruction tightly-coupled memory (ITCM). The cores have a variety of ways of communicating with each other and with the memory, the dominant of which is by packets. These are 5- or 9-byte (40- or 72-bit) quanta of information that are transmitted around the system under the aegis of a bespoke concurrent hardware routing system. [1] So, lots of relatively relatively tiny, interconnected nodes. They built their own SoC to handle this. With a built-in router in the middle. The router handles routing on the chip, and multicasts to its neighbors. > The heart of the communications infrastructure is a bespoke multicast router that is able to replicate packets where necessary to implement the multicast function associated with sending the same packet to several different destinations. [2] It also looks like they're developing dev boards [3] So basically, this looks like a giant, really awesome, custom ARM cluster that they want to do neural network stuff with. If anyone from the team is here, I'd love to hear more about how this will be used. Specifically, how will you prevent SpiNNaker from going down the same path as the Connection Machine - (stops doing AI stuff because, say, geneticists want to use it for protein sequencing)? Why do you see this as the future over something like NVIDIA's new HGX-2 or clusters of TPUs? [0] https://en.wikipedia.org/wiki/Connection_Machine https://en.wikipedia.org/wiki/Connection_Machine [1] http://apt.cs.manchester.ac.uk/projects/SpiNNaker/architecture/ http://apt.cs.manchester.ac.uk/projects/SpiNNaker/architectu... [2] http://apt.cs.manchester.ac.uk/projects/SpiNNaker/SpiNNchip/ http://apt.cs.manchester.ac.uk/projects/SpiNNaker/SpiNNchip/ [3] http://apt.cs.manchester.ac.uk/projects/SpiNNaker/hardware/index3.php http://apt.cs.manchester.ac.uk/projects/SpiNNaker/hardware/i...

digsmahler 8y ago
Thank you, that really helps! The original article alone is pretty confusing.
neuromantik8086 8y ago
Just as a bit of a nitpick the Connection Machine: a) Wasn't manufactured by Cray. It was made by Thinking Machines Corporation in the greater Boston area. b) Didn't have anything to do with neural nets, as it was developed during the period of time when GOFAI / symbolic AI was still in vogue (although by the late 80s the Japanese had revived connectionism / neural nets), and thus had far more in common with a LISP machine. c) Was mostly about developing a decent SIMD architecture.
dkfellows 8y ago
> Specifically, how will you prevent SpiNNaker from going down the same path as the Connection Machine - (stops doing AI stuff because, say, geneticists want to use it for protein sequencing)? I'm on the team. I can say that we're specifically funded to do and support computational neuroscience. However, if someone comes along with money wanting to do other kinds of projects on the hardware (and are able to handle the special characteristics of the machine itself) then they're welcome. The challenge is that it's non-conventional in a number of ways that make porting code tricky: in particular, the messages are small, designed to be multicast rather than unicast, the instruction memory per core is only 32kB, and there's no hardware floating point at all in the current generation. (You can do floating point in emulation. We do that in one of our projects.) > Why do you see this as the future over something like NVIDIA's new HGX-2 or clusters of TPUs? We see it as solving different problems. Those approaches you mention are great for solving problems that resolve to big matrix operations; SpiNNaker is better at tackling problems that are dominated in terms of description by communication. Neural simulations are really just vast hybrid ODE systems, but where it is possible to break up the simulation into lots of communicating domains (the synapses and neurons).
- peterlk 8y ago
  I'm a bit late in checking up on this, but just wanted to say thanks for taking the time to answer some questions

digsmahler 8y ago

Thank you, that really helps! The original article alone is pretty confusing.

neuromantik8086 8y ago

Just as a bit of a nitpick the Connection Machine: a) Wasn't manufactured by Cray. It was made by Thinking Machines Corporation in the greater Boston area. b) Didn't have anything to do with neural nets, as it was developed during the period of time when GOFAI / symbolic AI was still in vogue (although by the late 80s the Japanese had revived connectionism / neural nets), and thus had far more in common with a LISP machine. c) Was mostly about developing a decent SIMD architecture.

dkfellows 8y ago

> Specifically, how will you prevent SpiNNaker from going down the same path as the Connection Machine - (stops doing AI stuff because, say, geneticists want to use it for protein sequencing)? I'm on the team. I can say that we're specifically funded to do and support computational neuroscience. However, if someone comes along with money wanting to do other kinds of projects on the hardware (and are able to handle the special characteristics of the machine itself) then they're welcome. The challenge is that it's non-conventional in a number of ways that make porting code tricky: in particular, the messages are small, designed to be multicast rather than unicast, the instruction memory per core is only 32kB, and there's no hardware floating point at all in the current generation. (You can do floating point in emulation. We do that in one of our projects.) > Why do you see this as the future over something like NVIDIA's new HGX-2 or clusters of TPUs? We see it as solving different problems. Those approaches you mention are great for solving problems that resolve to big matrix operations; SpiNNaker is better at tackling problems that are dominated in terms of description by communication. Neural simulations are really just vast hybrid ODE systems, but where it is possible to break up the simulation into lots of communicating domains (the synapses and neurons).

peterlk 8y ago
I'm a bit late in checking up on this, but just wanted to say thanks for taking the time to answer some questions