8 ms·

GHz spiking neuromorphic photonic chip with in-situ training

115 points by juanviera23 1y ago

rf15 1y ago
Appreciating that not everyone tries to optimise for LLMs and we are still doing things like this. If you're looking at HN alone, it sometimes feels like the hype could drown out everything else.
- danielbln 1y ago
  There is massive hype, no doubt about it, but lets also not forget how LLMs have basically solved NLP, are a step change in many dimensions and are disrupting and changing things like software engineering like nothing else before it. So I hear you, but on the flip side we _should_ be reading a lot about LLMs here, as they have a direct impact on the work that most of us do. That said, seeing other papers pop up that are not related to transformer based networks is appreciated.
  - larodi 1y ago
    Thank you, brother. Besides not all that goes in HN is strictly LLM, really dunno why the scare.
- karanveer 1y ago
  I couldnt agree more.
msgodel 1y ago
It's just a single linear layer and it's not clear to me that the technology is capable of anything more. If I'm reading it correctly it sounds like running the model forward couldn't even use the technology, they had to record the weights and do it the old fashion way.
- roflmaostc 1y ago
  Would you have discredited early AI work because they could only train and compute a couple of weights? This is about first prototypes and scaling is often easier than the basic principle.
  - msgodel 1y ago
    Is this actually capable of propagating the gradient and training more complex layers though? A lot of these novel AI accelerators run into problems like that because they're not capable of general purpose computing. A good example of that are the boltzman machines on Dwave's stuff. Yeah it can do that but it can only do that because the machine is only capable of doing QUBO.
    - roflmaostc 1y ago
      For inference we do not care about training, right? But if we could make cheaper inference machines available, everyone would profit. Isn't it that LLMs use more energy in inference than training these days?
fjfaase 1y ago
Nice that they can do the processing in the GHz range, but from some pictures in the paper, it seems the system has only 60 'cells', which is rather low compared to the number of cells found in brains of animals that display complex behavior. To me it seems this is an optimization in the wrong dimension.
- _jab 1y ago
  I suspect practicality is not the goal here, but rather a proof of concept. Perhaps they saw speed as an important technical barrier to cross
khalic 1y ago
A lot of unrigorous claims for an abstract…
kadushka 1y ago
Maybe try simulating the algorithms in software before building hardware? People have been trying to get spiking networks to work for several decades now, with zero success. If it does not work in software, it's not going to work in hardware.
- vessenes 1y ago
  This seems to work in hardware, per the paper. At least to 80% accuracy.
- good_stuffs 1y ago
  >If it does not work in software, it's not going to work in hardware. Aren't there limits to what can be simulated in software? Analog systems dealing with infinite precision, and having large numbers of connections between neurons is bound to hit the von Neumann bottleneck for classical computers where memory and compute are separate?
  - naasking 1y ago
    It's not clear that "infinite precision" is a meaningful thing. All inputs and outputs even to analog systems will only ever have finite precision.
- juliangamble 1y ago
  “Zero success” seems a bit strong. People have been able to get 96% accuracy on MINST digits on their local machine. https://norse.github.io/notebooks/mnist_classifiers.html https://norse.github.io/notebooks/mnist_classifiers.html I think it may be more accurate to say “1970s level neural net performance”. The evidence suggests it is a nascent field of research.
cwmoore 1y ago
Retina-inspired video recognition using light. Cool. May be a visual cortex next year.
vessenes 1y ago
Ghz speed video processing, even if we only get very basic segmentation or recognition out of it, is probably crazy useful. Need to face recognize every seat at a stadium? Well, if you have enough cameras, 60,000 seats could be scanned 250 thousand times a second. Or if you want to scan a second of video at 60fps, you'd be able to check all of them at a mere 4 thousand times a second. Anyway, good to see interesting raw research. I imagine there are a number of military and security use cases here that could fund something to market (at least a small initial market).

115 points by juanviera23 1y ago

rf15 1y ago

Appreciating that not everyone tries to optimise for LLMs and we are still doing things like this. If you're looking at HN alone, it sometimes feels like the hype could drown out everything else.

danielbln 1y ago
There is massive hype, no doubt about it, but lets also not forget how LLMs have basically solved NLP, are a step change in many dimensions and are disrupting and changing things like software engineering like nothing else before it. So I hear you, but on the flip side we _should_ be reading a lot about LLMs here, as they have a direct impact on the work that most of us do. That said, seeing other papers pop up that are not related to transformer based networks is appreciated.
- larodi 1y ago
  Thank you, brother. Besides not all that goes in HN is strictly LLM, really dunno why the scare.
karanveer 1y ago
I couldnt agree more.

msgodel 1y ago

It's just a single linear layer and it's not clear to me that the technology is capable of anything more. If I'm reading it correctly it sounds like running the model forward couldn't even use the technology, they had to record the weights and do it the old fashion way.

roflmaostc 1y ago
Would you have discredited early AI work because they could only train and compute a couple of weights? This is about first prototypes and scaling is often easier than the basic principle.
- msgodel 1y ago
  Is this actually capable of propagating the gradient and training more complex layers though? A lot of these novel AI accelerators run into problems like that because they're not capable of general purpose computing. A good example of that are the boltzman machines on Dwave's stuff. Yeah it can do that but it can only do that because the machine is only capable of doing QUBO.
  - roflmaostc 1y ago
    For inference we do not care about training, right? But if we could make cheaper inference machines available, everyone would profit. Isn't it that LLMs use more energy in inference than training these days?

fjfaase 1y ago

Nice that they can do the processing in the GHz range, but from some pictures in the paper, it seems the system has only 60 'cells', which is rather low compared to the number of cells found in brains of animals that display complex behavior. To me it seems this is an optimization in the wrong dimension.

_jab 1y ago
I suspect practicality is not the goal here, but rather a proof of concept. Perhaps they saw speed as an important technical barrier to cross

khalic 1y ago

A lot of unrigorous claims for an abstract…

kadushka 1y ago

Maybe try simulating the algorithms in software before building hardware? People have been trying to get spiking networks to work for several decades now, with zero success. If it does not work in software, it's not going to work in hardware.

vessenes 1y ago
This seems to work in hardware, per the paper. At least to 80% accuracy.
good_stuffs 1y ago
>If it does not work in software, it's not going to work in hardware. Aren't there limits to what can be simulated in software? Analog systems dealing with infinite precision, and having large numbers of connections between neurons is bound to hit the von Neumann bottleneck for classical computers where memory and compute are separate?
- naasking 1y ago
  It's not clear that "infinite precision" is a meaningful thing. All inputs and outputs even to analog systems will only ever have finite precision.
juliangamble 1y ago
“Zero success” seems a bit strong. People have been able to get 96% accuracy on MINST digits on their local machine. https://norse.github.io/notebooks/mnist_classifiers.html https://norse.github.io/notebooks/mnist_classifiers.html I think it may be more accurate to say “1970s level neural net performance”. The evidence suggests it is a nascent field of research.

cwmoore 1y ago

Retina-inspired video recognition using light. Cool. May be a visual cortex next year.

vessenes 1y ago

Ghz speed video processing, even if we only get very basic segmentation or recognition out of it, is probably crazy useful. Need to face recognize every seat at a stadium? Well, if you have enough cameras, 60,000 seats could be scanned 250 thousand times a second. Or if you want to scan a second of video at 60fps, you'd be able to check all of them at a mere 4 thousand times a second. Anyway, good to see interesting raw research. I imagine there are a number of military and security use cases here that could fund something to market (at least a small initial market).