2 ms·

The key idea is you can use physical effects such as attenuating light to implement multiplication in a way that uses much less energy than the fairly complex a

The key idea is you can use physical effects such as attenuating light to implement multiplication in a way that uses much less energy than the fairly complex arrangement of digital logic necessary to implement the same operation. This is roughly the method they use in the paper: https://en.wikipedia.org/wiki/Spatial_light_modulator https://en.wikipedia.org/wiki/Spatial_light_modulator I've seen a lab bench prototype of a different implementation, there are a lot of engineering problems to solve but as the paper points out the potential payoff is big. Edit: The other key point is that one of the expensive components in transformers is effectively a giant matrix multiplication which implies many, many individual multiplications.

theptip 4y ago
Do you have insight into how this compares to equivalent silicon ASICS for the matrix mult operations?
- hedgehog 4y ago
  The optical stuff is still research territory and too early to be comparable to the current digital silicon. Lots of open questions about how to implement multiplication, implement activation functions, where to store weights, how to move data in and out, manufacturability of all of this, etc. Basically it's all questions. To get some intuition about the promise imagine being able to implement the weights of a layer (fully connected layer is essentially vector matrix multiplication) as a 2D hologram and the compute as pushing an image from an OLED display through to an image sensor. Multiplication as attenuation in the hologram, summation is just the accumulation of charge on the sensor side. Everything happens all at once and the number of photons required is potentially very small. An actual working implementation would be both more clever and more practical. The potential to do every multiply in parallel for almost no energy is so attractive that I expect people to chip away at this problem for the foreseeable future.

theptip 4y ago

Do you have insight into how this compares to equivalent silicon ASICS for the matrix mult operations?

hedgehog 4y ago
The optical stuff is still research territory and too early to be comparable to the current digital silicon. Lots of open questions about how to implement multiplication, implement activation functions, where to store weights, how to move data in and out, manufacturability of all of this, etc. Basically it's all questions. To get some intuition about the promise imagine being able to implement the weights of a layer (fully connected layer is essentially vector matrix multiplication) as a 2D hologram and the compute as pushing an image from an OLED display through to an image sensor. Multiplication as attenuation in the hologram, summation is just the accumulation of charge on the sensor side. Everything happens all at once and the number of photons required is potentially very small. An actual working implementation would be both more clever and more practical. The potential to do every multiply in parallel for almost no energy is so attractive that I expect people to chip away at this problem for the foreseeable future.