3 ms·

Huh. Optics are modeled by differential equations. They describe a massively parallel phenomenon, light waves passing through and being modified by layers of m

Huh. Optics are modeled by differential equations. They describe a massively parallel phenomenon, light waves passing through and being modified by layers of media that, essentially, performs calculations using them. Warps and scales and so on. Clearly these particular differential equations are highly robust to architectural changes. There's a parallel there to something else I've read about on Hacker News... what are those things called? Oh yeah, deep networks. Are there any deep nets out there that attempt to mimic optics? or is anyone applying ideas from robust differential equations to deep nets?

hydrox24 8y ago
This article[0] appeared on HN the other day. I think it describes Richard Feynman using exactly the approach your talking about with an early distributed/parallel architecture. See the eighth paragraph of the "An Algorithm For Logarithms" section. [0]: http://longnow.org/essays/richard-feynman-and-connection-machine/ http://longnow.org/essays/richard-feynman-and-connection-mac...
romwell 8y ago
The words you are looking for are "Fourier transform". Effectively, the lens computes a Fourier Transform for you[1] One can probably compute a FT with NN, but we already have FFT :) [1]http://web.mit.edu/2.710/Fall06/2.710-wk10-a-sl.pdf http://web.mit.edu/2.710/Fall06/2.710-wk10-a-sl.pdf
bnegreve 8y ago
There are Moczulski et al. with their ACDC paper [1]. They describe a structured layer that can approximate any linear transform and discuss an optical implementation of it (see section 1.1 "lightning fast deep sell" :). Our deep SELL offers several possibilities for analog physical implementation. Given the great demand for fast low energy neural networks, the possibility of harnessing physical phenomena to perform efficient computation in deep networks is worthy of consideration.... (see paper) [1] https://arxiv.org/pdf/1511.05946.pdf https://arxiv.org/pdf/1511.05946.pdf
mateo1 8y ago
>Clearly these particular differential equations are highly robust to architectural changes. Not really. There's an incredible amount of redundancy on the information received by the lens, because the exposure time needs to be short. The person writing the article purposefully didn't take pictures at higher apertures and different focus distances. Optics aren't modeled by differential equations, and if you want to model optics you need to choose a model that depends on your application. Modelling lens optics with quantum mechanics alone would be a massive pain in the ass, but completely ignoring them would also give you horribly wrong ideas. Also I don't really get by what you mean with "mimicking optics". Mimic which functions exactly?
dharma1 8y ago
Not directly that but there are companies working on optical computing for machine learning https://www.optalysys.com https://www.optalysys.com http://www.lighton.io http://www.lighton.io

hydrox24 8y ago

This article[0] appeared on HN the other day. I think it describes Richard Feynman using exactly the approach your talking about with an early distributed/parallel architecture. See the eighth paragraph of the "An Algorithm For Logarithms" section. [0]: http://longnow.org/essays/richard-feynman-and-connection-machine/ http://longnow.org/essays/richard-feynman-and-connection-mac...

romwell 8y ago

The words you are looking for are "Fourier transform". Effectively, the lens computes a Fourier Transform for you[1] One can probably compute a FT with NN, but we already have FFT :) [1]http://web.mit.edu/2.710/Fall06/2.710-wk10-a-sl.pdf http://web.mit.edu/2.710/Fall06/2.710-wk10-a-sl.pdf

bnegreve 8y ago

There are Moczulski et al. with their ACDC paper [1]. They describe a structured layer that can approximate any linear transform and discuss an optical implementation of it (see section 1.1 "lightning fast deep sell" :). Our deep SELL offers several possibilities for analog physical implementation. Given the great demand for fast low energy neural networks, the possibility of harnessing physical phenomena to perform efficient computation in deep networks is worthy of consideration.... (see paper) [1] https://arxiv.org/pdf/1511.05946.pdf https://arxiv.org/pdf/1511.05946.pdf

mateo1 8y ago

>Clearly these particular differential equations are highly robust to architectural changes. Not really. There's an incredible amount of redundancy on the information received by the lens, because the exposure time needs to be short. The person writing the article purposefully didn't take pictures at higher apertures and different focus distances. Optics aren't modeled by differential equations, and if you want to model optics you need to choose a model that depends on your application. Modelling lens optics with quantum mechanics alone would be a massive pain in the ass, but completely ignoring them would also give you horribly wrong ideas. Also I don't really get by what you mean with "mimicking optics". Mimic which functions exactly?

dharma1 8y ago

Not directly that but there are companies working on optical computing for machine learning https://www.optalysys.com https://www.optalysys.com http://www.lighton.io http://www.lighton.io