6 ms·

The future of deep learning is photonic

103 points by pcaversaccio 5y ago

varelse 5y ago
By the time any of this is viable commercially, I really wonder where commercial GPUs and accelerators will be. And they're going to have the home team advantage when that happens. So that means that unless these things are as accessible to tensorflow/pytorch or whatever the heck the framework de jour is at that point (hopefully more like Jax but better), they will get no traction. Evidence? Every single demonstrably superior CPU architecture that failed to dislodge x86 over the past 50 years. Sure, it's finally happening, but it's also 50 years later.
- kadoban 5y ago
  Better CPU arches were never faster, definitely not by much.
  - varelse 5y ago
    Going to disagree. I ran circles around contemporary x86 back in the early '90s because of specific instructions Intel denied they would ever need in their processor roadmap. But it really didn't matter and that's one of the most important lessons of my career. They did similar goofy thinking with respect to the magic transcendental unit on GPUs so it's not like they ever learned. It's not entirely about clock rate.
- etaioinshrdlu 5y ago
  Luckily chip companies know this now and all the alternative deep learning accelerators have some level of support for mainstream frameworks. Getting their support upstreamed to the main framework project is another matter, though, as it reaching the quality of implementation as CUDA and CPU...
- jorgBaller 5y ago
  Hint: Look at the performance delta between the last 3 generatons of nvidia gpus, keeping the TDP fixed. Current accelerators have nice tricks like reducing precision, adding more tensor multipliers etc but thats it. ALl they can do afte r these optimisations is scaling down the feature sizes which will eventually be limited by physics in a few generations. Then what? A fundamentally diferent approach is required
- Salgat 5y ago
  We've already seen disruptive architectures like Google's Tensor Processing Units. x86 has already been upended for ML, and photonics based processing units will simply be another PCI card you plug into your computer just like a GPU.
  - AbrahamParangi 5y ago
    How disruptive are TPUs really though? My understanding is that essentially everything is still trained on Nvidia architecture.
    - varelse 5y ago
      If you design your network on a TPU, you will tend to use operators that work well on a TPU. And in the end you will have a network that works best on the TPU. Lather rinse repeat for any other architecture. You can even make a network that runs best on Graphcore that way, but it won't be fun to do it. You might even get Graphcore to pay top dollar for it though as they both need some good publicity and they have lots of VC left to squander. This also tends to be true of video games where the platform on which they were developed is the best place to play them rather than their many ports.
      - mmmBacon 5y ago
        We’ve been doing this for years with DSP and networking. So kind of ho-hum from a HW perspective. If you ask me the thing that makes these things even remotely interesting is the willingness from the SW side to support new HW architectures. Without that you can’t have any innovation in HW.
    - singularity2001 5y ago
      That's only because google is stupidly refusing to sell their devices
- htkibar 5y ago
  Not to mention it is only happening due to the emergence of mobile with a different architecture (new field, existing moat is meaningless etc etc)
dogma1138 5y ago
Lightmatter is apparently shipping their photonic accelerator to early adopters right now https://lightmatter.co/products/envise/ https://lightmatter.co/products/envise/ They claim quite big performance uplift over current GPUs and even better performance per watt figures. However I haven’t seen independent benchmarks for their Envise ASIC yet. They seem to be using a different method with an external laser to “power” the chips using fiber optic cables rather than integrating solid state lasers into the silicon. It’s also not based on LCD technology like early attempts at photonic computing which was essentially a sandwich of LCD screens and solid state detectors and amplifiers.
- TaylorAlexander 5y ago
  Yep and the CEO claimed 10x faster than current tech while using 10% the electrical power. If it’s really 100x more efficient that is huge. But it’s the CEO saying it so hard to say how true it is. https://youtu.be/t1R7ElXEyag https://youtu.be/t1R7ElXEyag
  - dogma1138 5y ago
    It’s an interesting tech however the big issue with photonics is that building complex logic and memory is hard to near impossible it’s quite good at doing relatively basic operations at scale which is why the tech can work with as simple components as a bunch of LCD screens that hold your input values as a pixel mask and a detector in the end, since the LCD matrix can be quite big say 1024x1024 pixels you are able to do basic but wise ops like XOR on huge matrices the challenge was always to A) get photonics to be fast enough where their scale can outpace the switching frequency of traditional semiconductors and B) get to a point where you can actually do useful operations. My gut feeling is that Envise can actually do only a few basic operations in photonics, however if these operations are sufficiently tailored for specific tasks in ML that might be good enough. It’s not that different than what NVIDIA did with their Tensor cores they aren’t general purpose ALUs they only do a few things but they are very fast at those tasks.
    - mmmBacon 5y ago
Havoc 5y ago
Can we make fiber optics cables smaller than 2nm? Or is the game plan to make them bigger & nobody cares cause no heat & fast?
- jhgb 5y ago
  How could you make a 2nm optics cable? The wavelengths are two orders of magnitude larger than that.
- TaylorAlexander 5y ago
  I don’t think the size of silicon transistors has anything to do with this. There is no direct comparison. So they will prob bigger but no one cares.
- dasudasu 5y ago
  Photonic integrated circuits are almost as old as regular integrated circuits.
- adrusi 5y ago
  Not even remotely an expert on chip design, but deep learning dataflow is a lot more predictable and linear than what a CPU or even a GPU doing actual graphics needs to do. Speed of light latency is probably not an issue since the relevant distances are not one side of the die to the other but rather the distance between adjacent components.
civilized 5y ago
People have been talking about photonics for decades. Is photonics actually well-suited to deep learning? Or is this just another "new thing is happening, time to remind people of the old thing that keeps never happening" take?
- dasudasu 5y ago
  Nobody was talking seriously about photonic accelerators 10 years ago. Optical computing last had a hype cycle in the late 80s/early 90s.
BeeBoBub 5y ago
It is true that photonic systems can multiply/add very efficiently. The limiting factor for optical neural networks are activation functions. Non-linear relationships are hard for photonics. State-of-the-art nonlinear features require optical elements that are difficult to manufacture or op-amps which slow the computational potential of the optical neural network by orders of magnitude.

103 points by pcaversaccio 5y ago

By the time any of this is viable commercially, I really wonder where commercial GPUs and accelerators will be. And they're going to have the home team advantage when that happens. So that means that unless these things are as accessible to tensorflow/pytorch or whatever the heck the framework de jour is at that point (hopefully more like Jax but better), they will get no traction. Evidence? Every single demonstrably superior CPU architecture that failed to dislodge x86 over the past 50 years. Sure, it's finally happening, but it's also 50 years later.

kadoban 5y ago
Better CPU arches were never faster, definitely not by much.
- varelse 5y ago
  Going to disagree. I ran circles around contemporary x86 back in the early '90s because of specific instructions Intel denied they would ever need in their processor roadmap. But it really didn't matter and that's one of the most important lessons of my career. They did similar goofy thinking with respect to the magic transcendental unit on GPUs so it's not like they ever learned. It's not entirely about clock rate.
etaioinshrdlu 5y ago
Luckily chip companies know this now and all the alternative deep learning accelerators have some level of support for mainstream frameworks. Getting their support upstreamed to the main framework project is another matter, though, as it reaching the quality of implementation as CUDA and CPU...
jorgBaller 5y ago
Hint: Look at the performance delta between the last 3 generatons of nvidia gpus, keeping the TDP fixed. Current accelerators have nice tricks like reducing precision, adding more tensor multipliers etc but thats it. ALl they can do afte r these optimisations is scaling down the feature sizes which will eventually be limited by physics in a few generations. Then what? A fundamentally diferent approach is required
Salgat 5y ago
We've already seen disruptive architectures like Google's Tensor Processing Units. x86 has already been upended for ML, and photonics based processing units will simply be another PCI card you plug into your computer just like a GPU.
- AbrahamParangi 5y ago
  How disruptive are TPUs really though? My understanding is that essentially everything is still trained on Nvidia architecture.
  - varelse 5y ago
    If you design your network on a TPU, you will tend to use operators that work well on a TPU. And in the end you will have a network that works best on the TPU. Lather rinse repeat for any other architecture. You can even make a network that runs best on Graphcore that way, but it won't be fun to do it. You might even get Graphcore to pay top dollar for it though as they both need some good publicity and they have lots of VC left to squander. This also tends to be true of video games where the platform on which they were developed is the best place to play them rather than their many ports.
    - mmmBacon 5y ago
      We’ve been doing this for years with DSP and networking. So kind of ho-hum from a HW perspective. If you ask me the thing that makes these things even remotely interesting is the willingness from the SW side to support new HW architectures. Without that you can’t have any innovation in HW.
  - singularity2001 5y ago
    That's only because google is stupidly refusing to sell their devices
htkibar 5y ago
Not to mention it is only happening due to the emergence of mobile with a different architecture (new field, existing moat is meaningless etc etc)

dogma1138 5y ago

Lightmatter is apparently shipping their photonic accelerator to early adopters right now https://lightmatter.co/products/envise/ https://lightmatter.co/products/envise/ They claim quite big performance uplift over current GPUs and even better performance per watt figures. However I haven’t seen independent benchmarks for their Envise ASIC yet. They seem to be using a different method with an external laser to “power” the chips using fiber optic cables rather than integrating solid state lasers into the silicon. It’s also not based on LCD technology like early attempts at photonic computing which was essentially a sandwich of LCD screens and solid state detectors and amplifiers.

TaylorAlexander 5y ago
Yep and the CEO claimed 10x faster than current tech while using 10% the electrical power. If it’s really 100x more efficient that is huge. But it’s the CEO saying it so hard to say how true it is. https://youtu.be/t1R7ElXEyag https://youtu.be/t1R7ElXEyag
- dogma1138 5y ago
  It’s an interesting tech however the big issue with photonics is that building complex logic and memory is hard to near impossible it’s quite good at doing relatively basic operations at scale which is why the tech can work with as simple components as a bunch of LCD screens that hold your input values as a pixel mask and a detector in the end, since the LCD matrix can be quite big say 1024x1024 pixels you are able to do basic but wise ops like XOR on huge matrices the challenge was always to A) get photonics to be fast enough where their scale can outpace the switching frequency of traditional semiconductors and B) get to a point where you can actually do useful operations. My gut feeling is that Envise can actually do only a few basic operations in photonics, however if these operations are sufficiently tailored for specific tasks in ML that might be good enough. It’s not that different than what NVIDIA did with their Tensor cores they aren’t general purpose ALUs they only do a few things but they are very fast at those tasks.
  - mmmBacon 5y ago

Havoc 5y ago

Can we make fiber optics cables smaller than 2nm? Or is the game plan to make them bigger & nobody cares cause no heat & fast?

jhgb 5y ago
How could you make a 2nm optics cable? The wavelengths are two orders of magnitude larger than that.
TaylorAlexander 5y ago
I don’t think the size of silicon transistors has anything to do with this. There is no direct comparison. So they will prob bigger but no one cares.
dasudasu 5y ago
Photonic integrated circuits are almost as old as regular integrated circuits.
adrusi 5y ago
Not even remotely an expert on chip design, but deep learning dataflow is a lot more predictable and linear than what a CPU or even a GPU doing actual graphics needs to do. Speed of light latency is probably not an issue since the relevant distances are not one side of the die to the other but rather the distance between adjacent components.

civilized 5y ago

People have been talking about photonics for decades. Is photonics actually well-suited to deep learning? Or is this just another "new thing is happening, time to remind people of the old thing that keeps never happening" take?

dasudasu 5y ago
Nobody was talking seriously about photonic accelerators 10 years ago. Optical computing last had a hype cycle in the late 80s/early 90s.

BeeBoBub 5y ago

It is true that photonic systems can multiply/add very efficiently. The limiting factor for optical neural networks are activation functions. Non-linear relationships are hard for photonics. State-of-the-art nonlinear features require optical elements that are difficult to manufacture or op-amps which slow the computational potential of the optical neural network by orders of magnitude.