8 ms·

>According to LightOn, its Appliance can reach a peak performance of 1.5 PetaOPS at 30W TDP and can deliver performance that 8 to 40 times higher than GPU-only

>According to LightOn, its Appliance can reach a peak performance of 1.5 PetaOPS at 30W TDP and can deliver performance that 8 to 40 times higher than GPU-only acceleration. Impressive! LightOn hasn't received much discussion on here before. Some links have been submitted, and this is the only one I could find comments on: https://news.ycombinator.com/item?id=27797829 https://news.ycombinator.com/item?id=27797829

fsh 5y ago
From the website of the manufacturer [1] it appears that the co-processor is essentially an analog computer for matrix-vector multiplications. I am quite sceptical about the accuracy and value range of the computations. Even puny single-precision floating point operations are accurate to something like 7 decimal digits and have a dynamic range of hundreds of dB. According to the spec sheet, the appliance only uses 6-bit inputs and 8-bit outputs, so the relative errors are probably on the percent level. This makes it hard to believe that any signal will propagate through something like a DNN without completely drowning in noise. [1] https://lighton.ai/lighton-appliance/ https://lighton.ai/lighton-appliance/
- orlp 5y ago
  > Even puny single-precision floating point operations are accurate to something like 7 decimal digits and have a dynamic range of hundreds of dB. According to the spec sheet, the appliance only uses 6-bit inputs and 8-bit outputs, so the relative errors are probably on the percent level. This makes it hard to believe that any signal will propagate through something like a DNN without completely drowning in noise. Maybe you aren't aware, but half-precision (16 bit float) is already well-established in the AI community: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://en.wikipedia.org/wiki/Bfloat16_floating-point_format. In context single-precision isn't all that puny! And there have already been successful experiments with stronger quantization, like 8-bit neural nets, or even 1-bit (!) neural nets. There is a lot of evidence that neural networks can be very resilient to quantization noise.
  - ISL 5y ago
    I'd be real surprised if the neurons in our brains have ADC-equivalents better than ~4 bits.
    - tgv 5y ago
      True, but ANNs are nowhere near as good as our brains, nor do they operate in the same way.
    - xwolfi 5y ago
      I think you missed the fact we were talking about neural network, not an animal brain self replicating and branching and competing with itself for billions of years until it becomes aware of itself. Give us the the same time.
      - Retric 5y ago
        The earliest and simplest brains where still useful. Even insects can fly around in 3D space, I doubt you need something as complicated as a mouse brain to run a self driving car let alone a drone.
        ben_w 5y ago
        I’m not sure how much it matters given this thread looks like it’s going off on several successive tangents, but the important (and hard) thing with a self-driving car is making sure it doesn’t hit stuff, not the actual driving part. And drones, trivially agree: Megaphragma mymaripenne has 7400 neurones, compared to the 71/14 million in a house mouse nervous system/brain.
        robwwilliams 5y ago
        And to stay on this odd tangent: Best estimate I have for total cell numbers in mouse brain—about 75 million neurons and 35 million other cell types. This estimate is from a 476 mg brain of a C57BL/6J case—the standard mouse used by many researchers. Based on much other work with discrete neuron populations the range among different genometypes of mice probable +/- 40%. for details see: www.nervenet.org/papers/brainrev99.html Expect many more (and I hope better) estimates soon from Clarity/SHIELD whole brain lighsheet counting with Al Johnson and colleagues at Duke and team at Life Canvas Tech.
- kortex 5y ago
  It's not a transmission line though, SNR does not apply in the same way. It's more like CMOS where the signal is refreshed at each gate. Each stage of an ANN applies some weight and activation. You can think of each input vector as a vector with a true value plus some noise. As long as that feature stays within some bounds, it is going to represent the same "thought vector". It may require some architecture changes to make training feasible, but it's far from a nonstarter. And that is only considering backprop learning. The brain does not use backprop, and has way higher noise levels.
  - dahart 5y ago
    I think the parent was referring to the same noise that you are, compute precision, not transmission, and was suggesting that perhaps it won’t easily stay within bounds due to the fact that some kinds of repeated calculations lose more precision at every step. Maybe it’s application dependent, maybe NNs or other matrix-heavy domains can tolerate low precision much more easily than scientific simulations. It certainly wouldn’t surprise me if these “LightOPS” processors work well in a narrow range of applications, and won’t improve or speed up just anything that needs a matrix multiply.
- tdrdt 5y ago
  But arent there other applications where this is ok? For example path tracing (ray tracing) doesn't need to be very accurate because multiple samples per pixel are used. A gaming card that uses less power is very welcome in laptops for example.
  - moonchrome 5y ago
    What do you mean? What kind of worlds can you represent with 8 bits units ? Some small blocky voxel box ?
    - tdrdt 5y ago
      I mean inacurate vector math
      - moonchrome 5y ago
        But that is what I'm saying if your vectors are reduced to 8 bit scalar components you can represent a 256x256x256 worth of detail in the world (doesn't need to be linear but still really limited details) ?
        montjoy 5y ago
        I’m totally out of my expertise here but I have a question - from my understanding ray tracing is primarily used for lighting/shadows/reflections- wouldn’t it be OK for something like shadows to be inaccurate- maybe some sort of amalgamation over frame refreshes? Real world light is messy anyway. I’m talking about a game type scenario not something scientific. Maybe another way to ask is- we’re trying to simulate a real world “analog” scene - maybe using an analog processing technique could actually be quite faithful for generating it? Or not. Like I said I don’t understand too much of this.
        moonchrome 5y ago
        Think about it like this : you have a spaceship model fits in 256x256x256m - to get the maximum resolution while still fitting in 8 bits you would make each axis in 1m increments and you have 256 values. So you can't have sub 1m details in geometry. Floating point is different because you have an exponent so it's not linear, you can technically have larger scale, but you sacrifice even more precision in mantissa. Now I'm not sure what this 6/8 bit precision or analog precision means so I can't say with confidence, but if your scalars are that low precision you can't really do much. You could technically encode it with some fancy tricks like instead of storing coordinates for each vertex you store the delta from previous one etc. but I think this wouldn't work if the device was just some dumb analog matrix multiplier with baked logic. Also having low detail shadows creates visual artifacts, see this for example [1] [1] https://digitalrune.github.io/DigitalRune-Documentation/media/Shadow-Resolution.jpg https://digitalrune.github.io/DigitalRune-Documentation/medi...
- bjornsing 5y ago
  If there’s too much noise just lower the dropout probability from 50 to 30%. ;) Joking aside, it is interesting how much noise and quantization these neural networks can work with. I think there’s a lot of room for low precision noisy computation here.
robert_tweed 5y ago
For a few seconds I thought it was Lite-On, best known for their cheapo CD/DVD drives. Seems to be completely unconnected though.
- Datagenerator 5y ago
  Brings back memories of the Plextor automatic duplicator robot we had in company back in the nineties. Great times
zapdrive 5y ago
Nice. Can't wait for the new light based GPUs, all being grabbed by greedy crypto miners and me still using my 6 year old graphics card!
- JohnJamesRambo 5y ago
  Relief is on the horizon. Ethereum should switch to proof of stake in June 2022 and you are about to see an unholy torrent of used GPUs hit the market. I would expect you can pick up any you like for peanuts then.
  - demux 5y ago
    Ethereum PoW won't immediately disappear, and I'm sure bitcoin folk will be all too happy to grab those extra GPUs
    - vidarh 5y ago
      GPU mining Bitcoin hasn't been reasonable for many years unless you have very cheap energy and no access to ASIC miners.
  - zapdrive 5y ago
    I have been hearing "Ethereum is switching to proof of stake in a few months" for at least 6 years now. I don't think it's going to ever happen.
  - ohgodplsno 5y ago
    No, miners will just move to another valuable PoW shitcoin and/or fork ETH to stay on PoW.
inasio 5y ago
I've never heard of LightOn, and wish the website had a bit more concrete info on the specifics of the coprocessor, but I am somewhat familiar with a similar photonic coprocessor made by NTT (the Coherent Ising Machine). It's still in the research stage, the logic uses interferometry effects, and requires kilometers of fiber optic cables. Interestingly, there is a simulator based on mean field theory that runs on GPUs and FPGAs(*) that can solve some problems (e.g. SAT) with close to state of the art performance. (*) disclosure: my company helped build the simulator
- dasudasu 5y ago
  There are other startups in the space that do it in semiconductors. Look up Lightelligence and Lightmatter for instance.

fsh 5y ago

From the website of the manufacturer [1] it appears that the co-processor is essentially an analog computer for matrix-vector multiplications. I am quite sceptical about the accuracy and value range of the computations. Even puny single-precision floating point operations are accurate to something like 7 decimal digits and have a dynamic range of hundreds of dB. According to the spec sheet, the appliance only uses 6-bit inputs and 8-bit outputs, so the relative errors are probably on the percent level. This makes it hard to believe that any signal will propagate through something like a DNN without completely drowning in noise. [1] https://lighton.ai/lighton-appliance/ https://lighton.ai/lighton-appliance/

orlp 5y ago
> Even puny single-precision floating point operations are accurate to something like 7 decimal digits and have a dynamic range of hundreds of dB. According to the spec sheet, the appliance only uses 6-bit inputs and 8-bit outputs, so the relative errors are probably on the percent level. This makes it hard to believe that any signal will propagate through something like a DNN without completely drowning in noise. Maybe you aren't aware, but half-precision (16 bit float) is already well-established in the AI community: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://en.wikipedia.org/wiki/Bfloat16_floating-point_format. In context single-precision isn't all that puny! And there have already been successful experiments with stronger quantization, like 8-bit neural nets, or even 1-bit (!) neural nets. There is a lot of evidence that neural networks can be very resilient to quantization noise.
- ISL 5y ago
  I'd be real surprised if the neurons in our brains have ADC-equivalents better than ~4 bits.
  - tgv 5y ago
    True, but ANNs are nowhere near as good as our brains, nor do they operate in the same way.
  - xwolfi 5y ago
    I think you missed the fact we were talking about neural network, not an animal brain self replicating and branching and competing with itself for billions of years until it becomes aware of itself. Give us the the same time.
    - Retric 5y ago
      The earliest and simplest brains where still useful. Even insects can fly around in 3D space, I doubt you need something as complicated as a mouse brain to run a self driving car let alone a drone.
      - ben_w 5y ago
        I’m not sure how much it matters given this thread looks like it’s going off on several successive tangents, but the important (and hard) thing with a self-driving car is making sure it doesn’t hit stuff, not the actual driving part. And drones, trivially agree: Megaphragma mymaripenne has 7400 neurones, compared to the 71/14 million in a house mouse nervous system/brain.
        robwwilliams 5y ago
        And to stay on this odd tangent: Best estimate I have for total cell numbers in mouse brain—about 75 million neurons and 35 million other cell types. This estimate is from a 476 mg brain of a C57BL/6J case—the standard mouse used by many researchers. Based on much other work with discrete neuron populations the range among different genometypes of mice probable +/- 40%. for details see: www.nervenet.org/papers/brainrev99.html Expect many more (and I hope better) estimates soon from Clarity/SHIELD whole brain lighsheet counting with Al Johnson and colleagues at Duke and team at Life Canvas Tech.
kortex 5y ago
It's not a transmission line though, SNR does not apply in the same way. It's more like CMOS where the signal is refreshed at each gate. Each stage of an ANN applies some weight and activation. You can think of each input vector as a vector with a true value plus some noise. As long as that feature stays within some bounds, it is going to represent the same "thought vector". It may require some architecture changes to make training feasible, but it's far from a nonstarter. And that is only considering backprop learning. The brain does not use backprop, and has way higher noise levels.
- dahart 5y ago
  I think the parent was referring to the same noise that you are, compute precision, not transmission, and was suggesting that perhaps it won’t easily stay within bounds due to the fact that some kinds of repeated calculations lose more precision at every step. Maybe it’s application dependent, maybe NNs or other matrix-heavy domains can tolerate low precision much more easily than scientific simulations. It certainly wouldn’t surprise me if these “LightOPS” processors work well in a narrow range of applications, and won’t improve or speed up just anything that needs a matrix multiply.
tdrdt 5y ago
But arent there other applications where this is ok? For example path tracing (ray tracing) doesn't need to be very accurate because multiple samples per pixel are used. A gaming card that uses less power is very welcome in laptops for example.
- moonchrome 5y ago
  What do you mean? What kind of worlds can you represent with 8 bits units ? Some small blocky voxel box ?
  - tdrdt 5y ago
    I mean inacurate vector math
    - moonchrome 5y ago
      But that is what I'm saying if your vectors are reduced to 8 bit scalar components you can represent a 256x256x256 worth of detail in the world (doesn't need to be linear but still really limited details) ?
      - montjoy 5y ago
        I’m totally out of my expertise here but I have a question - from my understanding ray tracing is primarily used for lighting/shadows/reflections- wouldn’t it be OK for something like shadows to be inaccurate- maybe some sort of amalgamation over frame refreshes? Real world light is messy anyway. I’m talking about a game type scenario not something scientific. Maybe another way to ask is- we’re trying to simulate a real world “analog” scene - maybe using an analog processing technique could actually be quite faithful for generating it? Or not. Like I said I don’t understand too much of this.
        moonchrome 5y ago
        Think about it like this : you have a spaceship model fits in 256x256x256m - to get the maximum resolution while still fitting in 8 bits you would make each axis in 1m increments and you have 256 values. So you can't have sub 1m details in geometry. Floating point is different because you have an exponent so it's not linear, you can technically have larger scale, but you sacrifice even more precision in mantissa. Now I'm not sure what this 6/8 bit precision or analog precision means so I can't say with confidence, but if your scalars are that low precision you can't really do much. You could technically encode it with some fancy tricks like instead of storing coordinates for each vertex you store the delta from previous one etc. but I think this wouldn't work if the device was just some dumb analog matrix multiplier with baked logic. Also having low detail shadows creates visual artifacts, see this for example [1] [1] https://digitalrune.github.io/DigitalRune-Documentation/media/Shadow-Resolution.jpg https://digitalrune.github.io/DigitalRune-Documentation/medi...
bjornsing 5y ago
If there’s too much noise just lower the dropout probability from 50 to 30%. ;) Joking aside, it is interesting how much noise and quantization these neural networks can work with. I think there’s a lot of room for low precision noisy computation here.

robert_tweed 5y ago

For a few seconds I thought it was Lite-On, best known for their cheapo CD/DVD drives. Seems to be completely unconnected though.

Datagenerator 5y ago
Brings back memories of the Plextor automatic duplicator robot we had in company back in the nineties. Great times

zapdrive 5y ago

Nice. Can't wait for the new light based GPUs, all being grabbed by greedy crypto miners and me still using my 6 year old graphics card!

JohnJamesRambo 5y ago
Relief is on the horizon. Ethereum should switch to proof of stake in June 2022 and you are about to see an unholy torrent of used GPUs hit the market. I would expect you can pick up any you like for peanuts then.
- demux 5y ago
  Ethereum PoW won't immediately disappear, and I'm sure bitcoin folk will be all too happy to grab those extra GPUs
  - vidarh 5y ago
    GPU mining Bitcoin hasn't been reasonable for many years unless you have very cheap energy and no access to ASIC miners.
- zapdrive 5y ago
  I have been hearing "Ethereum is switching to proof of stake in a few months" for at least 6 years now. I don't think it's going to ever happen.
- ohgodplsno 5y ago
  No, miners will just move to another valuable PoW shitcoin and/or fork ETH to stay on PoW.

inasio 5y ago

I've never heard of LightOn, and wish the website had a bit more concrete info on the specifics of the coprocessor, but I am somewhat familiar with a similar photonic coprocessor made by NTT (the Coherent Ising Machine). It's still in the research stage, the logic uses interferometry effects, and requires kilometers of fiber optic cables. Interestingly, there is a simulator based on mean field theory that runs on GPUs and FPGAs(*) that can solve some problems (e.g. SAT) with close to state of the art performance. (*) disclosure: my company helped build the simulator

dasudasu 5y ago
There are other startups in the space that do it in semiconductors. Look up Lightelligence and Lightmatter for instance.