4 ms·
ASIC/FPGA or even GPU's aren't the future of neural computing. The future is analog(orders of magnitude perf/watt and perf/$). That require older fabs, optimize
by petra 10y ago
ASIC/FPGA or even GPU's aren't the future of neural computing. The future is analog(orders of magnitude perf/watt and perf/$). That require older fabs, optimized for analog, and having good embedded flash, which TSMC has and Intel mostly hasn't got.
And that same future applies not only for neural computing, but for a field called approximate-computing, i.e. computing where results aren't accurate.Some/many signal and image processing work well with that. I've also seen some research about doing scientific computing on approximate hardware and correcting errors.
- krastanov 10y agoBut analog computers do not permit error correction! Crosstalk of 10% does not affect digital data busses, but ruins analog. ECC memory is trivial with digital but not possible even in theory with analog. This (and programmability) is why we moved from super fast analog to slow digital half a century ago. Given all these constraints I am actually quite excited to hear more of the dissenting view. Can you describe some of the research you mentioned in the last sentence? I have not heard of it and if true it would be at least interesting engineering, even if right now I doubt it would actually work.
- cjhanks 10y agoA significant portion of any real-world AI system is dedicated to extracting relevant features from raw sensor information. Doing that digitally has proven to have a very high cost rated in both watts and operating temperature. Most machine learning algorithms are by design resilient to random noise. They can even be learned to be resilient to systematic noise (due to say.. variable hardware performance in mass production). In those cases; a low power analog device with lower reliability guarantees is 'ok'. I suspect the "decision engines" (the computation enforcing complex logic based on sensor readings) will probably continue to be on CPU's for some time.
- Retric 10y agoThe problem is you now dramatically increased the cost of training. Which might be ok for speech recognition, but not for self driving cars etc.
- cjhanks 10y agoAny feature extractor worth using will be robust against some amount of Gaussian noise. The neural nets will probably never even see the bad readings. It would surprise me if even unfiltered analog noise in the sensors significantly changed recall. I suppose I don't know off-hand know the magnitude of noise in typical analog systems.
- petra 10y agoISOcline.com has one approach, building an analog neuron , and digitizing the inputs and outputs to minimize noise . As a proof of concept they did a GPS chip , with 66 times higher energy efficiency. There are others working on analog neural compute , some released good research results .
- planteen 10y agoWhat analog function exactly are we talking about here? I'm genuinely curious. There's certainly a fading art for lots of analog computing techniques, like taking the arctan or log of an input signal. But how is that useful for neural computing? I worked for a while as the software/FPGA guy on a high-power RF amplifier for MRI machines. I worked with some very talented RF engineers. There were obviously some parts of the chain that were analog (like the preamps and high power amp itself). But lots of it was being replaced with digital (DSP techniques) in the next generation and the RF engineers loved it. Filters no longer needed a 15 minute tuning session by a technician with a network analyzer. A Hilbert transform is far easier and better in digital than analog. Design changes could be experimented with in a day instead of PCB spins that took weeks.
- petra 10y agoSee my reply to krastanov.
- ansgri 10y agoI'm sure AC will be a major revolution in computing in general, though ANN-based systems may come there earlier. Also, there's a huge application domain in image preprocessing: the pixel value is analog and noisy anyway, so building an analog primary visual cortex with a bank of common filters (line segments, stable features etc.) makes a lot of sense. ADD: I'm also thinking about accelerating RANSAC-type algorithms, where you perform lots of preliminary evaluations to filter out the outliers. I bet one can cheaply implement an "100-argmins of 100 x 100 fixed10 distance matrix" computation in a handful of cycles with analog circuits. And high-dimensional RANSAC-type algorithms are what needed for robust and precise 3D visual navigation.