3 ms·

I think the combination, especially in AI systems where accuracy doesn't need to be perfect and can be modeled in analog, can perfectly comeback. i.e. If some

I think the combination, especially in AI systems where accuracy doesn't need to be perfect and can be modeled in analog, can perfectly comeback. i.e. If some circuitry is interfacing with an analog counterpart where 16-bit precision is enough at the digital side and the analog circuit can produce outputs matching that accuracy much more efficiently than digital while still being able to be adjusted externally by the digital, why not? I think it can open up some new doors in AI, though what excites me is that what else other than AI could it be applied to? Circuit simulations? Signal generation? Perhaps some analog "programming languages"?

dr_dshiv 4y ago
Isn’t Dwave’s quantum annealing an analog process? Much neuromorphic computing or coupled oscillator computing (including von neuman’s alt architecture) is analog.
zozbot234 4y ago
Analog has a limited dynamic range, in addition to its well-known problems with noise, sensitivity to random variation within devices or the in surrounding environment, etc. Most AI systems are using floating point compute because of its much greater dynamic range compared to fixed point, even though the latter is way more efficient.
- adrian_b 4y ago
  For ML inference, the dynamic range and rounding noise of the 16-bit or 8-bit floating-point numbers preferred now for this application is well within the analog capabilities. Analog processing cannot compete with 32-bit floating-point numbers, when their precision and/or dynamic range is required.
  - amelius 4y ago
    How would power compare?
    - thfuran 4y ago
      And order of magnitude or two better, probably.
    - adrian_b 4y ago
      The power requirements for analog processing depend a lot on the required signal-to-noise ratio (SNR) and dynamic range, like also for digital processing. Increasing the SNR for digital processing requires the processing of more bits, which means either the multiplication of digital blocks such as adders, or the increase of the clock frequencies, when the existing digital blocks must process more bits during the same time, in serial manner. Both methods increase the power consumption. Increasing the SNR for analog processing means the use of more complex circuits, with more devices, and the use of larger devices, e.g. larger transistors, larger capacitors, larger resistors, so that the effect of various noise sources will be relatively smaller (due to averaging). This also increases the power consumption. The relative power consumptions for analog and digital processing depend a lot on the technologies used, but in general analog processing needs much less power when the requirements for SNR/dynamic range are low, but the power increases rapidly with increased requirements and at some threshold the power needed becomes larger than for implementing the same task by digital processing. In modern CMOS technologies designed for making digital devices, the analog components have become worse and worse in comparison with old technologies. That has moved the threshold where digital processing becomes preferable lower and lower. For example, because of this, at every smartphone generation where a new CMOS process is used, an important part of the migration of the previous designs from the old process to the new process is to identify some old analog blocks whose functions could be implemented digitally in the new process, so that those analog blocks could be replaced by digital blocks, because otherwise the new chip could be worse than the old due to worse analog parts, even if the digital parts have been improved. The parent article makes the point that this trend which was continuous for the last few decades can be reversed now due to the advances in packaging and interconnection technologies, which can make a modern multi-chip device to be similar in price to a monolithic device. If the restriction of using a single chip is removed, then it is no longer necessary to use the bad analog components provided by the modern CMOS technologies but it is possible to use a separate analog chip, made with an appropriate manufacturing process. In this case the threshold where digital processing becomes preferable to analog processing can be moved again to higher SNR/dynamic range values, where it was many years ago.
- genewitch 4y ago
  Digital is just constraints laid over analog components; before a motherboard gets produced the traces are analyzed with tools for analog. This is why when looking at data sheets for ICs, gates, etc, there's timing constraints to not get garbage on the output pins. I've been meaning to mess around with some slower digital ICs and see what happens to the output pin(s), but I don't have a fast enough oscilloscope.
  - deleted 4y ago
    [deleted]
  - agumonkey 4y ago
    I often feels that when digital components are numerous enough, you fall back into statistical/analog thinking anyway (even if it's just a case of a few thousands of VMs respawning on need). No more discrete perfect information.. but flocks of potentials.
- adamnemecek 4y ago
  You can use photonics.
- deleted 4y ago
  [deleted]
oplaadpunt 4y ago
A 16 bit signal already represents an almost 100dB signal to noise ratio, which is extremely difficult to do operations on. I think 8 bits would be more realistic, but it would still suffer from the problems mentioned by the other reply.

dr_dshiv 4y ago

Isn’t Dwave’s quantum annealing an analog process? Much neuromorphic computing or coupled oscillator computing (including von neuman’s alt architecture) is analog.

zozbot234 4y ago

Analog has a limited dynamic range, in addition to its well-known problems with noise, sensitivity to random variation within devices or the in surrounding environment, etc. Most AI systems are using floating point compute because of its much greater dynamic range compared to fixed point, even though the latter is way more efficient.

adrian_b 4y ago
For ML inference, the dynamic range and rounding noise of the 16-bit or 8-bit floating-point numbers preferred now for this application is well within the analog capabilities. Analog processing cannot compete with 32-bit floating-point numbers, when their precision and/or dynamic range is required.
- amelius 4y ago
  How would power compare?
  - thfuran 4y ago
    And order of magnitude or two better, probably.
  - adrian_b 4y ago
    The power requirements for analog processing depend a lot on the required signal-to-noise ratio (SNR) and dynamic range, like also for digital processing. Increasing the SNR for digital processing requires the processing of more bits, which means either the multiplication of digital blocks such as adders, or the increase of the clock frequencies, when the existing digital blocks must process more bits during the same time, in serial manner. Both methods increase the power consumption. Increasing the SNR for analog processing means the use of more complex circuits, with more devices, and the use of larger devices, e.g. larger transistors, larger capacitors, larger resistors, so that the effect of various noise sources will be relatively smaller (due to averaging). This also increases the power consumption. The relative power consumptions for analog and digital processing depend a lot on the technologies used, but in general analog processing needs much less power when the requirements for SNR/dynamic range are low, but the power increases rapidly with increased requirements and at some threshold the power needed becomes larger than for implementing the same task by digital processing. In modern CMOS technologies designed for making digital devices, the analog components have become worse and worse in comparison with old technologies. That has moved the threshold where digital processing becomes preferable lower and lower. For example, because of this, at every smartphone generation where a new CMOS process is used, an important part of the migration of the previous designs from the old process to the new process is to identify some old analog blocks whose functions could be implemented digitally in the new process, so that those analog blocks could be replaced by digital blocks, because otherwise the new chip could be worse than the old due to worse analog parts, even if the digital parts have been improved. The parent article makes the point that this trend which was continuous for the last few decades can be reversed now due to the advances in packaging and interconnection technologies, which can make a modern multi-chip device to be similar in price to a monolithic device. If the restriction of using a single chip is removed, then it is no longer necessary to use the bad analog components provided by the modern CMOS technologies but it is possible to use a separate analog chip, made with an appropriate manufacturing process. In this case the threshold where digital processing becomes preferable to analog processing can be moved again to higher SNR/dynamic range values, where it was many years ago.
genewitch 4y ago
Digital is just constraints laid over analog components; before a motherboard gets produced the traces are analyzed with tools for analog. This is why when looking at data sheets for ICs, gates, etc, there's timing constraints to not get garbage on the output pins. I've been meaning to mess around with some slower digital ICs and see what happens to the output pin(s), but I don't have a fast enough oscilloscope.
- deleted 4y ago
  [deleted]
- agumonkey 4y ago
  I often feels that when digital components are numerous enough, you fall back into statistical/analog thinking anyway (even if it's just a case of a few thousands of VMs respawning on need). No more discrete perfect information.. but flocks of potentials.
adamnemecek 4y ago
You can use photonics.
deleted 4y ago
[deleted]

oplaadpunt 4y ago

A 16 bit signal already represents an almost 100dB signal to noise ratio, which is extremely difficult to do operations on. I think 8 bits would be more realistic, but it would still suffer from the problems mentioned by the other reply.