4 ms·

This runs into the problem that the power efficiency of analogue circuitry is dramatically worse. A FET dissipates the least heat when it's fully on or fully of

by morei 6y ago

This runs into the problem that the power efficiency of analogue circuitry is dramatically worse. A FET dissipates the least heat when it's fully on or fully off. Operating in the resistive regime will result in orders of magnitude more dissipation. It's difficult to over-come that, particularly because it's not a comparison between 'analogue v 64-bit float', but 'analogue versus 8-bit int'. (it's tough for scale analogue circuits to operate even when 8-bit accuracy).

DennisP 6y ago
That just means an FET isn't the right device for analog circuitry. Maybe this memristor device will be more efficient.
- simcop2387 6y ago
  It's a bit more fundamental than that. A FET goes between say, 0.01 Ohms and 1,000,000 Ohms for fully on and fully off. The times in between are where it passes the most power through it and that's because of how Ohms law and Power being current times voltage (ignoring differing phase between current and voltage in AC circuits). This all still remains true even for the memristors. You can do analog stuff with higher resistance and lower currents, but then you end up more susceptible to noise and other randomness induced by the environment: thermal noise, electric noise, magnetic noise, and RF noise (technically also electric noise, but more organized), etc. All that being said, for a NN this might actually be somewhat acceptable, but for general analog circuitry you can't get past this problem at all. That's a large reason why most analog circuitry these days does as much as possible in a digital domain before converting back to analog, even simple amplification. A class D amplifier does the whole job by chopping up a large voltage into just on and off and then filters it afterwards. Even though you have power loss in the filtering, the end result is much easier to have stable, and get the desired amplification. You still find class A and B amplifiers where the high frequency chopping noise isn't possible to filter out or would cause other problems, particularly when you look at RF circuitry since you'd end up transmitting that high frequency noise and causing other problems for yourself.
  - sgtnoodle 6y ago
    It's my understanding that, to some extent, nerves and muscles operate with hysteresis based on charge thresholds, and rely on pulse-density encoding for variability rather than proportional voltage or current modulation. Presumably, that seems to be why your muscles shake when under heavy strain. It sorta makes sense, considering your body is generally electrically conductive everywhere. Signals propagate as waves of charge sustained by cells opening and closing little pores that selectively release ions. "Conductors" have pores, and "insulators" don't. That's also why nerve signals are relatively slow to propagate vs. straight up electricity. That's also why you can measure muscle activity via an ECG; you're measuring the transient voltages being generated by the waves of charge moving around. The voltages themselves don't have much meaning to your body. Of course, if you apply enough voltage to build up a charge inside a muscle, you can cause it to actuate. I recently had to get a pacemaker, and apparently the particular lower threshold to trigger my heart muscle to contract is around 0.7V for 350uS, at whatever impedance the lead happens to be. Below that, and my heart muscle does nothing. Anything above that, any my heart muscle does a full beat. The device applies a 2V pulse so that there's plenty of margin, and can go up to 5V if needed, in case the lead's impedance increases. The lower the voltage the better, in terms of battery life. The cool thing is that the device can safely coexist with any natural electrical activity my faulty nerves may have, since the muscle simply responds to whichever pulse happens to arrive first.
typon 6y ago
Ideally each analog device that simulates a neuron is equivalent to tens of thousands of digital transistors that simulate a logical neuron (in space-time for GPUs or CPUs and just space for neuromorphic asics) . This makes the trade-off much more feasible. .
317070 6y ago
We don't need to go fully analogue though, going asynchronous might already be a step up. In neural networks some groups do research into so-called spiking neural networks, which you could think of as electronic circuits operating asynchronously on binary signals. They are very energy efficient, but nobody really knows how to use them yet.
- kevin_b_er 6y ago
  Clockless CPUs have been a theoretical idea for over a decade before the "neutral network" became a common phrase. Felt like a decade ago it was still a pipe dream. Resolving the asynchronous signals becomes a chore. The world feels stateful and ordered to humans. In the same way functional programming can be difficult, getting this right is very hard.
  - sasaf5 6y ago
    There had been purely asynchronous CPUs before [0]. Then automated clock gate insertion made it possible to approach the gains of asynchronous, but still designing in a synchronous paradigm. [0] https://en.m.wikipedia.org/wiki/AMULET_microprocessor https://en.m.wikipedia.org/wiki/AMULET_microprocessor
  - dfischer 6y ago
    Greenarraychips are clockless: http://www.greenarraychips.com/home/products/index.html http://www.greenarraychips.com/home/products/index.html Very interesting chips.
glial 6y ago
> the power efficiency of analogue circuitry is dramatically worse The human brain would like to have a word with you.
- eloff 6y ago
  At the slow "clock speed" the brain runs at, it produces a lot of heat. A quick search turns up 10-20 watts as a rough estimate. It's still petty competitive given what it's capable of.
  - tehjoker 6y ago
    Good thing we have liquid cooling
  - anthropodie 6y ago
    Why do you say slow "clock speed"? Can we even use clock speed w.r.t brain? Genuinely curious.
    - falcor84 6y ago
      I assume the comment was referring to the duration of the action potential and subsequent refectory period[0], which are on the order of milliseconds. [0] https://en.wikipedia.org/wiki/Refractory_period_%28physiology%29 https://en.wikipedia.org/wiki/Refractory_period_%28physiolog...
      - candiodari 6y ago
        That makes no sense because the brain is not digital (it's "pulse-based"). There is no signal synchronizing all neurons in the brain, which is probably where a lot of the power savings come from too.
    - eloff 6y ago
      It's a very rough analogy. The brain doesn't work like a computer chip, but it does work much slower than modern chips in neuron firings per second.

by morei 6y ago

DennisP 6y ago

That just means an FET isn't the right device for analog circuitry. Maybe this memristor device will be more efficient.

simcop2387 6y ago
It's a bit more fundamental than that. A FET goes between say, 0.01 Ohms and 1,000,000 Ohms for fully on and fully off. The times in between are where it passes the most power through it and that's because of how Ohms law and Power being current times voltage (ignoring differing phase between current and voltage in AC circuits). This all still remains true even for the memristors. You can do analog stuff with higher resistance and lower currents, but then you end up more susceptible to noise and other randomness induced by the environment: thermal noise, electric noise, magnetic noise, and RF noise (technically also electric noise, but more organized), etc. All that being said, for a NN this might actually be somewhat acceptable, but for general analog circuitry you can't get past this problem at all. That's a large reason why most analog circuitry these days does as much as possible in a digital domain before converting back to analog, even simple amplification. A class D amplifier does the whole job by chopping up a large voltage into just on and off and then filters it afterwards. Even though you have power loss in the filtering, the end result is much easier to have stable, and get the desired amplification. You still find class A and B amplifiers where the high frequency chopping noise isn't possible to filter out or would cause other problems, particularly when you look at RF circuitry since you'd end up transmitting that high frequency noise and causing other problems for yourself.
- sgtnoodle 6y ago
  It's my understanding that, to some extent, nerves and muscles operate with hysteresis based on charge thresholds, and rely on pulse-density encoding for variability rather than proportional voltage or current modulation. Presumably, that seems to be why your muscles shake when under heavy strain. It sorta makes sense, considering your body is generally electrically conductive everywhere. Signals propagate as waves of charge sustained by cells opening and closing little pores that selectively release ions. "Conductors" have pores, and "insulators" don't. That's also why nerve signals are relatively slow to propagate vs. straight up electricity. That's also why you can measure muscle activity via an ECG; you're measuring the transient voltages being generated by the waves of charge moving around. The voltages themselves don't have much meaning to your body. Of course, if you apply enough voltage to build up a charge inside a muscle, you can cause it to actuate. I recently had to get a pacemaker, and apparently the particular lower threshold to trigger my heart muscle to contract is around 0.7V for 350uS, at whatever impedance the lead happens to be. Below that, and my heart muscle does nothing. Anything above that, any my heart muscle does a full beat. The device applies a 2V pulse so that there's plenty of margin, and can go up to 5V if needed, in case the lead's impedance increases. The lower the voltage the better, in terms of battery life. The cool thing is that the device can safely coexist with any natural electrical activity my faulty nerves may have, since the muscle simply responds to whichever pulse happens to arrive first.

typon 6y ago

Ideally each analog device that simulates a neuron is equivalent to tens of thousands of digital transistors that simulate a logical neuron (in space-time for GPUs or CPUs and just space for neuromorphic asics) . This makes the trade-off much more feasible. .

317070 6y ago

We don't need to go fully analogue though, going asynchronous might already be a step up. In neural networks some groups do research into so-called spiking neural networks, which you could think of as electronic circuits operating asynchronously on binary signals. They are very energy efficient, but nobody really knows how to use them yet.

kevin_b_er 6y ago
Clockless CPUs have been a theoretical idea for over a decade before the "neutral network" became a common phrase. Felt like a decade ago it was still a pipe dream. Resolving the asynchronous signals becomes a chore. The world feels stateful and ordered to humans. In the same way functional programming can be difficult, getting this right is very hard.
- sasaf5 6y ago
  There had been purely asynchronous CPUs before [0]. Then automated clock gate insertion made it possible to approach the gains of asynchronous, but still designing in a synchronous paradigm. [0] https://en.m.wikipedia.org/wiki/AMULET_microprocessor https://en.m.wikipedia.org/wiki/AMULET_microprocessor
- dfischer 6y ago
  Greenarraychips are clockless: http://www.greenarraychips.com/home/products/index.html http://www.greenarraychips.com/home/products/index.html Very interesting chips.

glial 6y ago

> the power efficiency of analogue circuitry is dramatically worse The human brain would like to have a word with you.

eloff 6y ago
At the slow "clock speed" the brain runs at, it produces a lot of heat. A quick search turns up 10-20 watts as a rough estimate. It's still petty competitive given what it's capable of.
- tehjoker 6y ago
  Good thing we have liquid cooling
- anthropodie 6y ago
  Why do you say slow "clock speed"? Can we even use clock speed w.r.t brain? Genuinely curious.
  - falcor84 6y ago
    I assume the comment was referring to the duration of the action potential and subsequent refectory period[0], which are on the order of milliseconds. [0] https://en.wikipedia.org/wiki/Refractory_period_%28physiology%29 https://en.wikipedia.org/wiki/Refractory_period_%28physiolog...
    - candiodari 6y ago
      That makes no sense because the brain is not digital (it's "pulse-based"). There is no signal synchronizing all neurons in the brain, which is probably where a lot of the power savings come from too.
  - eloff 6y ago
    It's a very rough analogy. The brain doesn't work like a computer chip, but it does work much slower than modern chips in neuron firings per second.