5 ms·

This is really cool, but I was confused by the framing as a resistor network since I think that should be linear (to first order? I’m not an EE) What they have

by rsfern 2y ago

This is really cool, but I was confused by the framing as a resistor network since I think that should be linear (to first order? I’m not an EE) What they have is a transistor network, and they constrain all the transistors to the ohmic regime, so now the resistivity of an individual transistor can be some nonlinear function of its inputs, which is really cool, like detuning transistors to do analog computation instead of digital. Here’s the preprint: https://arxiv.org/abs/2311.00537 https://arxiv.org/abs/2311.00537

fsckboy 2y ago
>the framing as a resistor network since I think that should be linear transistors are essentially non-linear, but they amplify and can be made to amplify linearly through the use of feedback resistors: if you divide the output voltage across a resistor pair which fixes the output as ratio to the input voltage, that geometric relationship will hold across broad range of input/outputs. for most applications you want linear amplification. (transistors work as a function of current, but passing the current through resistors yields a voltage measurement) a transistor can be thought of as resistive if you treat its voltage:current relationship as a measurement of resistance.
fastglass 2y ago
essentially when there's an analog, or non-digital change in resistance over time then you can have non-linear function over that circuit
fastglass 2y ago
essentially when there's an analog, or non-computational change in resistance, like a drift in value, then you can have non-linear function over that circuit
inhumantsar 2y ago
so, a new type of memristor?
pfdietz 2y ago
> linear (to first order? I love it when things are linear to first order. >.>
- rsfern 2y ago
  Doesn’t everyone? But seriously, my actual question is whether an ideal resistor network can compute nonlinear functions since the individual resistors are linear in their input, ignoring possible nonlinear effects like temperature dependence of the conductivity of the resistors
  - JKCalhoun 2y ago
    Just spitballing: but I'm thinking about how slide rules are linear but can do calculations that, I believe?, are not limited to being linear. Due I imagine to logarithmic rulings and that logarithms can be added/subtracted in a linear fashion.
    - utensil4778 2y ago
      That's more or less what we have now. Slide rules and computers are linear systems which can compute nonlinear functions. The problem is that computing anything takes time and must be a linear set of instructions executed in series (multiplied by many parallel cores). Using an analog approach could be vastly more efficient as operations are inherently parallel. You can fire off every neuron in a layer simultaneously and produce a result within nanoseconds, for basically any number of neurons. You could probably do the entire network as a single atomic operation, but that's a bit beyond my knowledge of neural networks
  - duped 2y ago
    By definition it can't. An ideal resistor is a linear relation between the voltage across it and the current through it. If you are allowed switches in the network and the input is a fixed voltage then things get interesting.
    - 2y ago
utensil4778 2y ago
Yeah, a purely resistive network will be linear in all respects (discounting thermal and related drift). A network of transistors operating below saturation makes much, much more sense. It's really directly analogous to how we compute neuron activation in software, but inherently massively parallel.
RobotToaster 2y ago
Interesting, I wonder if this can be used to create a programmable BEAM nervous network https://en.wikipedia.org/wiki/BEAM_robotics https://en.wikipedia.org/wiki/BEAM_robotics
arbuge 2y ago
Once the training is complete, one thing I didn't see mentioned in the paper was how they maintain the charge on the gate capacitors, which is analogous to the weights in a traditional neural network if I'm understanding this correctly. Any practical implementation will need to have some practical way to refresh that on a continuous basis so that the weights don't drift. Was this perhaps mentioned somewhere and I missed it?
- trueismywork 2y ago
  One can use MOS capacitors for this which essentially double up as flash drives so have stprage and refresh built in. Just like flash drives, you can only do limited amount of training on them.

by rsfern 2y ago

fsckboy 2y ago

>the framing as a resistor network since I think that should be linear transistors are essentially non-linear, but they amplify and can be made to amplify linearly through the use of feedback resistors: if you divide the output voltage across a resistor pair which fixes the output as ratio to the input voltage, that geometric relationship will hold across broad range of input/outputs. for most applications you want linear amplification. (transistors work as a function of current, but passing the current through resistors yields a voltage measurement) a transistor can be thought of as resistive if you treat its voltage:current relationship as a measurement of resistance.

fastglass 2y ago

essentially when there's an analog, or non-digital change in resistance over time then you can have non-linear function over that circuit

fastglass 2y ago

essentially when there's an analog, or non-computational change in resistance, like a drift in value, then you can have non-linear function over that circuit

inhumantsar 2y ago

so, a new type of memristor?

pfdietz 2y ago

> linear (to first order? I love it when things are linear to first order. >.>

rsfern 2y ago
Doesn’t everyone? But seriously, my actual question is whether an ideal resistor network can compute nonlinear functions since the individual resistors are linear in their input, ignoring possible nonlinear effects like temperature dependence of the conductivity of the resistors
- JKCalhoun 2y ago
  Just spitballing: but I'm thinking about how slide rules are linear but can do calculations that, I believe?, are not limited to being linear. Due I imagine to logarithmic rulings and that logarithms can be added/subtracted in a linear fashion.
  - utensil4778 2y ago
    That's more or less what we have now. Slide rules and computers are linear systems which can compute nonlinear functions. The problem is that computing anything takes time and must be a linear set of instructions executed in series (multiplied by many parallel cores). Using an analog approach could be vastly more efficient as operations are inherently parallel. You can fire off every neuron in a layer simultaneously and produce a result within nanoseconds, for basically any number of neurons. You could probably do the entire network as a single atomic operation, but that's a bit beyond my knowledge of neural networks
- duped 2y ago
  By definition it can't. An ideal resistor is a linear relation between the voltage across it and the current through it. If you are allowed switches in the network and the input is a fixed voltage then things get interesting.
  - 2y ago

utensil4778 2y ago

Yeah, a purely resistive network will be linear in all respects (discounting thermal and related drift). A network of transistors operating below saturation makes much, much more sense. It's really directly analogous to how we compute neuron activation in software, but inherently massively parallel.

RobotToaster 2y ago

Interesting, I wonder if this can be used to create a programmable BEAM nervous network https://en.wikipedia.org/wiki/BEAM_robotics https://en.wikipedia.org/wiki/BEAM_robotics

arbuge 2y ago

Once the training is complete, one thing I didn't see mentioned in the paper was how they maintain the charge on the gate capacitors, which is analogous to the weights in a traditional neural network if I'm understanding this correctly. Any practical implementation will need to have some practical way to refresh that on a continuous basis so that the weights don't drift. Was this perhaps mentioned somewhere and I missed it?

trueismywork 2y ago
One can use MOS capacitors for this which essentially double up as flash drives so have stprage and refresh built in. Just like flash drives, you can only do limited amount of training on them.