4 ms·

Memristor and analogue processors to accelerate ML are doing this. Neural networks are ok with operations on noisy, analogue states. Analogue operations such

Memristor and analogue processors to accelerate ML are doing this. Neural networks are ok with operations on noisy, analogue states. Analogue operations such as op-amp multiplication are more power hungry than individual digital gates, but some are less power hungry than digital multipliers which use thousands of gates. For digital, noise-free logic, it's generally more power efficient and physically simpler to have logic gates operate on two cleanly separated states with more gates, than to have bulkier, more complicated gates that do the same thing with combined states. A transistor which is fully on or fully off in a logic circuit uses low power either way (like a wire or a gap in the circuit), but the in-between state uses more power (like a resistor, it produces heat). This is one reason why power consumption goes up with the amount of logic state switching (the in-between resistor-like state occurs briefly during each state change), and also why many-level stable logic states aren't so efficient, except with complicated gates that use many transistors to implement many thresholds. For non-volatile storage, that doesn't apply. Information density is more important, individual memory cells do not change state often so switching power is less of a thing per memory cell, and the logic sits at the edge of the memory array, shared among many cells. The edge circuitry can afford to be more complicated, to optimise the bulk of the memory array. In magnetic storage and communication, the signal processing to encode many states in a small signal takes considerable power, but the trade off is worth it.

202210212010 4y ago
> Memristor and analogue processors to accelerate ML are doing this. Neural networks are ok with operations on noisy, analogue states. Analogue operations such as op-amp multiplication are more power hungry than individual digital gates, but some are less power hungry than digital multipliers which use thousands of gates. I checked Mouser just now and memristors are not in stock: https://www.mouser.com/c/?q=memristor https://www.mouser.com/c/?q=memristor If we are talking about imaginary components, then I think quantistors will change everything.
- Lt_Riza_Hawkeye 4y ago
  In-memory computing is just barely in active research. Unlike quantistors, memristors are being built (and have already been built) in many research laboratories. If you are interested in learning, the idea of memristors (and other computational memory technology) is not to replace traditional memory, but rather to augment small portions of it with increased/additional computational functionality. For example, you could add a (relatively speaking) extremely small number of memristors to an existing memory module, load two matrices into that area of memory, and reading from the adjacent region of memory would immediately yield the result of multiplying those two matrices. If you could simply instruct the RAM module where the existing data lies, this would be an immense boost in efficiency to AI/deep learning prediction algorithms. Here is a video explanation of how this could be use to perform matrix vector multiplication in O(1): https://youtu.be/30K5i8bdiyg?t=1492 https://youtu.be/30K5i8bdiyg?t=1492 . That video mentions later that something similar can be used in solving linear and partial differential equations. Applications for deep learning training is discussed at 40:20 . You'll notice everything discussed in the keynote has experimental results. At 41:27, he mentions that you can actually send images to their memristor chip over their website . Though the link is unfortunately dead, it definitely worked, and was in fact performing neural network computations using memristors over the network.

202210212010 4y ago

> Memristor and analogue processors to accelerate ML are doing this. Neural networks are ok with operations on noisy, analogue states. Analogue operations such as op-amp multiplication are more power hungry than individual digital gates, but some are less power hungry than digital multipliers which use thousands of gates. I checked Mouser just now and memristors are not in stock: https://www.mouser.com/c/?q=memristor https://www.mouser.com/c/?q=memristor If we are talking about imaginary components, then I think quantistors will change everything.

Lt_Riza_Hawkeye 4y ago
In-memory computing is just barely in active research. Unlike quantistors, memristors are being built (and have already been built) in many research laboratories. If you are interested in learning, the idea of memristors (and other computational memory technology) is not to replace traditional memory, but rather to augment small portions of it with increased/additional computational functionality. For example, you could add a (relatively speaking) extremely small number of memristors to an existing memory module, load two matrices into that area of memory, and reading from the adjacent region of memory would immediately yield the result of multiplying those two matrices. If you could simply instruct the RAM module where the existing data lies, this would be an immense boost in efficiency to AI/deep learning prediction algorithms. Here is a video explanation of how this could be use to perform matrix vector multiplication in O(1): https://youtu.be/30K5i8bdiyg?t=1492 https://youtu.be/30K5i8bdiyg?t=1492 . That video mentions later that something similar can be used in solving linear and partial differential equations. Applications for deep learning training is discussed at 40:20 . You'll notice everything discussed in the keynote has experimental results. At 41:27, he mentions that you can actually send images to their memristor chip over their website . Though the link is unfortunately dead, it definitely worked, and was in fact performing neural network computations using memristors over the network.