5 ms·
An Analog Network of Resistors Promises Machine Learning Without a Processor
- grantmuller 2y agoI read George Dyson’s “Analogia” with a bit of skepticism a few years ago, now all of the sudden it feels relevant (if you can make it past all of the chapters on kayak-building).
- alexpotato 2y agoThis post reminded me of the below post (from 2007!): https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/ Summary: - Researcher took a FPGA - used genetic algorithms to have the FPGA identify first tones and then more complex audio sequences - there was no clock or timer used - when they found a good solution, they tried to copy over the FPGA "configuration" to another identical FPGA. - that didn't work! - they assumed it was b/c the genetic algorithm + no timer had found a quirk in the specific FPGA unit and used that to help improve the processing quality
- vessenes 2y agoFinding analog architectures for something that is largely continuous but quantized for digital circuitry right now (gradient descent) is pretty appealing. I’d love to see a toy network built out; I wonder how physically large this breadboard setup would have to be to get good results on MNIST, for instance.
- robxorb 2y agoAs I understand it, they were able to train and classify XOR using 32 of their breadboard nodes, which visually looks roughly a 1 metre square. XOR can usually be done with 1 or 2 neurons. And MNIST digit classification with ~25,000. Bringing all those numbers together we get 1 neuron = 32 nodes = 1sqm, would give a size of about a 900m square if breadboarded out for ordinary MNIST digits. (Assuming of course, no power transmission losses...!) I'm hoping, if not outright assuming, that I've made some kind of catastrophic error here.
- rsfern 2y agoThis is really cool, but I was confused by the framing as a resistor network since I think that should be linear (to first order? I’m not an EE) What they have is a transistor network, and they constrain all the transistors to the ohmic regime, so now the resistivity of an individual transistor can be some nonlinear function of its inputs, which is really cool, like detuning transistors to do analog computation instead of digital. Here’s the preprint: https://arxiv.org/abs/2311.00537 https://arxiv.org/abs/2311.00537
- fsckboy 2y ago>the framing as a resistor network since I think that should be linear transistors are essentially non-linear, but they amplify and can be made to amplify linearly through the use of feedback resistors: if you divide the output voltage across a resistor pair which fixes the output as ratio to the input voltage, that geometric relationship will hold across broad range of input/outputs. for most applications you want linear amplification. (transistors work as a function of current, but passing the current through resistors yields a voltage measurement) a transistor can be thought of as resistive if you treat its voltage:current relationship as a measurement of resistance.
- fastglass 2y agoessentially when there's an analog, or non-digital change in resistance over time then you can have non-linear function over that circuit
- fastglass 2y agoessentially when there's an analog, or non-computational change in resistance, like a drift in value, then you can have non-linear function over that circuit
- inhumantsar 2y agoso, a new type of memristor?
- pfdietz 2y ago> linear (to first order? I love it when things are linear to first order. >.>
- superlopuh 2y agoHere's a keynote talk by Andrea Liu, the lead of the project, it's a much better resource about one of the most exciting things going on in ML right now: https://youtu.be/7hz4cs-hGew?si=64O3Q7g-qeRQ0Td4 https://youtu.be/7hz4cs-hGew?si=64O3Q7g-qeRQ0Td4
- deleted 2y ago[deleted]
- pmorici 2y agoIs this similar to the Extropic approach but a different mechanism?
- motohagiography 2y agowas going to ask the same thing, "thermodynamic computing," was what I interpreted as an ASIC for training models, and then once you can train and run models on it, what do you need classical compute for.
- quantum_state 2y agoWhen memristor came into the scene, thought it would be a reasonable substrate for network based learning system … anyone know more about what happened after?
- delfinom 2y ago15 years later and nobody can make it in economical price and volume. Perhaps demand just wasn't there and we may see something take off on the coming years.
- kc0bfv 2y agoAs the newcomer to the, very entrenched, block, I think the memristor has a lot of momentum to overcome. In a EE undergrad (2007, so it has been a bit) we spent plenty of time understanding resistors, capacitors, and inductors. We looked at example circuits and uses, we learned the math and theory... We developed intuition around them. Memristors were the missing fourth, and "imagine what you could do with that!" My imagination did not extend very far. Everything was being built with those other three and the non-linear components. It'll take a while to overcome that momentum. I feel like IPv6 has a similar barrier. I'm mostly an infosec nerd and I've been through a lot of training and education. Never once seen IPv6 treated beyond, "it has more bytes, firewall it off".
- cjk2 2y agoThere were no applications that couldn’t be solved cheaper an easier using some other method.
- riedel 2y agoIMHO this can become really cool if combined with mass customisation e.g. by using printed electronics. My colleague who will shortly defend his PhD is working on this [1] Edit: downloadable link [2] [1] https://ieeexplore.ieee.org/abstract/document/10323917/ https://ieeexplore.ieee.org/abstract/document/10323917/ [2] https://publikationen.bibliothek.kit.edu/1000161182/151107287 https://publikationen.bibliothek.kit.edu/1000161182/15110728...
- osigurdson 2y agoRemoving entropy from transistors is expensive - computers use just two states separated by large voltage differences. In AI, entropy isn't a problem as we don't care about repeatable results. Therefore, why not use more of the linear or even non linear range of the transistor for this purpose?
- linsomniac 2y agoA couple years ago Veritasium did a video on analog computers, which included a segment on Mythic AI that uses NAND flash cells kind of "undervolted" as an analog computer to run neural networks. https://youtu.be/GVsUOuSjvcg?si=GGsEWELZyjb0TQfG&t=898 https://youtu.be/GVsUOuSjvcg?si=GGsEWELZyjb0TQfG&t=898 https://mythic.ai/ https://mythic.ai/
- shrubble 2y agoThe traditional problem that analog computers faced were that voltages could vary from run to run and thus give different results, to the point that analog computer manufacturers made their own power supplies and capacitors to extremely high tolerances. It's not clear in the paper if this problem was addressed or if the rapid training possible meant that in practice they never had this issue.
- ryukoposting 2y agoIt's also a possibility that a bit of innate noise could be desirable, depending on the use case.
- chewbaxxa 2y agoIf it was desirable you would still want to isolate it
- hulitu 2y ago> An Analog Network of Resistors Promises Machine Learning Without a Processor Ecerybody promises "Machine Learning", but the machines never learn. /s
- beedeebeedee 2y agoThere won't be any energy efficiency improvements until they are able to make analog VLSI chips like Carver Mead's. Nice to see this idea is getting more recognition. The potential has been there for a long time, but the business went digital.
- klysm 2y agoConceptually I expect compiling NNs to hardware to eventually be done at a small scale at first. Imagine you have a relatively simple task with a fairly small NN which is being used a low latency, mass production application. If you could compile that sensor into a passive component array, that could be massive.