4 ms·
This is essentially how the very first neural network, the Perceptron, was implemented in the late 50's. It even has potentiometer dials to set the weights, and
by cshimmin 3y ago
This is essentially how the very first neural network, the Perceptron, was implemented in the late 50's. It even has potentiometer dials to set the weights, and they were driven by motors during training!
As for feasibility of doing analog inference today, it's an interesting idea and I'm not sure. Billions of parameters would mean billions of wires though, there may be substantial parasitic losses in building such a thing.
- buescher 3y agoCarver Mead tried back in the eighties. There’s a book - and Synaptics.
- cubefox 3y agoThe most advanced form of analog neuromorphic computer seems to be this: https://www.nature.com/articles/s41586-022-04992-8 https://www.nature.com/articles/s41586-022-04992-8 It makes use of a special ReRAM ("memristor") chip. But, as I suspected, it is far too small to run large language models.
- fulafel 3y agoAnalog nature by itself doesn't have to mean individual wires or knobs. I woner what the analog equivalent of a FPGA (mutable connections) would be.
- rapjr9 3y agoSimpler machine learning algorithms are being implemented in analog hardware. It costs much less power: https://sites.dartmouth.edu/odame/research/asthma-symptom-monitoring/ https://sites.dartmouth.edu/odame/research/asthma-symptom-mo...