4 ms·
The paper describes it pretty well in appendix C. A matrix of integrators is constructed with a bunch of opamps, RC time constants (using digital potentiometer
by CamperBob2 2y ago
The paper describes it pretty well in appendix C. A matrix of integrators is constructed with a bunch of opamps, RC time constants (using digital potentiometers, presumably) and a multichannel ADC/DAC interface to the PC. Essentially a dedicated differential-equation solver.
So it's a combination of old-school analog computation and modern GPU-based code. Takes longer in practice due to the overhead of interfacing with the hardware and waiting for the integrators to settle, but the authors are claiming that an optimized implementation could outperform a purely-digital solution, as I understand it, by accelerating convergence.
The core idea being that conventional gradient descent is a linear operation at heart, while the gradients actually being traversed are curved surfaces that have to be approximated with multiple unnecessary steps if everything is done in the digital domain.
The trouble, as everybody from Seymour Cray onward has learned the hard way, is that CMOS always wins in the end, simply because the financial power of an entire industry goes into optimizing it.
- stefanpie 2y agoI didn't realize they included details about the hardware. Lie you said these just look like analog computers, compute in memory, analog arrays, which have also made a resurgence with deep leaning.
- kaelan123 2y agoFirst author of the paper here. That's it indeed! One thing is that this is entirely CMOS-compatible. You could also do something similar with optics or other platforms, but we chose electronic circuits for this reason specifically.
- CamperBob2 2y agoBy that remark I meant "digital CMOS," in the sense of elements that store state information discretely in flip-flops or gate insulators rather than continuously with analog integrators. Very cool work in any event, though! Best of luck with the ongoing R&D.
- anigbrowl 2y agoCan you implement this on an FPGA?
- simne 2y agoAny good electronics engineer can. Question is just who will pay for it? Or closer to YC environment, question is monetization.
- nolist_policy 2y agoYes, digital wins over analog because of all the money that went into digital. I am wondering if one could create a digital analog computer by using pwm instead of analog signals.