4 ms·
I love tangents! I think all of your points are valid. I also think that we have optimized heavily for this state of technology. If we figured out that analog
by imoverclocked 2y ago
I love tangents!
I think all of your points are valid. I also think that we have optimized heavily for this state of technology. If we figured out that analog computing was somehow superior in a big way, I bet we would find ways of reducing power etc in analog designs.
One way that analog computing would be really neat for neural networks is in speed. The way it might not be so great is in reliability (or repeatability, specifically.) Analog systems are more susceptible to noise as well as variation from fabrication processes. Running things at saturation makes them easier to design, test and mass produce.
- akdom 2y agoSpecifically to the point of the comparatively low reliability / high variation of analog systems: an interesting property of neural nets is that they can be robust relative to noise when trained with the same type of noise witnessed under inference. Whether or not speed/etc. would be better in the digital vs. analog design-space, it's an interesting thing to consider that neural nets can automatically account for the encoding-medium's variability. This perhaps makes neural solutions a good fit for low power analog media which otherwise aren't useful for classical computing. See https://arxiv.org/abs/2104.13386 https://arxiv.org/abs/2104.13386 for an exploration of physically encoding neural architectures.