3 ms·
Deep learning is actually very tolerant to imprecision, which is why it is typically given as an application for analog computing. It is already common practic
by Legend2440 1y ago
Deep learning is actually very tolerant to imprecision, which is why it is typically given as an application for analog computing.
It is already common practice to deliberately inject noise into the network (dropout) at rates up to 50% in order to prevent overfitting.
- red75prime 1y agoIsn't it just for inference? Also, differentiating thru an analog circuit looks... interesting. Keep inputs constant, wiggle one weight a bit, store how the output changed, go to the next weight, repeat. Is there something more efficient, I wonder.
- dgfl 1y agoDefinitely, if your analog substrate is implementing matrix vector multiplications (one of the most common approaches in this area). Then your differentiation algorithm is the usual backpropagation, which has rank 1 weight updates. With some architectures this can be implemented in O(1) time simply by running the circuit in "reverse" configuration (inputs become outputs and vice-versa). With ideal analog devices, this would be many orders of magnitude more efficient than a GPU.