3 ms·
Seems like continuous-time gradient descent on something approximating a ReLU network would not be too terribly difficult to implement in analog hardware. Inste
by blt 5y ago
Seems like continuous-time gradient descent on something approximating a ReLU network would not be too terribly difficult to implement in analog hardware. Instead of summing gradients over a minibatch we could do pure SGD with only one sample at a time, using a latch to hold the sample constant while loading another one from storage. Has this been done?
- dekhn 5y agoIsn't that the Mark I perceptron? https://en.wikipedia.org/wiki/Perceptron https://en.wikipedia.org/wiki/Perceptron with some exceptions (it's single-layer, and the entire dataset is a single batch input).