3 ms·
This is a reply to multiple sibling comments. There is actually recent work which shows that deep learning methods can also work WITHOUT any reverse signals: ht
by sherjilozair 12y ago
This is a reply to multiple sibling comments. There is actually recent work which shows that deep learning methods can also work WITHOUT any reverse signals: http://s.yosinski.com/dan_cownden_presentation.pdf http://s.yosinski.com/dan_cownden_presentation.pdf
- jimduk 12y agoInteresting paper. Any more details on the architecture of the feedback connections. Also I can't tell from the paper where and how weights are being updated e..g what does "train" mean in this context
- asdavis 12y agoI believe instead of multiplying the delta by W^t to backpropagate the error from layer l to l-1, you multiply it by a random projection B. It's hard to dig deeper because there doesn't appear to be any other information on it except here: http://isis-innovation.com/licence-details/accelerating-machine-learning/ http://isis-innovation.com/licence-details/accelerating-mach... As an aside, are they really trying to patent a slight twist on backpropagation? That seems pretty counter-productive to me.
- asdavis 12y agoIt looks like there are still reverse signals (eg., deltas), but they are multiplied by a random matrix B instead of the usual transpose of the weight matrix for that layer. Am I misunderstanding?