4 ms·
When I told Hinton about SPWs and that they are related to RBMs, he told me that he had a theory which required the existence of SPWs. That is not a coincidence
by jayajay 10y ago
When I told Hinton about SPWs and that they are related to RBMs, he told me that he had a theory which required the existence of SPWs. That is not a coincidence, they seem to have been inspired by neuroscience and real neural network phenomena.
- giardini 10y agoWhat's an SPW?
- jayajay 10y agoSPW: High frequency oscillation in the hippocampus which occurs during rest and sleep. The characteristics would suggest they are critical in memory management. If you google "SPW neuroscience", you will find many resources. https://en.wikipedia.org/wiki/Sharp_waves_and_ripples https://en.wikipedia.org/wiki/Sharp_waves_and_ripples Brain science is highly interdisciplinary. Each expert should spend time to familiarize themselves with the various approaches used by other experts. In particular, machine learning theorists should spend time to understand the latest results (their implications, at least) in neuroscience and neuron biophysics. At some point, an ML theorist is not too different than a biophysics theorist. They just use different abstractions. This is why I am particularly interested in Google's and Hinton's work. That work seems to be subtly motivated by neuroscience and natural science...