4 ms·
"I wouldn't be so sure that Boltzmann machines were designed without how the brain operated in mind." Sure, actually it seems highly unlikely to me that they w
by tprice7 10y ago
"I wouldn't be so sure that Boltzmann machines were designed without how the brain operated in mind."
Sure, actually it seems highly unlikely to me that they were designed without considering at all how they brain works. Let me clarify what I meant: given the definition of how a Boltzmann machine infers (i.e. how to determine which neurons are on or off), the training algorithm can be derived from purely mathematical considerations.
- jayajay 10y agoWhen I told Hinton about SPWs and that they are related to RBMs, he told me that he had a theory which required the existence of SPWs. That is not a coincidence, they seem to have been inspired by neuroscience and real neural network phenomena.
- giardini 10y agoWhat's an SPW?
- jayajay 10y agoSPW: High frequency oscillation in the hippocampus which occurs during rest and sleep. The characteristics would suggest they are critical in memory management. If you google "SPW neuroscience", you will find many resources. https://en.wikipedia.org/wiki/Sharp_waves_and_ripples https://en.wikipedia.org/wiki/Sharp_waves_and_ripples Brain science is highly interdisciplinary. Each expert should spend time to familiarize themselves with the various approaches used by other experts. In particular, machine learning theorists should spend time to understand the latest results (their implications, at least) in neuroscience and neuron biophysics. At some point, an ML theorist is not too different than a biophysics theorist. They just use different abstractions. This is why I am particularly interested in Google's and Hinton's work. That work seems to be subtly motivated by neuroscience and natural science...