11 ms·

As someone with no neuroscience background I’ve suspected this for a while just based on how Boltzmann machines work. These are a type of unsupervised artificia

• dontreact 10y ago
  I wouldn't be so sure that Boltzmann machines were designed without how the brain operated in mind. Terry Seknowski, one of the inventors, is a computational neuroscientist. In general, I think the influence of neuroscience on neural network research has been subtle and perhaps underrated. For example, modern day convolutional neural networks Havea lineage going back to Fukushima's neocognitron, which was heavily inspired by Hubel an Wiesel's simple/complex cell model of visual cortex based on single cell electrophysiology.
  • tprice7 10y ago
    "I wouldn't be so sure that Boltzmann machines were designed without how the brain operated in mind." Sure, actually it seems highly unlikely to me that they were designed without considering at all how they brain works. Let me clarify what I meant: given the definition of how a Boltzmann machine infers (i.e. how to determine which neurons are on or off), the training algorithm can be derived from purely mathematical considerations.
    • jayajay 10y ago
      When I told Hinton about SPWs and that they are related to RBMs, he told me that he had a theory which required the existence of SPWs. That is not a coincidence, they seem to have been inspired by neuroscience and real neural network phenomena.
      • giardini 10y ago
        What's an SPW?
        • jayajay 10y ago
          SPW: High frequency oscillation in the hippocampus which occurs during rest and sleep. The characteristics would suggest they are critical in memory management. If you google "SPW neuroscience", you will find many resources. https://en.wikipedia.org/wiki/Sharp_waves_and_ripples https://en.wikipedia.org/wiki/Sharp_waves_and_ripples Brain science is highly interdisciplinary. Each expert should spend time to familiarize themselves with the various approaches used by other experts. In particular, machine learning theorists should spend time to understand the latest results (their implications, at least) in neuroscience and neuron biophysics. At some point, an ML theorist is not too different than a biophysics theorist. They just use different abstractions. This is why I am particularly interested in Google's and Hinton's work. That work seems to be subtly motivated by neuroscience and natural science...
  • mlechha 10y ago
    They weren't. They were a generalization of the Hopfield networks. Boltzmann machines are a stochastic version of the Hopfield network. The training algorithm simply tries to minimize the KL divergence between the network activity and real data. So it was quite surprising when it turned out that the algorithm needed a "dream phase" as they call it. Francis Crick was inspired by this and proposed a theory of sleep.
    • redler 10y ago
      Unpacking a comment like this is one of the quieter pleasures of reading HN.
      • vkreso 10y ago
        Couldn't agree more
      • mlechha 10y ago
        Haha I'm not sure if you're being sarcastic so I'll try to unpack the comment. Hopfield networks were one of the first models of associative memory. They themselves were based on a model of simple magnets called ising model (generalized). Basically a group of binary units, each connected its nearest neighbors with a coupling strength. Each unit prefers to be like their neighbors. Hopfield developed a clever method to change the coupling so that the networks can store and retrieve patterns of activity. In the Hopfield network everything is deterministic, Hopfield himself realized that if this constraint was relaxed this model could become a very powerful computational machine. Which means that if instead of being always on or off, the units had a probability of being on or off the networks could perform very general computational tools [1]. Unfortunately, training these general stochastic systems was not easy. With their Boltzmann machines Sejnowski and Hinton proposed a possible solution. The activity of stochastic binary units effectively encodes a probability distribution, so all they had to do was make sure that the probability distribution being encoded by the activity of the units was the same as that of the input. They did this by changing the connection strengths between the units such that the activity pattern minimized something called the Kullback-Leibler or KL divergence, which is a measure of how close two probability distributions are (the one encoded by the network activity, or the dream activity of the network and the probability distribution of the real data e.g. a set of natural images). If two distributions match exactly then the KLd is zero and if not it's large. When they wrote out the math it turned out that the algorithm required two phases, an awake phase where the connections were changed according to the real data, and the sleep phase where the connections were pruned by the spontaneous activity of the network without any input (or dreams). This analogy got a lot of people excited, including Francis Crick and several others tried to test this idea in real brains, but we are still waiting for a convincing result.
  • espeed 10y ago
    Numenta [1] (co-founded by Jeff Hawkins, author of "On Intelligence") has been working on a model of ANN designed around how the brain operates. Hierarchical temporal memory (HTM) [2] is one aspect of their model. 1. http://numenta.com http://numenta.com 2. https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://en.wikipedia.org/wiki/Hierarchical_temporal_memory
• ryukafalz 10y ago
  That's really interesting. From a naïve and purely pop-science perspective... if true, wouldn't this explain why it's hard to remember dreams after waking up?
• jayajay 10y ago
  There is evidence to suggest that sleep and rest are important in verifying which memories to keep, as well. I did some undergraduate research (as a biophysics student) which actually looks at this from a neuroscience perspective. If you are interested in memory and neural networks, read about SPWs, from some scientists in the field: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2039924/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2039924/ http://neuralcircuits.uwm.edu/sharp-wave-ripples/ http://neuralcircuits.uwm.edu/sharp-wave-ripples/ In summary, the hippocampus will organize neurons into clumps which encode spatial dimension. As a rat runs around throughout a track, certain groups of neurons fire[1] depending on where the rat is located. Before the task even starts, hippocampal neurons in a rat actually fire in quick succession (e.g. ABCDEFG). When the task is completed, the same pattern fires in the reverse order (e.g. GFEDCBA). Scientists believe that these SPWs play an important role in memory consolidation. I have always thought of this as "the rat is thinking about where it's going to run to". So I started learning machine learning and landed on things like RBMs. I was so excited because RBMs are reminiscent of this actual phenomenon that occurs in the brain. So, I asked Hinton about it because this is no mere coincidence, and he told me that he had a theory that required reverse order firing during sleep state. If you get a ton of new memories and patterns during wake, sleep may be responsible for not only pruning less important memories, but strengthening more important memories. For example, sleep might be asking the brain for a ranked list of memories, and then strengthens the memories above some threshold, and prunes the rest (just an idea). [1] Synchronized firing, see Kuramoto coupled differential equations: https://en.wikipedia.org/wiki/Kuramoto_model https://en.wikipedia.org/wiki/Kuramoto_model
  • giardini 10y ago
    Anyone know what jayajay means by "SPWs"? Neither article speaks of SPWs but both speak of "sharp waves" or "sharp wave ripples"(designated "SWRs"). Maybe jayajay needs a nap!8-))
    • jayajay 10y ago
      I am not sure why you are attacking me. If you do not understand jargon, please read some more literature. SPW-R complexes. Look it up! Edit: SPWs, Sharp Waves, SWR, etc. are just labels for sharp waves. I can't believe I am getting attacked for using jargon. The names don't matter. The physics and science matters. Richard Feynman. Edit 2: The link I sent uses the same jargon that I use. So, honestly mate, what is your deal here? Are you trolling?
• dicroce 10y ago
  (first, I should say I didn't read the article... I know, I know)... BUT, something about this reminds me of simulated annealing. Just throwing that out there.
• spacehacker 10y ago
  There are brief sections about this in the Deep Learning book by Bengio, Courville and Goodfellow (2016): > 18.2 Because the negative phase involves drawing samples from the model’s distri- bution, we can think of it as finding points that the model believes in strongly. Because the negative phase acts to reduce the probability of those points, they are generally considered to represent the model’s incorrect beliefs about the world. They are frequently referred to in the literature as “hallucinations” or “fantasy particles.” In fact, the negative phase has been proposed as a possible explanation for dreaming in humans and other animals (Crick and Mitchison, 1983), the idea being that the brain maintains a probabilistic model of the world and follows the gradient of log p ̃ while experiencing real events while awake and follows the negative gradient of log p ̃ to minimize log Z while sleeping and experiencing events sampled from the current model. This view explains much of the language used to describe algorithms with a positive and negative phase, but it has not been proven to be correct with neuroscientific experiments. In machine learning models, it is usually necessary to use the positive and negative phase simultaneously, rather than in separate time periods of wakefulness and REM sleep. As we will see in Sec. 19.5, other machine learning algorithms draw samples from the model distribution for other purposes and such algorithms could also provide an account for the function of dream sleep. > 19.5.1 Wake-Sleep One of the main difficulties with training a model to infer h from v is that we do not have a supervised training set with which to train the model. Given a v,we do not know the appropriate h. The mapping from v to h depends on the choice of model family, and evolves throughout the learning process as θ changes. The wake-sleep algorithm (Hinton et al., 1995b; Frey et al., 1996) resolves this problem by drawing samples of both h and v from the model distribution. For example, in a directed model, this can be done cheaply by performing ancestral sampling beginning at h and ending at v. The inference network can then be trained to perform the reverse mapping: predicting which h caused the present v. The main drawback to this approach is that we will only be able to train the inference network on values of v that have high probability under the model. Early in learning, the model distribution will not resemble the data distribution, so the inference network will not have an opportunity to learn on samples that resemble data. Another possible explanation for biological dreaming is that it is providing samples from p(h,v) which can be used to train an inference network to predict h given v. In some senses, this explanation is more satisfying than the partition function explanation. Monte Carlo algorithms generally do not perform well if they are run using only the positive phase of the gradient for several steps then with only the negative phase of the gradient for several steps. Human beings and animals are usually awake for several consecutive hours then asleep for several consecutive hours. It is not readily apparent how this schedule could support Monte Carlo training of an undirected model. Learning algorithms based on maximizing L can be run with prolonged periods of improving q and prolonged periods of improving θ, however. If the role of biological dreaming is to train networks for predicting q, then this explains how animals are able to remain awake for several hours (the longer they are awake, the greater the gap between L and log p(v), but L will remain a lower bound) and to remain asleep for several hours (the generative model itself is not modified during sleep) without damaging their internal models. Of course, these ideas are purely speculative, and there is no hard evidence to suggest that dreaming accomplishes either of these goals. Dreaming may also serve reinforcement learning rather than probabilistic modeling, by sampling synthetic experiences from the animal’s transition model, on which to train the animal’s policy. Or sleep may serve some other purpose not yet anticipated by the machine learning community.