4 ms·

I really liked the mention of the "three narratives" of neural networks: 1) human brain 2) nonlinear "squashing and folding" of the input space 3) probabilistic

I really liked the mention of the "three narratives" of neural networks: 1) human brain 2) nonlinear "squashing and folding" of the input space 3) probabilistic / latent variables. Seeing different ways to look at something is always refreshing and opens up deep learning to rich analytical techniques developed in optimization, type theory, etc. HN: how do YOU think about NNs? As a matter of pure preference, most of my daydreaming of NNs comes from my inductive biases of (1) and (2), with a few ideas from MCMC methods and optimization thrown in the mix. > Representation theory One question I've always wondered is whether it is better to get a network to (1) learn representation transformations A->B->C->D via a series of layers, or (2) try to learn A->D via a single (highly nonlinear) transformation. Obviously the former lends itself better to analytical understanding, and this seems to be what emerges when you train ImageNet. However, do researchers have any control over whether the learned net does (1) or (2)? I'm thinking that scattering "skip arcs" that pass gradients through layers (or an LSTM) would cause representations to mix more between layers. > Each layer is a function, acting on the output of a previous layer. As a whole, the network is a chain of composed functions. This chain of composed functions is optimized to perform a task. This is certainly true for the majority of models in use today, but I wonder if this will remain true for future architectures. In practice, most layer types store some state, and implement "forward" and "backward" functions that may utilize that state. One could argue that state is not necessary if we only consider the network to be defined by its feedforward passes (de-coupled from the optimization problem, which requires the "backward" functions). But the neuroscience narrative argues that function and learning cannot be decoupled. In the case of a network that is supposed to update it's own weights during normal operation (i.e. topic discovery), the functional narrative is less clear to me. Thoughts?

seiji 11y ago
Starting at 1 is always amusing when talking to computer people. Computer people seem happy to dismiss most of the brain because "neural network—all we need is data!" But, the brain isn't just a bowl of neurons freely connecting to each other. There's a quite rigid structure to brains that give rise to things like the perception of free will and language acquisition (i.e. why no other animals have language). The brain is organized in layers, but the layers don't care about respecting order. layer 5 can be directly connected to layer 0, forwards and backwards, as well as anywhere else in-between. Brains rely a lot more on inhibiting signals than generating signals. Plus, depending on how you count, we are made up of three brains put together: left, right, cerebellum, and those are seldom considered in our current fascination with single-task "AI" systems. You can put a shark fin on your head then swim for a while, but you're still not a shark. To go deeper down the rabbit hole, start off with http://www.amazon.com/Cerebellum-Brain-Implicit-Press-Science-ebook/dp/B005DKQQG4 http://www.amazon.com/Cerebellum-Brain-Implicit-Press-Scienc... (warning: dense) or skim http://neuroscience.uth.tmc.edu/s3/chapter05.html http://neuroscience.uth.tmc.edu/s3/chapter05.html or use http://www.amazon.com/Mapping-Mind-Rita-Carter/dp/0520266285 http://www.amazon.com/Mapping-Mind-Rita-Carter/dp/0520266285 as an easy getting started reference. As for "neuroscience narrative argues that function and learning cannot be decoupled" — saying learning is a bit of a stretch. The brain only cares about correlating coincident events ("neurons that fire together wire together") and wiring together neurons that fire at the same time ends up causing what we see as emergent behaviors (semantic memories bound to episodic memory, operant conditioning, etc).
- colah3 11y ago
  The problem I worry about with the neuroscience narrative is that it seems easy to justify everything. As I understand it, we don't understand the brain very well, and there's lots of different theories. A fair number of neural net papers will make appeals to neuroscience to motivate something. Whenever I read these, I wonder to what extent neuroscience could be cited to argue the opposite. There's a lot of ex-neuroscientists in deep learning. Amusingly, my experience has been that they're generally the most skeptical of the neuroscience narrative. Of course, some amazing researchers seem to really like the neuroscience narrative -- this is obviously a strong point in its favor. I don't mean to dismiss it. I just think that one wants to be kind of careful when using it as a vantage for thinking.
  - seiji 11y ago
    I wonder to what extent neuroscience could be cited to argue the opposite. That's a great point. Often, in brains and bodies in general, the same mechanism causes more than one result. Sometimes even the absence of the mechanism causes the same result too. Biological understanding has a high number of latent variables we may not even know exist. (plus, arguing "because biology" is just a few steps away from arguing "because quantum" when motivated by vague optimism.) The best overall insight from neuroscience is: AI, as envisioned by the 1950s pioneers, is possible. Brains are complex, but not irreducibly complex. Just slam some gametes together and a new brain appears. We can, at some level, eventually, get it all working digitally. The best useful insight from neuroscience seems to be: the never-ending a/b test of natural selection converged on neurons+synapses as a universal computation system. But, neurons+synapses aren't quite enough detail to replicate a brain—that's like saying our computers use transistors, so all we need is transistors to build a CPU. Sure, that's important, but the structure and topology and interconnections are what make the magic happens. What's the end result? More design or more brute force? Our GPUs are still Moore's Lawing themselves every few years. We are in early days of experimentation-at-scale here. Since we will always need an overwhelmingly higher number of model parameters than data, we can't intricately tune models by hand (alternative: make an ANN to optimize ANN architectures). One breakthrough we don't really have is a minimum-description for ANN architectures. If that were to come about, we could have "ANN DNA" then 'breed' ANNs together so their topologies and functions merge to generate a (hopefully) more fit offspring. Then just keep repeating until they rise up and destroy us all.
colah3 11y ago
> Seeing different ways to look at something is always refreshing ... how do YOU think about NNs? A year or two ago, when I was really doing the wandering researcher thing, I made a point of asking lots of people at different groups how they saw deep learning. I think these three narratives capture most of what I heard, although a people blended them in different ways. Certain views are more common in particular groups. For example, people at the Montreal group seem more inclined to think about things from the representations narrative than other groups. (Although, I think I have just about everyone else beat in my extreme version of seeing everything from the manifold perspective. :P ) I feel like there's some pretty interesting quasi-sociological work to be done on deep learning. Because the community has grown so fast, and there's a lot of variety in how people think. >> ... chain of composed functions is optimized.. > This is certainly true for the majority of models in use today, but I wonder if this will remain true for future architectures. I suspect there's something very fundamental about chains of composed functions. I can't formalize my feeling into a strong argument though. > In the case of a network that is supposed to update it's own weights during normal operation (i.e. topic discovery), the functional narrative is less clear to me. I don't think matters too much. The functional graph becomes more complicated, but it's just a matter of the output of one function going to multiple places.
- ericjang 11y ago
  Cool, thanks again for your great post. Big fan of your articles. > The functional graph becomes more complicated Indeed. I once tried to come up with a "higher order function" that takes in a feedforward network and computes a separate function that computes the backward pass (like Theano's autodiff, but abstraction at the layer-level rather than ops). Here's a diagram for a simple forward MLP (left to right), with the backward pass network below (right to left). I found this hard to work with, because of the explosion in the size of the computational graph when you try to decouple optimization / function. I notice something similar when trying to unroll a RNN visually across time. http://imgur.com/ATNwknh http://imgur.com/ATNwknh Let me know if this is way off base from what you were talking about in your post.
chestervonwinch 11y ago
For classification, I think of it simply as a nonlinear transformation + multivariable logistic regression where parameters are learned jointly. In particular the nonlinear transformation is assumed to be of the form of some number of affine transformations, each followed by a nonlinear component-wise mapping. I tend to intentionally avoid brain comparisons because: 1) there's more than enough of that already 2) I don't know enough of the neurophysiology to speculate. I'd like to see some mathematical analysis on what classes of function are more efficiently represented (and/or learned) by networks with increasing numbers of layers.
tfgg 11y ago
This is an unformed thought, but to me the philosophy of emergence, intelligence and deep learning have something to do with each other. In emergent systems, you have a stack of levels of phenomena, and you can describe the phenomena at one level without reference to the details of the lower levels. It's like those details have been blurred out and you're left with a compressed representation with just enough information to be useful at the new level (e.g. pixels->edges, renormalization in physics). The success of architectures with multiple layers, including bottlenecks in layer size, seems very similar, and show that this is a good way to look at the world. This seems to be quite a natural way that humans see the world, and in fact we might not think of something that doesn't behave this way as being intelligent -- we want there to be some sort of compressed 'trick' that doesn't just brute force the data.
- unabst 11y ago
  I am surprised you didn't use the word abstraction even once in your description of emergent systems. I admit I have some catching up to do on the state of that topic but I couldn't help but notice the symmetry between the stack of phenomena as you described it and abstraction layers of basic computational systems. APIs and programming languages expose an interface that completely abstracts away the implementation beneath it. And much of the layers added have indeed emerged in the sense that they have been built based on what we thought might be practical on top of established layers. Anyway, just some food for that unformed thought of yours ;)

seiji 11y ago

Starting at 1 is always amusing when talking to computer people. Computer people seem happy to dismiss most of the brain because "neural network—all we need is data!" But, the brain isn't just a bowl of neurons freely connecting to each other. There's a quite rigid structure to brains that give rise to things like the perception of free will and language acquisition (i.e. why no other animals have language). The brain is organized in layers, but the layers don't care about respecting order. layer 5 can be directly connected to layer 0, forwards and backwards, as well as anywhere else in-between. Brains rely a lot more on inhibiting signals than generating signals. Plus, depending on how you count, we are made up of three brains put together: left, right, cerebellum, and those are seldom considered in our current fascination with single-task "AI" systems. You can put a shark fin on your head then swim for a while, but you're still not a shark. To go deeper down the rabbit hole, start off with http://www.amazon.com/Cerebellum-Brain-Implicit-Press-Science-ebook/dp/B005DKQQG4 http://www.amazon.com/Cerebellum-Brain-Implicit-Press-Scienc... (warning: dense) or skim http://neuroscience.uth.tmc.edu/s3/chapter05.html http://neuroscience.uth.tmc.edu/s3/chapter05.html or use http://www.amazon.com/Mapping-Mind-Rita-Carter/dp/0520266285 http://www.amazon.com/Mapping-Mind-Rita-Carter/dp/0520266285 as an easy getting started reference. As for "neuroscience narrative argues that function and learning cannot be decoupled" — saying learning is a bit of a stretch. The brain only cares about correlating coincident events ("neurons that fire together wire together") and wiring together neurons that fire at the same time ends up causing what we see as emergent behaviors (semantic memories bound to episodic memory, operant conditioning, etc).

colah3 11y ago
The problem I worry about with the neuroscience narrative is that it seems easy to justify everything. As I understand it, we don't understand the brain very well, and there's lots of different theories. A fair number of neural net papers will make appeals to neuroscience to motivate something. Whenever I read these, I wonder to what extent neuroscience could be cited to argue the opposite. There's a lot of ex-neuroscientists in deep learning. Amusingly, my experience has been that they're generally the most skeptical of the neuroscience narrative. Of course, some amazing researchers seem to really like the neuroscience narrative -- this is obviously a strong point in its favor. I don't mean to dismiss it. I just think that one wants to be kind of careful when using it as a vantage for thinking.
- seiji 11y ago
  I wonder to what extent neuroscience could be cited to argue the opposite. That's a great point. Often, in brains and bodies in general, the same mechanism causes more than one result. Sometimes even the absence of the mechanism causes the same result too. Biological understanding has a high number of latent variables we may not even know exist. (plus, arguing "because biology" is just a few steps away from arguing "because quantum" when motivated by vague optimism.) The best overall insight from neuroscience is: AI, as envisioned by the 1950s pioneers, is possible. Brains are complex, but not irreducibly complex. Just slam some gametes together and a new brain appears. We can, at some level, eventually, get it all working digitally. The best useful insight from neuroscience seems to be: the never-ending a/b test of natural selection converged on neurons+synapses as a universal computation system. But, neurons+synapses aren't quite enough detail to replicate a brain—that's like saying our computers use transistors, so all we need is transistors to build a CPU. Sure, that's important, but the structure and topology and interconnections are what make the magic happens. What's the end result? More design or more brute force? Our GPUs are still Moore's Lawing themselves every few years. We are in early days of experimentation-at-scale here. Since we will always need an overwhelmingly higher number of model parameters than data, we can't intricately tune models by hand (alternative: make an ANN to optimize ANN architectures). One breakthrough we don't really have is a minimum-description for ANN architectures. If that were to come about, we could have "ANN DNA" then 'breed' ANNs together so their topologies and functions merge to generate a (hopefully) more fit offspring. Then just keep repeating until they rise up and destroy us all.

colah3 11y ago

> Seeing different ways to look at something is always refreshing ... how do YOU think about NNs? A year or two ago, when I was really doing the wandering researcher thing, I made a point of asking lots of people at different groups how they saw deep learning. I think these three narratives capture most of what I heard, although a people blended them in different ways. Certain views are more common in particular groups. For example, people at the Montreal group seem more inclined to think about things from the representations narrative than other groups. (Although, I think I have just about everyone else beat in my extreme version of seeing everything from the manifold perspective. :P ) I feel like there's some pretty interesting quasi-sociological work to be done on deep learning. Because the community has grown so fast, and there's a lot of variety in how people think. >> ... chain of composed functions is optimized.. > This is certainly true for the majority of models in use today, but I wonder if this will remain true for future architectures. I suspect there's something very fundamental about chains of composed functions. I can't formalize my feeling into a strong argument though. > In the case of a network that is supposed to update it's own weights during normal operation (i.e. topic discovery), the functional narrative is less clear to me. I don't think matters too much. The functional graph becomes more complicated, but it's just a matter of the output of one function going to multiple places.

ericjang 11y ago
Cool, thanks again for your great post. Big fan of your articles. > The functional graph becomes more complicated Indeed. I once tried to come up with a "higher order function" that takes in a feedforward network and computes a separate function that computes the backward pass (like Theano's autodiff, but abstraction at the layer-level rather than ops). Here's a diagram for a simple forward MLP (left to right), with the backward pass network below (right to left). I found this hard to work with, because of the explosion in the size of the computational graph when you try to decouple optimization / function. I notice something similar when trying to unroll a RNN visually across time. http://imgur.com/ATNwknh http://imgur.com/ATNwknh Let me know if this is way off base from what you were talking about in your post.

chestervonwinch 11y ago

For classification, I think of it simply as a nonlinear transformation + multivariable logistic regression where parameters are learned jointly. In particular the nonlinear transformation is assumed to be of the form of some number of affine transformations, each followed by a nonlinear component-wise mapping. I tend to intentionally avoid brain comparisons because: 1) there's more than enough of that already 2) I don't know enough of the neurophysiology to speculate. I'd like to see some mathematical analysis on what classes of function are more efficiently represented (and/or learned) by networks with increasing numbers of layers.

tfgg 11y ago

This is an unformed thought, but to me the philosophy of emergence, intelligence and deep learning have something to do with each other. In emergent systems, you have a stack of levels of phenomena, and you can describe the phenomena at one level without reference to the details of the lower levels. It's like those details have been blurred out and you're left with a compressed representation with just enough information to be useful at the new level (e.g. pixels->edges, renormalization in physics). The success of architectures with multiple layers, including bottlenecks in layer size, seems very similar, and show that this is a good way to look at the world. This seems to be quite a natural way that humans see the world, and in fact we might not think of something that doesn't behave this way as being intelligent -- we want there to be some sort of compressed 'trick' that doesn't just brute force the data.

unabst 11y ago
I am surprised you didn't use the word abstraction even once in your description of emergent systems. I admit I have some catching up to do on the state of that topic but I couldn't help but notice the symmetry between the stack of phenomena as you described it and abstraction layers of basic computational systems. APIs and programming languages expose an interface that completely abstracts away the implementation beneath it. And much of the layers added have indeed emerged in the sense that they have been built based on what we thought might be practical on top of established layers. Anyway, just some food for that unformed thought of yours ;)