3 ms·

> Study urges caution when comparing neural networks to the brain They keep telling me this and, yet, I can't stop doing it. The more I learn about neural netw

> Study urges caution when comparing neural networks to the brain They keep telling me this and, yet, I can't stop doing it. The more I learn about neural networks, the more I feel like I understand my own brain (whether accurate or not). And conversely, the more I think about thinking, the better my theories about how I'd build ML-based system to solve specific problems (admittedly, most untested). Neural networks seem like too useful of a model to simply give up because they aren't completely accurate. Of course, this is all just for personal use - mostly introspection. I wouldn't exactly do medical work based on the model.

colechristensen 4y ago
People always do things like this, like an astrologer trying to say something profound using Heisenberg's uncertainty principle applying it to your romantic relationships, or whatever. These "just so" stories are attractive but it is quite important to realize a metaphor which is intuitive and you perceive as useful is nothing at all like the process for finding real scientific truth. There is also a lot of introspective value to modeling the world as being controlled by mysterious gods who are pleased or appalled at your behavior and that's why good and bad things happen. Perhaps useful for some people but nothing at all like truth. https://en.wikipedia.org/wiki/Just-so_story https://en.wikipedia.org/wiki/Just-so_story
- tobyjsullivan 4y ago
  This is definitely what I was thinking about when I started my comment. I torpedoed my own point with the comment about the model being "too useful". Your explanation is much better. When I said "I can't stop," I was referring more to this tendency to borrow models to explain unrelated systems. It's just a thing my brain wants to do and I can't help it (and again, I seem to convince myself that it's somehow accurate or useful even if, rationally, I'm quite sure it's not).
  - colechristensen 4y ago
    Yup, unrelated things do indeed often look and behave in similar ways. Humans are just a little bit overdriven to find patterns and end up finding some that don’t exist. It’s a useful trait for finding difficult patterns and there’s probably a stable point to maximize benefits which lands on finding a few too many.
- JJMcJ 4y ago
  Or how the Matrix movies came out and suddenly "Are we living in a simulation?" became an immensely important philosophical question.
  - goatlover 4y ago
    It isn't really that much different than, "Are we living in a dream?", of the movie Inception that's been asked since antiquity. Is the world some sort of illusion? How would we know?
heavyset_go 4y ago
I once had the same feeling, but it was dispelled by acknowledging that NN neurons are not even approximations of how neurons work. At most, NNs are inspired by the topology of a subset of neurons, and that's where the similarity between NNs and biological neurons stop. It's like the connection between objects in real life and objects in programming. They're both useful abstractions that are inspired by things in the real world, but the similarities to things in the real world stop there. Neurons have a lot going on, they send and receive signals through a multitude of mediums, not just neural impulses, and they're capable of plasticity when it comes to the connections they make between other neurons. Neurons also don't have simplistic activation functions, they're capable of doing a lot more with the information they receive and send. Also, gradient descent and back propogation don't take place in any part of the brain. Through that lens, I see NNs as if they're like really complex and impressive Markov chain generators. They can produce results that look intelligent, but it's just statistical correlations, and not at all how the brain works.
- BrianOnHN 4y ago
  > It's like the connection between objects in real life and objects in programming What if this a more accurate representation: "It's like the connection between dictionaries in real life and dictionaries in programming." The neuron could be implementing NN theory in a way that is optimized for it's environment.
- a_wild_dandan 4y ago
  What attributes does a system need for you to accept its comparison to a brain/neuron? Without defining what's essential, I'm nervous to call the comparison insufficient. If a topological subset of neurons isn't good enough, what do we need in addition/instead? If we stuff NNs full of complicated (how complicated?) activation functions, does that new system do the trick? Or add...47 new "neuron" variants? Or swap the learning scheme from gradient descent to something fancier? (For that matter, do we even know what the brain's scheme is, and why GA/back prop isn't an acceptably extremely crude approximation of it?) The brain is so unimaginably intricate. Our models are hilariously simple in contrast, of course. But what of those mismatches are differences in kind vs. differences in magnitude?

colechristensen 4y ago

People always do things like this, like an astrologer trying to say something profound using Heisenberg's uncertainty principle applying it to your romantic relationships, or whatever. These "just so" stories are attractive but it is quite important to realize a metaphor which is intuitive and you perceive as useful is nothing at all like the process for finding real scientific truth. There is also a lot of introspective value to modeling the world as being controlled by mysterious gods who are pleased or appalled at your behavior and that's why good and bad things happen. Perhaps useful for some people but nothing at all like truth. https://en.wikipedia.org/wiki/Just-so_story https://en.wikipedia.org/wiki/Just-so_story

tobyjsullivan 4y ago
This is definitely what I was thinking about when I started my comment. I torpedoed my own point with the comment about the model being "too useful". Your explanation is much better. When I said "I can't stop," I was referring more to this tendency to borrow models to explain unrelated systems. It's just a thing my brain wants to do and I can't help it (and again, I seem to convince myself that it's somehow accurate or useful even if, rationally, I'm quite sure it's not).
- colechristensen 4y ago
  Yup, unrelated things do indeed often look and behave in similar ways. Humans are just a little bit overdriven to find patterns and end up finding some that don’t exist. It’s a useful trait for finding difficult patterns and there’s probably a stable point to maximize benefits which lands on finding a few too many.
JJMcJ 4y ago
Or how the Matrix movies came out and suddenly "Are we living in a simulation?" became an immensely important philosophical question.
- goatlover 4y ago
  It isn't really that much different than, "Are we living in a dream?", of the movie Inception that's been asked since antiquity. Is the world some sort of illusion? How would we know?

heavyset_go 4y ago

I once had the same feeling, but it was dispelled by acknowledging that NN neurons are not even approximations of how neurons work. At most, NNs are inspired by the topology of a subset of neurons, and that's where the similarity between NNs and biological neurons stop. It's like the connection between objects in real life and objects in programming. They're both useful abstractions that are inspired by things in the real world, but the similarities to things in the real world stop there. Neurons have a lot going on, they send and receive signals through a multitude of mediums, not just neural impulses, and they're capable of plasticity when it comes to the connections they make between other neurons. Neurons also don't have simplistic activation functions, they're capable of doing a lot more with the information they receive and send. Also, gradient descent and back propogation don't take place in any part of the brain. Through that lens, I see NNs as if they're like really complex and impressive Markov chain generators. They can produce results that look intelligent, but it's just statistical correlations, and not at all how the brain works.

BrianOnHN 4y ago
> It's like the connection between objects in real life and objects in programming What if this a more accurate representation: "It's like the connection between dictionaries in real life and dictionaries in programming." The neuron could be implementing NN theory in a way that is optimized for it's environment.
a_wild_dandan 4y ago
What attributes does a system need for you to accept its comparison to a brain/neuron? Without defining what's essential, I'm nervous to call the comparison insufficient. If a topological subset of neurons isn't good enough, what do we need in addition/instead? If we stuff NNs full of complicated (how complicated?) activation functions, does that new system do the trick? Or add...47 new "neuron" variants? Or swap the learning scheme from gradient descent to something fancier? (For that matter, do we even know what the brain's scheme is, and why GA/back prop isn't an acceptably extremely crude approximation of it?) The brain is so unimaginably intricate. Our models are hilariously simple in contrast, of course. But what of those mismatches are differences in kind vs. differences in magnitude?