9 ms·

> That needs to work before moving to more complexity. It really depends on what level of abstraction you care to simulate. OpenWorm is working at the physics

> That needs to work before moving to more complexity. It really depends on what level of abstraction you care to simulate. OpenWorm is working at the physics and cellular level, far below the concept level as in most deep learning research looking to apply neuroscience discoveries, for example. It’s likely easier to get the concepts of a functional nematode model working or a functional model of memory, attention, or consciousness than a full cellular model of these. More specifically, a thousand cells sounds small in comparison to a thousand layer ResNet with millions of functional units but the mechanics of those cells are significantly more complex than a ReLU unit. Yet the simple ReLU units are functionally very useful and can do much more complex things that we still can’t simulate with spiking neurons. The concepts of receptive fields, cortical columns, local inhibition, winner-take-all, functional modules and how they communicate / are organized may all be relevant and applicable learnings from mapping an organism even if we can’t fully simulate every detail.

Reelin 6y ago
The trouble is that (assuming sufficient computational power) if we can't simulate it then we don't really understand it. It's one thing to say "that's computationally intractable", but entirely another to say "for some reason our computationally tractable model doesn't work, and we don't know why". Present day ANNs may well be inspired by biological systems but (as you noted) they're not even remotely similar in practice. The reality is that for a biological system the wiring diagram is just the tip of the iceberg - there's lots of other significant chemical things going on under the hood. I don't mean to detract from the usefulness of present day ML, just to agree with and elaborate on the original point that was raised (ie that "we have a neural wiring diagram" doesn't actually mean that we have a complete schematic).
- staticassertion 6y ago
  But we understand tons of things without simulating them.
  - Reelin 6y ago
    If you think you understand something, write a simulation which you expect to work based on that understanding, and it doesn't work - did you really understand it?
    - staticassertion 6y ago
      OK, what if the simulation works, did you understand it before?
      - Reelin 6y ago
        Not necessarily. A working simulation (for some testable subset of states) doesn't carry any hard and fast logical implications about your understanding. On the other hand, assuming no errors in implementation then a broken simulation which you had expected to work directly implies that your understanding is flawed.
    - burnte 6y ago
      Maybe, maybe your simulation is just buggy. I can write a simulator of how my wife would react to the news I'm cheating on her, and fail miserably, but I'm quite positive I understand how she would actually react.
      - Reelin 6y ago
        Yes, you have to debug your code. I suspect that the people who implemented OpenWorm are capable of and have done that.
  - gogoincar 6y ago
    Can you give some examples? I'm guessing there is a different in definition of understanding here. As I interpret GP, the claim is you can't describe something in sufficient detail to simulate it, then you don't actually understand it. You may have a higher-order model that generally holds, or holds given some constraints, but that's more of a "what" understanding rather than the higher-bar of "why".
- jimfleming 6y ago
  I'm aware of that and I've done quite a bit of work on both spiking neural networks and modern deep learning. My point is that those complexities are not required to implement many important functional aspects of the brain: most basically "learning" and more specifically, attention, memory, etc. Consciousness may fall into the list of things we can get functional without all of the incidental complexities that evolution brought along the way. It may also critically depend on complexities like multi-channel chemical receptors but since we don't know we can't say either way. It's a tired analogy but we can understand quite a lot about flight and even build a plane without first birthing a bird.
  - kbenson 6y ago
    > It's a tired analogy but we can understand quite a lot about flight and even build a plane without first birthing a bird. The problem is we don't know if we're attempting to solve something as "simple" as flight with a rudimentary understanding of airflow and lift, or if we're attempting to achieve stable planetary orbit without fully understanding gravity and with a rudimentary understanding of chemistry. I think it's still worth trying stuff because it could be closer to the former, and trying more stuff may help us better understand where it is on that spectrum, and because if it is closer to the the harder end, the stuff we're doing is probably so cheap and easy compared to what needs to be done to get to the end that it's a drop in the bucket compared to the eventual output required, even if it adds nothing.
  - echelon 6y ago
    > we can understand quite a lot about flight and even build a plane without first birthing a bird Or fully understanding fluid dynamics and laminar flow. No doubt that the Wright Brothers didn't fully grok it, at least.
  - ramraj07 6y ago
    Your analogy is actually quite apt here - the wright brothers took inspiration from birds but clearly went with a different model of flight, just like ANN field has. The fundamental concept of the neurons are same, but that doesn't mean the complexity is similar. Minimally, whatever the complexity inside a Biological neuron maybe, one fundamental propery we need to obtain is thr connection strengths for the entire connectome, which we don't have. Without that we actually don't know the full connectome even of the simplest organisms, and no one to my knowledge has hence actually studied the kind of algorithms that are running in these systems. I would love to be corrected here of xourse.
__s 6y ago
A good comparison exists with emulators, where transistor level emulation is ill advised for most hardware
Animats 6y ago
It really depends on what level of abstraction you care to simulate. The article starts out "At the Allen Institute for Brain Science in Seattle, a large-scale effort is underway to understand how the 86 billion neurons in the human brain are connected. The aim is to produce a map of all the connections: the connectome. Scientists at the Institute are now reconstructing one cubic millimeter of a mouse brain, the most complex ever reconstructed." So the article is about starting with the wiring diagram and working up. My point is that, even where we already have the wiring diagram for an biological neural system, simulating what it does is just barely starting to work.
jes5199 6y ago
I recently saw this video of living neurons: https://www.youtube.com/watch?v=2TIK9oXc5Wo https://www.youtube.com/watch?v=2TIK9oXc5Wo (I don't actually know the original source of this) and just looking at the way they dance around - they're in motion, they're changing their connections, they're changing shape - is so entirely unlike the software idea of a neural network that it makes me really doubt that we're even remotely on the right track with AI research
- jacobush 6y ago
  Amazing video! And these poor neurons are squished between glass, imagine them crawling around in a 3D space.

Reelin 6y ago

The trouble is that (assuming sufficient computational power) if we can't simulate it then we don't really understand it. It's one thing to say "that's computationally intractable", but entirely another to say "for some reason our computationally tractable model doesn't work, and we don't know why". Present day ANNs may well be inspired by biological systems but (as you noted) they're not even remotely similar in practice. The reality is that for a biological system the wiring diagram is just the tip of the iceberg - there's lots of other significant chemical things going on under the hood. I don't mean to detract from the usefulness of present day ML, just to agree with and elaborate on the original point that was raised (ie that "we have a neural wiring diagram" doesn't actually mean that we have a complete schematic).

staticassertion 6y ago
But we understand tons of things without simulating them.
- Reelin 6y ago
  If you think you understand something, write a simulation which you expect to work based on that understanding, and it doesn't work - did you really understand it?
  - staticassertion 6y ago
    OK, what if the simulation works, did you understand it before?
    - Reelin 6y ago
      Not necessarily. A working simulation (for some testable subset of states) doesn't carry any hard and fast logical implications about your understanding. On the other hand, assuming no errors in implementation then a broken simulation which you had expected to work directly implies that your understanding is flawed.
  - burnte 6y ago
    Maybe, maybe your simulation is just buggy. I can write a simulator of how my wife would react to the news I'm cheating on her, and fail miserably, but I'm quite positive I understand how she would actually react.
    - Reelin 6y ago
      Yes, you have to debug your code. I suspect that the people who implemented OpenWorm are capable of and have done that.
- gogoincar 6y ago
  Can you give some examples? I'm guessing there is a different in definition of understanding here. As I interpret GP, the claim is you can't describe something in sufficient detail to simulate it, then you don't actually understand it. You may have a higher-order model that generally holds, or holds given some constraints, but that's more of a "what" understanding rather than the higher-bar of "why".
jimfleming 6y ago
I'm aware of that and I've done quite a bit of work on both spiking neural networks and modern deep learning. My point is that those complexities are not required to implement many important functional aspects of the brain: most basically "learning" and more specifically, attention, memory, etc. Consciousness may fall into the list of things we can get functional without all of the incidental complexities that evolution brought along the way. It may also critically depend on complexities like multi-channel chemical receptors but since we don't know we can't say either way. It's a tired analogy but we can understand quite a lot about flight and even build a plane without first birthing a bird.
- kbenson 6y ago
  > It's a tired analogy but we can understand quite a lot about flight and even build a plane without first birthing a bird. The problem is we don't know if we're attempting to solve something as "simple" as flight with a rudimentary understanding of airflow and lift, or if we're attempting to achieve stable planetary orbit without fully understanding gravity and with a rudimentary understanding of chemistry. I think it's still worth trying stuff because it could be closer to the former, and trying more stuff may help us better understand where it is on that spectrum, and because if it is closer to the the harder end, the stuff we're doing is probably so cheap and easy compared to what needs to be done to get to the end that it's a drop in the bucket compared to the eventual output required, even if it adds nothing.
- echelon 6y ago
  > we can understand quite a lot about flight and even build a plane without first birthing a bird Or fully understanding fluid dynamics and laminar flow. No doubt that the Wright Brothers didn't fully grok it, at least.
- ramraj07 6y ago
  Your analogy is actually quite apt here - the wright brothers took inspiration from birds but clearly went with a different model of flight, just like ANN field has. The fundamental concept of the neurons are same, but that doesn't mean the complexity is similar. Minimally, whatever the complexity inside a Biological neuron maybe, one fundamental propery we need to obtain is thr connection strengths for the entire connectome, which we don't have. Without that we actually don't know the full connectome even of the simplest organisms, and no one to my knowledge has hence actually studied the kind of algorithms that are running in these systems. I would love to be corrected here of xourse.

__s 6y ago

A good comparison exists with emulators, where transistor level emulation is ill advised for most hardware

Animats 6y ago

It really depends on what level of abstraction you care to simulate. The article starts out "At the Allen Institute for Brain Science in Seattle, a large-scale effort is underway to understand how the 86 billion neurons in the human brain are connected. The aim is to produce a map of all the connections: the connectome. Scientists at the Institute are now reconstructing one cubic millimeter of a mouse brain, the most complex ever reconstructed." So the article is about starting with the wiring diagram and working up. My point is that, even where we already have the wiring diagram for an biological neural system, simulating what it does is just barely starting to work.

jes5199 6y ago

I recently saw this video of living neurons: https://www.youtube.com/watch?v=2TIK9oXc5Wo https://www.youtube.com/watch?v=2TIK9oXc5Wo (I don't actually know the original source of this) and just looking at the way they dance around - they're in motion, they're changing their connections, they're changing shape - is so entirely unlike the software idea of a neural network that it makes me really doubt that we're even remotely on the right track with AI research

jacobush 6y ago
Amazing video! And these poor neurons are squished between glass, imagine them crawling around in a 3D space.