5 ms·

This is curious. During my research into neutral networks we depended on a neuroscience result that found waves of trophic factor (i.e. the stuff used to build

This is curious. During my research into neutral networks we depended on a neuroscience result that found waves of trophic factor (i.e. the stuff used to build up neurons [i.e. learn]) travel in cascading waves from the core of the brain to the periphery. By adding a sense of location and a learning term contingent on the factor we saw learning (xor) prohibited by Minsky's result (not really prohibited because we changed the model). The take away was that our models of ANNs benefit from considering more that just neural signaling (i.e. solution contents and fluid dynamics). Is anyone aware of modern work expanding on that front?

alok-g 5y ago
Not what you asked - I (no expert in neuroscience) had come across a neuroscience article in around 2004, which I could not find again, and on which I never came across anything else either. This said that myelination happens differently in different parts of the brain, culminating everywhere by adulthood. I had found that very interesting. If someone knows more about this, I would like to hear. :-)
whimsicalism 5y ago
Was this research conducted prior to 2005? I find the whole xor result to be pretty irrelevant, which I think is representative of the field writ large. > Is anyone aware of modern work expanding on that front? I am curious about this as well, as well as the general state of biological inspiration in the field. My impression was that biological analogy had largely been pretty much completely moved away from, except as someplace to take inspiration from in terms of what optimization can achieve.
- erikerikson 5y ago
  Yes, 2004 but now I'm curious why you ask. ?
  - whimsicalism 5y ago
    just because 90s-early 00s is the time period where I perceive this style of research to have been at its peak, so I was curious if my assumptions were correct
- chaxor 5y ago
  Itwas moved away from, but in recent years it's been picked up again, with growing intensity. Just look at deepmind for example, which has been publishing with neuroscientist wet labs. Two papers come to mind - one showing the "prefrontal cortex as meta reinforcement learning", and another about distributional temporal difference learning wherein specific neurons were examined.
- ninesnines 5y ago
  I will also be interested to see how this develops our understanding of the relationship between inflammation and psychiatric diseases, e.g. depression
suifbwish 5y ago
That’s pretty cool. What is interesting about your studies is you have the opportunity to apply modified fluid dynamics principles to a neural environment. You might also try an approach of allowing the network to have a refresh cycle where you add byproducts that accumulate as a result of operation and must be cleansed during a separate cycle. In this other cycle you could apply other NN operations such as GAN ect to allow the network to “dream” and create/remove various pathways. This is probably beyond the scope of your work but it seems like pretty cool opportunities for experimentation since there are probably only a few thousand people on the planet that fully understand all of these concepts.
robg 5y ago
Where AI and neural networks still have so much to learn from actual intelligence. Seems like the technical and computational problem set for this century, where a greater understanding of brains will branch out more broadly.
loceng 5y ago
My own evolving hypothesis is that pressure(s) in the body is a very important "setting," as well as the effects of natural Faraday cages that occur throughout the body/brain and at different layers. To note, I'm really curious how a no-to-low inflammatory vs. high inflammatory diet would impact this visual; likewise seeing differences between short-term and long-term, and things like before and after if switching between/cleaning up a diet or going from low inflammation to high inflammation over 1-5-10 years.
l33t2328 5y ago
What was Minsky's result?
- erikerikson 5y ago
  This [0] is probably a reasonable overview though I didn't read it thoroughly, it was on the right track. Basically that Hopfield networks couldn't learn XOR. The significance is that it moved us away from biologically plausible networks and precipitated back propagation which was the dominant learning paradigm in neural networks and mostly remains so. [0] https://alan.do/minskys-and-or-theorem-a-single-perceptron-s-limitations-490c63a02e9f https://alan.do/minskys-and-or-theorem-a-single-perceptron-s...
  - erikerikson 5y ago
    The problem with XOR is that you need a line with dimension N, where N is the number of distinct inputs to the XOR, to separate true from false in the truth space. What I demonstrated was up to 6 dimensional XOR functions being learned in Hopfield networks using Hebbian learning with an additional learning term. I ran a "wave of learning" over the network modifying that terms. In a sense it is a layer free network because no explicit later exists but in another sense the modulation provided a transient layers since the learning term caused different parts of the network to learn at different times out of sync with one another.
    - ad404b8a372f2b9 5y ago
      Could you share a link to your work? I'd love to read more about it.
      - erikerikson 5y ago
        It's rather dated but my dissertation is saved for posterity [0] and includes references and old Octave code. [0] https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT3DiZz/view?usp=drivesdk https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT...

alok-g 5y ago

Not what you asked - I (no expert in neuroscience) had come across a neuroscience article in around 2004, which I could not find again, and on which I never came across anything else either. This said that myelination happens differently in different parts of the brain, culminating everywhere by adulthood. I had found that very interesting. If someone knows more about this, I would like to hear. :-)

whimsicalism 5y ago

Was this research conducted prior to 2005? I find the whole xor result to be pretty irrelevant, which I think is representative of the field writ large. > Is anyone aware of modern work expanding on that front? I am curious about this as well, as well as the general state of biological inspiration in the field. My impression was that biological analogy had largely been pretty much completely moved away from, except as someplace to take inspiration from in terms of what optimization can achieve.

erikerikson 5y ago
Yes, 2004 but now I'm curious why you ask. ?
- whimsicalism 5y ago
  just because 90s-early 00s is the time period where I perceive this style of research to have been at its peak, so I was curious if my assumptions were correct
chaxor 5y ago
Itwas moved away from, but in recent years it's been picked up again, with growing intensity. Just look at deepmind for example, which has been publishing with neuroscientist wet labs. Two papers come to mind - one showing the "prefrontal cortex as meta reinforcement learning", and another about distributional temporal difference learning wherein specific neurons were examined.
ninesnines 5y ago
I will also be interested to see how this develops our understanding of the relationship between inflammation and psychiatric diseases, e.g. depression

suifbwish 5y ago

That’s pretty cool. What is interesting about your studies is you have the opportunity to apply modified fluid dynamics principles to a neural environment. You might also try an approach of allowing the network to have a refresh cycle where you add byproducts that accumulate as a result of operation and must be cleansed during a separate cycle. In this other cycle you could apply other NN operations such as GAN ect to allow the network to “dream” and create/remove various pathways. This is probably beyond the scope of your work but it seems like pretty cool opportunities for experimentation since there are probably only a few thousand people on the planet that fully understand all of these concepts.

robg 5y ago

Where AI and neural networks still have so much to learn from actual intelligence. Seems like the technical and computational problem set for this century, where a greater understanding of brains will branch out more broadly.

loceng 5y ago

My own evolving hypothesis is that pressure(s) in the body is a very important "setting," as well as the effects of natural Faraday cages that occur throughout the body/brain and at different layers. To note, I'm really curious how a no-to-low inflammatory vs. high inflammatory diet would impact this visual; likewise seeing differences between short-term and long-term, and things like before and after if switching between/cleaning up a diet or going from low inflammation to high inflammation over 1-5-10 years.

l33t2328 5y ago

What was Minsky's result?

erikerikson 5y ago
This [0] is probably a reasonable overview though I didn't read it thoroughly, it was on the right track. Basically that Hopfield networks couldn't learn XOR. The significance is that it moved us away from biologically plausible networks and precipitated back propagation which was the dominant learning paradigm in neural networks and mostly remains so. [0] https://alan.do/minskys-and-or-theorem-a-single-perceptron-s-limitations-490c63a02e9f https://alan.do/minskys-and-or-theorem-a-single-perceptron-s...
- erikerikson 5y ago
  The problem with XOR is that you need a line with dimension N, where N is the number of distinct inputs to the XOR, to separate true from false in the truth space. What I demonstrated was up to 6 dimensional XOR functions being learned in Hopfield networks using Hebbian learning with an additional learning term. I ran a "wave of learning" over the network modifying that terms. In a sense it is a layer free network because no explicit later exists but in another sense the modulation provided a transient layers since the learning term caused different parts of the network to learn at different times out of sync with one another.
  - ad404b8a372f2b9 5y ago
    Could you share a link to your work? I'd love to read more about it.
    - erikerikson 5y ago
      It's rather dated but my dissertation is saved for posterity [0] and includes references and old Octave code. [0] https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT3DiZz/view?usp=drivesdk https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT...