4 ms·

It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from

by andy99 3y ago

It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.

voidhorse 3y ago
The concept has a long history going back to the 1940s (Pitts and McCulloch) and was based on their ideas at the time about how neurons in the brain functioned. That's why the association with the brain is still with us, even if our biological and neurological understanding has evolved. I wish the industry would be more transparent about where all these brilliant ideas come from (spoiler alert: academics from the 1930s-70s, not "genius founders").
nightski 3y ago
The inspiration isn't top down. It's bottom up.
somewhereoutth 3y ago
Bear in mind that for most (all?) advanced economies the state runs about 40-50%, so top down is working fine for about half of the economy.
- polotics 3y ago
  Going on a tangent sorry, you started it! So... in a private company the employees are very often a lot more beholden to the managers/owners, so top-down command and control can be applied a lot more than in an administration where job security is often much higher. For better or for worse, this can result in a lot more bottom-up decision-making... In view of general wastefulness, lack of longer term (environmental) planning, I would challenge the "working fine" a bit.
famouswaffles 3y ago
ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/ https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
- p1esk 3y ago
  A quote from the link you posted: "Then, when you refer to “Lambda”, “ChatGPT”, “Bard”, or “Claude” then, it’s not the model weights that you are referring to. It’s the dataset." Yet all four of these examples use the same model architecture (transformers).
  - famouswaffles 3y ago
    "Everything else is a means to an end in efficiently delivery compute..." Without tweaking anything (so not RWKV), you could train a GPT level RNN...if you had the compute to burn.
    - p1esk 3y ago
      We don't know that. No one has demonstrated it. It's very likely that at larger scales, for a given amount of compute you cannot train a traditional RNN to be as a good as a transformer.
      - famouswaffles 3y ago
        We are saying the same thing. Transformers are more compute efficient than RNNs. Nobody is denying that but the switch from RNNs didn't precede some performance wall(i.e it's not like we were training bigger RNNs that weren't getting better). We use Transformers today in large part because they got rid of recursion and in effect could massively parallelize compute.
- HarHarVeryFunny 3y ago
  Sure - one only needs a sufficiently powerful learning architecture to extract the signal from the data, and assuming that is the case then what is learnt depends on the data not the architecture. That said, I don't think that LLMs are yet close to fully utilizing the training data. e.g. they are prone to hallucinating due to not thinking ahead and backing themselves into a corner where they have to say something. One obvious improvement for that one is more forward looking prediction (i.e. "thinking ahead" - engage brain before opening mouth), which for LLMs can be addressed by tree of thought rollouts and RL learning. So, while architecture is not important if all architectures are equally powerful (in which case you'll learn what's available to learn), it certainly does matter if not all are fully up to the job, as it would appear none currently are.
demosthanos 3y ago
If you look at the actual process for building these deep neural networks, it's actually far more emergent and bottom-up than you might think just reading the news. I took a deep learning class before things really took off [0], and what we were taught was that a lot of deep learning research was just trying different structures to see what worked. Even the people who were good at it didn't base their architectures on strong theories for how and why things would work, they just had developed an intuition through lots of trial and error. All the explanations for what the various structures in a deep learning model are doing are post hoc—they're observations made of the behavior of the systems after we build them. [0] Edit: Not just a random deep learning class either. The TA for that class created AI Dungeon that semester. The professor came in one day with a story of how they accidentally racked up tens of thousands of dollars in Google Cloud bills in 24 hours when that went viral.
- dundarious 3y ago
  Early architecture/structural engineering (figuring out arches, up to building cathedrals) was done by "just trying different structures to see what worked". Calling this "emergent" is not what is typically meant by the word -- we don't consider cathedrals to be "emergent structures". It's fair to say that many people imprecisely use the term, but then you are contributing to its drift into becoming (or perhaps always having been) a floating signifier.
  - wolframhempel 3y ago
    But Cathedrals still had central planning, blueprints, ropes stuck in the ground to outline what should be built. Thermites on the other hand use a process called "Stigmergy" in which work allocation and building structures emerge organically without central coordination.
    - dundarious 3y ago
      They justified their point by saying the research was "emergent". My point is that almost all pre-modern research was "emergent" by their usage, and a large amount of contemporaneous research is "emergent" as well. That would be slightly annoying, but they did this to then justify points about how the resulting model is "emergent". The research process, and the resulting output, can and (my main point) almost always do, differ as to whether they are truly emergent.
FreddaGennaio 3y ago
It's insightful to observe the difference between the emergent behavior of natural systems and the architectured approach often used in AI development. The focus on neural networks, while beneficial for certain applications, might overlook the potential of emergent, decentralized systems that could offer solutions for complex, dynamic challenges. Hivekit's work is a step towards exploring these alternative approaches, emphasizing the importance of communication and interconnectedness in creating effective complex systems.
HarHarVeryFunny 3y ago
> when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected I wouldn't characterize it like that ... The brain has a specific learning architecture / dynamics that has been created via evolution under selection pressure to learn (i.e predict outcomes) better. As a product of evolution I wouldn't want to call it "architected" or a top-down design, but for the time being it's the only example we have of such a successful learning system so it would make sense to copy what evolution has done, which means copying how it works on all levels. A simpler example is convolutional neural nets for vision which were explicitly designed to simplistically mimic some of the behavior of our visual system with it's multi-layer (V1, V2, etc) architecture and local learning rules. Sure there's emergent behavior occurring when data is fed into such a system (both brain's visual system or CNN), such as the pattern detectors we see emerge at lowest level, but this emergent behavior is a result of the overall architecture.
tlarkworthy 3y ago
The retina, dopamine system, basal ganglia, cerebellum, entorhinal cortex all have very specialised coarse grained structures for their specialisations. The eye is part of the brain for instance. The details are emergent but there is most definitely strong top down architecture encoded in DNA
guyomes 3y ago
Actually the link between economy and neural network is quite strong historically. Friedrich Hayek is an economist who invented the word catallaxy to describe "the order brought about by the mutual adjustment of many individual economies in a market" [2]. Then he went on to work on a global theory of the brain, and was cited by Frank Rosenblatt in 1958 as an inspiration for the model of the perceptron [3]. [1]: https://en.wikipedia.org/wiki/Friedrich_Hayek#Economic_calculation_problem https://en.wikipedia.org/wiki/Friedrich_Hayek#Economic_calcu... [2]: https://en.wikipedia.org/wiki/Catallaxy https://en.wikipedia.org/wiki/Catallaxy [3]: https://citeseerx.ist.psu.edu/doc_view/pid/2688969848b75336891b1e58a2b6ee433343867c https://citeseerx.ist.psu.edu/doc_view/pid/2688969848b753368...

by andy99 3y ago

voidhorse 3y ago

The concept has a long history going back to the 1940s (Pitts and McCulloch) and was based on their ideas at the time about how neurons in the brain functioned. That's why the association with the brain is still with us, even if our biological and neurological understanding has evolved. I wish the industry would be more transparent about where all these brilliant ideas come from (spoiler alert: academics from the 1930s-70s, not "genius founders").

nightski 3y ago

The inspiration isn't top down. It's bottom up.

somewhereoutth 3y ago

Bear in mind that for most (all?) advanced economies the state runs about 40-50%, so top down is working fine for about half of the economy.

polotics 3y ago
Going on a tangent sorry, you started it! So... in a private company the employees are very often a lot more beholden to the managers/owners, so top-down command and control can be applied a lot more than in an administration where job security is often much higher. For better or for worse, this can result in a lot more bottom-up decision-making... In view of general wastefulness, lack of longer term (environmental) planning, I would challenge the "working fine" a bit.

famouswaffles 3y ago

ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/ https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.

p1esk 3y ago
A quote from the link you posted: "Then, when you refer to “Lambda”, “ChatGPT”, “Bard”, or “Claude” then, it’s not the model weights that you are referring to. It’s the dataset." Yet all four of these examples use the same model architecture (transformers).
- famouswaffles 3y ago
  "Everything else is a means to an end in efficiently delivery compute..." Without tweaking anything (so not RWKV), you could train a GPT level RNN...if you had the compute to burn.
  - p1esk 3y ago
    We don't know that. No one has demonstrated it. It's very likely that at larger scales, for a given amount of compute you cannot train a traditional RNN to be as a good as a transformer.
    - famouswaffles 3y ago
      We are saying the same thing. Transformers are more compute efficient than RNNs. Nobody is denying that but the switch from RNNs didn't precede some performance wall(i.e it's not like we were training bigger RNNs that weren't getting better). We use Transformers today in large part because they got rid of recursion and in effect could massively parallelize compute.
HarHarVeryFunny 3y ago
Sure - one only needs a sufficiently powerful learning architecture to extract the signal from the data, and assuming that is the case then what is learnt depends on the data not the architecture. That said, I don't think that LLMs are yet close to fully utilizing the training data. e.g. they are prone to hallucinating due to not thinking ahead and backing themselves into a corner where they have to say something. One obvious improvement for that one is more forward looking prediction (i.e. "thinking ahead" - engage brain before opening mouth), which for LLMs can be addressed by tree of thought rollouts and RL learning. So, while architecture is not important if all architectures are equally powerful (in which case you'll learn what's available to learn), it certainly does matter if not all are fully up to the job, as it would appear none currently are.

demosthanos 3y ago

If you look at the actual process for building these deep neural networks, it's actually far more emergent and bottom-up than you might think just reading the news. I took a deep learning class before things really took off [0], and what we were taught was that a lot of deep learning research was just trying different structures to see what worked. Even the people who were good at it didn't base their architectures on strong theories for how and why things would work, they just had developed an intuition through lots of trial and error. All the explanations for what the various structures in a deep learning model are doing are post hoc—they're observations made of the behavior of the systems after we build them. [0] Edit: Not just a random deep learning class either. The TA for that class created AI Dungeon that semester. The professor came in one day with a story of how they accidentally racked up tens of thousands of dollars in Google Cloud bills in 24 hours when that went viral.

dundarious 3y ago
Early architecture/structural engineering (figuring out arches, up to building cathedrals) was done by "just trying different structures to see what worked". Calling this "emergent" is not what is typically meant by the word -- we don't consider cathedrals to be "emergent structures". It's fair to say that many people imprecisely use the term, but then you are contributing to its drift into becoming (or perhaps always having been) a floating signifier.
- wolframhempel 3y ago
  But Cathedrals still had central planning, blueprints, ropes stuck in the ground to outline what should be built. Thermites on the other hand use a process called "Stigmergy" in which work allocation and building structures emerge organically without central coordination.
  - dundarious 3y ago
    They justified their point by saying the research was "emergent". My point is that almost all pre-modern research was "emergent" by their usage, and a large amount of contemporaneous research is "emergent" as well. That would be slightly annoying, but they did this to then justify points about how the resulting model is "emergent". The research process, and the resulting output, can and (my main point) almost always do, differ as to whether they are truly emergent.

FreddaGennaio 3y ago

It's insightful to observe the difference between the emergent behavior of natural systems and the architectured approach often used in AI development. The focus on neural networks, while beneficial for certain applications, might overlook the potential of emergent, decentralized systems that could offer solutions for complex, dynamic challenges. Hivekit's work is a step towards exploring these alternative approaches, emphasizing the importance of communication and interconnectedness in creating effective complex systems.

HarHarVeryFunny 3y ago

> when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected I wouldn't characterize it like that ... The brain has a specific learning architecture / dynamics that has been created via evolution under selection pressure to learn (i.e predict outcomes) better. As a product of evolution I wouldn't want to call it "architected" or a top-down design, but for the time being it's the only example we have of such a successful learning system so it would make sense to copy what evolution has done, which means copying how it works on all levels. A simpler example is convolutional neural nets for vision which were explicitly designed to simplistically mimic some of the behavior of our visual system with it's multi-layer (V1, V2, etc) architecture and local learning rules. Sure there's emergent behavior occurring when data is fed into such a system (both brain's visual system or CNN), such as the pattern detectors we see emerge at lowest level, but this emergent behavior is a result of the overall architecture.

tlarkworthy 3y ago

The retina, dopamine system, basal ganglia, cerebellum, entorhinal cortex all have very specialised coarse grained structures for their specialisations. The eye is part of the brain for instance. The details are emergent but there is most definitely strong top down architecture encoded in DNA

guyomes 3y ago

Actually the link between economy and neural network is quite strong historically. Friedrich Hayek is an economist who invented the word catallaxy to describe "the order brought about by the mutual adjustment of many individual economies in a market" [2]. Then he went on to work on a global theory of the brain, and was cited by Frank Rosenblatt in 1958 as an inspiration for the model of the perceptron [3]. [1]: https://en.wikipedia.org/wiki/Friedrich_Hayek#Economic_calculation_problem https://en.wikipedia.org/wiki/Friedrich_Hayek#Economic_calcu... [2]: https://en.wikipedia.org/wiki/Catallaxy https://en.wikipedia.org/wiki/Catallaxy [3]: https://citeseerx.ist.psu.edu/doc_view/pid/2688969848b75336891b1e58a2b6ee433343867c https://citeseerx.ist.psu.edu/doc_view/pid/2688969848b753368...