5 ms·

I'm becoming convinced that these algorithms are huge steps towards AGI simply because AGI might end up being a collection of many of these domain-specific netw

I'm becoming convinced that these algorithms are huge steps towards AGI simply because AGI might end up being a collection of many of these domain-specific networks with a network sitting above them who's only role is to interrogate the sub networks for solutions to the problem at hand, and discriminate which solution(s) are most worth trying, simulating those and then picking one out and executing it in the real world. That seems to me to be pretty close to what we as humans do.

WitCanStain 4y ago
Cognitive scientists have been exploring this idea for a while, e.g. the Modularity of Mind Hypothesis. It would be very interesting to see if learnings from this field can be further applied to AI development.
ausbah 4y ago
the problem you run into then is how does it adapt to new domains? sure large models display some degree of generalization, but new tasks will certainly ask things the network couldn't get
- TrevorJ 4y ago
  I think that's valid, however, I think the modular approach would allow for a lot more cross domain flexibility because I suspect that a large subset of disparate tasks do actually decompose themselves into discrete subtasks that can be useful across domains. Adding to that, if you can compose multiple sub networks together then you've really got something. You can build a lot of different buildings from bricks without needing to invent new kinds of brick basically. For instance, think about the large number of domains that robust computer vision would be useful in. Then think about the fact that if the computer understands the 3D space around it, it can hand that model off to a network that does predictive physics simulation. Now you've got something that would be useful across a extremely wide range of domains.
  - naugtur 4y ago
    The modular approach doesn't need to be enforced by strict boundaries. One network that starts with various components but can be remodeled and even merged together as it goes could adapt. What we're discussing here is merely a starting state of something that'd have to rebuild itself. Or maybe evolving modalities within one network is easier than putting it together.
p1necone 4y ago
You just need a neural network that knows how to interrogate other neural networks, and a neural network that knows how to find new datasets and train new neural networks and you potentially have something very scary right? Maybe a third neural network that knows how to purchase cloud compute?
- TrevorJ 4y ago
  >Maybe a third neural network that knows how to purchase cloud compute? Amazon has entered the chat
  - RootKitBeerCat 4y ago
    Private angel investors watch from above all of the “clouds”

WitCanStain 4y ago

Cognitive scientists have been exploring this idea for a while, e.g. the Modularity of Mind Hypothesis. It would be very interesting to see if learnings from this field can be further applied to AI development.

ausbah 4y ago

the problem you run into then is how does it adapt to new domains? sure large models display some degree of generalization, but new tasks will certainly ask things the network couldn't get

TrevorJ 4y ago
I think that's valid, however, I think the modular approach would allow for a lot more cross domain flexibility because I suspect that a large subset of disparate tasks do actually decompose themselves into discrete subtasks that can be useful across domains. Adding to that, if you can compose multiple sub networks together then you've really got something. You can build a lot of different buildings from bricks without needing to invent new kinds of brick basically. For instance, think about the large number of domains that robust computer vision would be useful in. Then think about the fact that if the computer understands the 3D space around it, it can hand that model off to a network that does predictive physics simulation. Now you've got something that would be useful across a extremely wide range of domains.
- naugtur 4y ago
  The modular approach doesn't need to be enforced by strict boundaries. One network that starts with various components but can be remodeled and even merged together as it goes could adapt. What we're discussing here is merely a starting state of something that'd have to rebuild itself. Or maybe evolving modalities within one network is easier than putting it together.

p1necone 4y ago

You just need a neural network that knows how to interrogate other neural networks, and a neural network that knows how to find new datasets and train new neural networks and you potentially have something very scary right? Maybe a third neural network that knows how to purchase cloud compute?

TrevorJ 4y ago
>Maybe a third neural network that knows how to purchase cloud compute? Amazon has entered the chat
- RootKitBeerCat 4y ago
  Private angel investors watch from above all of the “clouds”