3 ms·

If we ever achieve AGI, my guess is that something akin to this approach is how it will probably be accomplished. I don’t think the human brain will really be a

If we ever achieve AGI, my guess is that something akin to this approach is how it will probably be accomplished. I don’t think the human brain will really be able to intuitively grok however AGI works, so I see an iterative process where we invent a program that tries to find a better program to accomplish some goal. Then we feed that program back into itself, and it recursively finds a better program finder. It seems like if we can just get past that critical threshold where the recursive process builds on itself rather than fizzles out due to resource constraints, then progress should accelerate quickly from there.

interstice 5y ago
I’m also thinking along these lines. With little to no knowledge of the domain (other than being human) a lot of life seems to come down to referencing which subsystem or set of subsystems to use in a given situation and training them over time. Like yes - we need the ‘make small talk with checkout operator’ skill in this situation
mkaic 5y ago
Hard agree with this. Humans are always going to insert human biases into the systems we design, so I think the only way we can really get past those biases is by going several layers of abstraction deep. I actually suspect that the first AGI will behave eerily organically when we take a look under the hood. Evolution is the ultimate optimization function, and it made organic brains work a certain way based on a cost function. I'll bet we'll see patterns in AGI that remind us of biological brains, but with stark, alien components mixed in and structures reorganized to be more efficient. All in all I'm just incredibly excited to see how this line of research plays out and desperately want to get involved in it myself. I was actually considering learning JAX just the other day specifically so I could look more into the topic, but I'm lazy so I never got around to it.
vagab0nd 5y ago
> and it recursively finds a better program finder This has been bugging me for a while. I can't articulate it well but I feel there's a hole in the logic. What exactly is the "program finder" finding? A better version of the program at finding... what exactly? You need a stop condition for the recursion.

interstice 5y ago

I’m also thinking along these lines. With little to no knowledge of the domain (other than being human) a lot of life seems to come down to referencing which subsystem or set of subsystems to use in a given situation and training them over time. Like yes - we need the ‘make small talk with checkout operator’ skill in this situation

mkaic 5y ago

Hard agree with this. Humans are always going to insert human biases into the systems we design, so I think the only way we can really get past those biases is by going several layers of abstraction deep. I actually suspect that the first AGI will behave eerily organically when we take a look under the hood. Evolution is the ultimate optimization function, and it made organic brains work a certain way based on a cost function. I'll bet we'll see patterns in AGI that remind us of biological brains, but with stark, alien components mixed in and structures reorganized to be more efficient. All in all I'm just incredibly excited to see how this line of research plays out and desperately want to get involved in it myself. I was actually considering learning JAX just the other day specifically so I could look more into the topic, but I'm lazy so I never got around to it.

vagab0nd 5y ago

> and it recursively finds a better program finder This has been bugging me for a while. I can't articulate it well but I feel there's a hole in the logic. What exactly is the "program finder" finding? A better version of the program at finding... what exactly? You need a stop condition for the recursion.