3 ms·

Basically, any problem with a solution fits into RL: reward of 1 if you are AGI and 0 otherwise. Go learn. This setting on its own is meaningless! The “how” of

Basically, any problem with a solution fits into RL: reward of 1 if you are AGI and 0 otherwise. Go learn. This setting on its own is meaningless! The “how” of the RL agent is not even 99% of the problem, it is all of it. Given our understanding of both DL and neuroscience, it is not even clear to me that we can say with confidence that Neural Networks are a sufficiently expressive architecture to cover an AGI. The human brain is a deep net, sort of, but there is also plenty going on in our brains that we don’t understand. It could be that the magic sprinkle is orthogonal to DL and we just don’t know about it yet.

svantana 5y ago
> The “how” of the RL agent is not even 99% of the problem, it is all of it I'm not sure that's true anymore - pretty much any objective devised is being solved by ML solutions within months (with some exceptions such as Chollet's ARC, maybe Winogrande). But those same models will perform poorly on other unseen tasks, because ML takes shortcuts if it can. We used to have unsolved tasks for decades, such as Go. It's now comparably hard (if not harder) to create a good objective measure of intelligence than to reach human parity on said measure.
- rich_sasha 5y ago
  Maybe I’m misinterpreting your comment, but what about self-driving cars? Hugely researched and funded area, not without success, but even the best self-driving system is so inferior to a human driver. I’m not dissing self-driving car research, just not sure we’re anywhere close to parity, and the problem is fairly well defined.
  - svantana 5y ago
    Right, I was talking about mathematically defined objectives and inputs (algorithms & data, in practice). With self-driving, arguably the main stumbling block is that the objective is not well-defined and the inputs are potentially wrong (faulty sensor data).
cynusx 5y ago
Thank you for stating what should be obvious. I think there are two currently unsolved problems 1/ We have no idea what the reward function looks like that leads to AGI 2/ Deep networks are artificially constricted for computational efficiency and always optimized to solve the problem at hand; Any solution that delivers AGI should rely imo on: 1/ reinforcement learning 2/ Happen with an unstructured reservoir of randomly connected neurons There was a research trend towards reservoir computing and recurrent neural networks but this was mostly abandoned because progress in deep learning was amazing. These techniques are akin to a 2D-plane in a 3D object, it's heavily simplified and circular references are prohibited. I have some good ideas on what the reward function should look like in a reservoir setting and happy to discuss them with any active independent researcher in the field.
candiodari 5y ago
I assure you 99% of the problem of any RL project is the simulator. Generally you can't let an RL algorithm control anything real from the start, so you have to implement a reasonably reliable simulator for whatever you want done. This is the big challenge in practice.
- ronsor 5y ago
  Correct me if I'm wrong, but wouldn't that mean the entire world would have to be simulated? Or at least some subset of society?
  - ggggtez 5y ago
    The human brain does have a simulator. It's well known. How do you know where to move your hand to catch a ball? Or what is happening when you blink? Your brain is constantly simulating a few milliseconds ahead.
    - candiodari 5y ago
      That may be true, but that's not to simulate. "Dreaming" is used to "simulate", and it's only a hack, in humans probably not even explicitly programmed. The hack is to have the simulator itself ... also be a learned system, and not to simulate the world, because you don't act on the real world, only a tiny part of it that you can measure and actually get into your brain (which is a simplified version of what you see, or a "latent variable"). There's no need to simulate anything that doesn't affect your reasoning. The information flows in reality with an intelligent actor is like this. World (say, a tree falls) -> input representation (e.g. eyes) -> simplified version ("latent" version) -> intelligent actor -> muscles -> affects world. Now what everybody thinks of as a simulator is something that simulates the whole thing. But if you insert one more link (output of reasoning agent at time T -> simplified representation at time T+1) you can then run a "simulation": random simplified version ("what if your car became a tree and fell ?") -> intelligent actor -> next input for latent representation ("then what happens ?") For safety reasons, it is probably prudent to disconnect the muscles in this state. You know, so you don't knock out your mother when you dream about boxing. And as you say, this "predict the future" network is probably useful by itself in dealing with the world. So you can catch tennis balls and the like. https://arxiv.org/abs/1803.10122 https://arxiv.org/abs/1803.10122 Or, if you like: https://www.youtube.com/watch?v=dPsXxLyqpfs https://www.youtube.com/watch?v=dPsXxLyqpfs
bobthechef 5y ago
Concepts can't be represented in matter. That's your secret sauce. Well, the beginning of the recipe, anyway. But you won't be able to make the dish.
ggggtez 5y ago
Problem: you don't understand it therefore you think RL isn't sufficient. There is no evidence that the thing you don't understand isn't based on RL too.

svantana 5y ago

> The “how” of the RL agent is not even 99% of the problem, it is all of it I'm not sure that's true anymore - pretty much any objective devised is being solved by ML solutions within months (with some exceptions such as Chollet's ARC, maybe Winogrande). But those same models will perform poorly on other unseen tasks, because ML takes shortcuts if it can. We used to have unsolved tasks for decades, such as Go. It's now comparably hard (if not harder) to create a good objective measure of intelligence than to reach human parity on said measure.

rich_sasha 5y ago
Maybe I’m misinterpreting your comment, but what about self-driving cars? Hugely researched and funded area, not without success, but even the best self-driving system is so inferior to a human driver. I’m not dissing self-driving car research, just not sure we’re anywhere close to parity, and the problem is fairly well defined.
- svantana 5y ago
  Right, I was talking about mathematically defined objectives and inputs (algorithms & data, in practice). With self-driving, arguably the main stumbling block is that the objective is not well-defined and the inputs are potentially wrong (faulty sensor data).

cynusx 5y ago

Thank you for stating what should be obvious. I think there are two currently unsolved problems 1/ We have no idea what the reward function looks like that leads to AGI 2/ Deep networks are artificially constricted for computational efficiency and always optimized to solve the problem at hand; Any solution that delivers AGI should rely imo on: 1/ reinforcement learning 2/ Happen with an unstructured reservoir of randomly connected neurons There was a research trend towards reservoir computing and recurrent neural networks but this was mostly abandoned because progress in deep learning was amazing. These techniques are akin to a 2D-plane in a 3D object, it's heavily simplified and circular references are prohibited. I have some good ideas on what the reward function should look like in a reservoir setting and happy to discuss them with any active independent researcher in the field.

candiodari 5y ago

I assure you 99% of the problem of any RL project is the simulator. Generally you can't let an RL algorithm control anything real from the start, so you have to implement a reasonably reliable simulator for whatever you want done. This is the big challenge in practice.

ronsor 5y ago
Correct me if I'm wrong, but wouldn't that mean the entire world would have to be simulated? Or at least some subset of society?
- ggggtez 5y ago
  The human brain does have a simulator. It's well known. How do you know where to move your hand to catch a ball? Or what is happening when you blink? Your brain is constantly simulating a few milliseconds ahead.
  - candiodari 5y ago
    That may be true, but that's not to simulate. "Dreaming" is used to "simulate", and it's only a hack, in humans probably not even explicitly programmed. The hack is to have the simulator itself ... also be a learned system, and not to simulate the world, because you don't act on the real world, only a tiny part of it that you can measure and actually get into your brain (which is a simplified version of what you see, or a "latent variable"). There's no need to simulate anything that doesn't affect your reasoning. The information flows in reality with an intelligent actor is like this. World (say, a tree falls) -> input representation (e.g. eyes) -> simplified version ("latent" version) -> intelligent actor -> muscles -> affects world. Now what everybody thinks of as a simulator is something that simulates the whole thing. But if you insert one more link (output of reasoning agent at time T -> simplified representation at time T+1) you can then run a "simulation": random simplified version ("what if your car became a tree and fell ?") -> intelligent actor -> next input for latent representation ("then what happens ?") For safety reasons, it is probably prudent to disconnect the muscles in this state. You know, so you don't knock out your mother when you dream about boxing. And as you say, this "predict the future" network is probably useful by itself in dealing with the world. So you can catch tennis balls and the like. https://arxiv.org/abs/1803.10122 https://arxiv.org/abs/1803.10122 Or, if you like: https://www.youtube.com/watch?v=dPsXxLyqpfs https://www.youtube.com/watch?v=dPsXxLyqpfs

bobthechef 5y ago

Concepts can't be represented in matter. That's your secret sauce. Well, the beginning of the recipe, anyway. But you won't be able to make the dish.

ggggtez 5y ago

Problem: you don't understand it therefore you think RL isn't sufficient. There is no evidence that the thing you don't understand isn't based on RL too.