3 ms·

There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. It's easiest to train on IID data, and also common to train

There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. It's easiest to train on IID data, and also common to train in specific RL environments, because those paper over some issues with catastrophic forgetting and overfitting, and it's also preferable to do all the compute-intensive updates during training for efficiency. However, lots of people agree that online learning is fundamental to cracking AGI, and it's neither neglected nor unsuccessful.

username90 6y ago
> There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. For some definition of plenty. I'd argue there isn't since almost all resources poured into AI are spent getting better at optimizing immutable functions. I don't see progress in getting better at optimizing immutable functions as progress towards general intelligence, hence I don't see almost anything people do in AI today as progress towards general intelligence. Do you understand now? I don't say nobody works on the things I think is important, just that it is mostly neglected compared to the enormous amount of resources spent on dumb models.
- Veedrac 6y ago
  Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I don't really agree with your stance on it, but I was not objecting to the claim that people should focus more on online learning. I was objecting to the claims “super computers can't even compete with the intelligence of an ant”, which is false, since the impressive behaviours an ant displays are not learned, and the learned aspects aren't competitive with what computers can do, and that “[the] way we do AI can never achieve that, since it always relies on human intelligence to decide all of those things”, which is also false, because online learning research exists and works, if imperfectly.
  - username90 6y ago
    > Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I disagree with this assertion, treating online learning like offline learning means you aren't making any progress. The main difference is that online learning lets you interact with the problem and seek more information, offline learning doesn't. Offline learning doesn't help you at all solving that problem.
    - Veedrac 6y ago
      Exploration is a major part of RL already, see AlphaGo for example.

username90 6y ago

> There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. For some definition of plenty. I'd argue there isn't since almost all resources poured into AI are spent getting better at optimizing immutable functions. I don't see progress in getting better at optimizing immutable functions as progress towards general intelligence, hence I don't see almost anything people do in AI today as progress towards general intelligence. Do you understand now? I don't say nobody works on the things I think is important, just that it is mostly neglected compared to the enormous amount of resources spent on dumb models.

Veedrac 6y ago
Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I don't really agree with your stance on it, but I was not objecting to the claim that people should focus more on online learning. I was objecting to the claims “super computers can't even compete with the intelligence of an ant”, which is false, since the impressive behaviours an ant displays are not learned, and the learned aspects aren't competitive with what computers can do, and that “[the] way we do AI can never achieve that, since it always relies on human intelligence to decide all of those things”, which is also false, because online learning research exists and works, if imperfectly.
- username90 6y ago
  > Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I disagree with this assertion, treating online learning like offline learning means you aren't making any progress. The main difference is that online learning lets you interact with the problem and seek more information, offline learning doesn't. Offline learning doesn't help you at all solving that problem.
  - Veedrac 6y ago
    Exploration is a major part of RL already, see AlphaGo for example.