3 ms·

I'm curious why author decided to completely disregard reinforcement learning. "Note that neural nets are neither. There has been a relatively small amount of

I'm curious why author decided to completely disregard reinforcement learning. "Note that neural nets are neither. There has been a relatively small amount of non-mainstream work of getting neural nets to control very simple robots, mostly in simulation only." This is just plain false. Playing Arcade Games involves controlling an agent and there is a lot of widely published, very mainstream work from DeepMind, OpenAI and many other less known research groups. The big part of hype is about using deep neural networks in tandem with reinforcement learning. It's a very reactive system. In complex environment a work from Nvidia is notable "End to End Learning for Self-Driving Cars": https://arxiv.org/abs/1604.07316 https://arxiv.org/abs/1604.07316 Also, Alpha Zero is very clever combination of reinforcement learning, neural networks and symbolic reasoning (tree search). Alpha Zero is not only reactive, but also deliberative. Interestingly Alpha Zero, in very limited domain, managed to surpass not only unaided humans, but many years of programmers effort based on purely symbolic methods by beating Stockfish in Chess. The author is no doubt a world class expert in the Good Old AI, but to my surprise he seems to be out of loops with more recent advancements. Or am I missing something / misunderstood something?

AstralStorm 8y ago
I wouldn't take that "Stockfish paper" announcement at face value. As far as we could tell, Stockfish was used in a situation it is not optimized for (very many cores) and wrong settings. What we can say is that Alpha Zero is comparable after throwing a lot of specialised compute power at it. Most importantly, neither system works at a symbolic level or does anything even close to thinking. They handle statistics and see searches extremely well, sometimes tactics, but most of it is still hardcoded - they are fed a high level symbolic representation of the game. When Alpha Zero can learn playing chess by experimenting with board physics and observing a few videos or at least reading a book, then we can say it can actually produce some form of symbolic thought. (OpenAI is closer but nowhere near close - it falls back on memory of play as a full state model.)
- Isinlor 8y ago
  There is ongoing open source, community based effort to replicate Alpha Zero Chess. The effort is lead by Gary Linscott. He is also one of the main developers of Stockfish (I believe the framework for testing Stockfish). https://github.com/LeelaChessZero/lczero https://github.com/LeelaChessZero/lczero Here is the current state of the effort: https://docs.google.com/spreadsheets/d/18UWR4FVhPi0vNwwPreu_avd9ycujGQ5ayR2LzJOWP4s/edit#gid=1045682900 https://docs.google.com/spreadsheets/d/18UWR4FVhPi0vNwwPreu_... Estimated date of surpassing Stockfish with settings and hardware reported in the spreadsheet is the end of this year. @edit I have missed your statements about symbolic reasoning. Yes, I'm referring to the same type of symbolic reasoning as the author of the article. I.e. system that manipulates high-level symbols without any grounded understanding of them in the surrounding us reality. Deliberating whether it is thinking or not is pointless. It's effective and that matters.
YeGoblynQueenne 8y ago
>> I'm curious why author decided to completely disregard reinforcement learning. Most probably because the vast majority of its applications is in game-playing and not human-level AI (as in general intelligence, capable of tackling free-form tasks in the real world), which is the subject of the article. To substantiate this intuition further: a) Agents playing arcade games, Go, chess, etc, are still constrained in "blocks worlds", even if they can beat humans in those worlds. b) To my knowledge, self-driving car systems actually deployed are not trained using reinforcement learning, or even deep learning (for anything other than vision) but instead make use of the SLAM approach discussed in the article. I might be wrong on this and would welcome a correction.

AstralStorm 8y ago

I wouldn't take that "Stockfish paper" announcement at face value. As far as we could tell, Stockfish was used in a situation it is not optimized for (very many cores) and wrong settings. What we can say is that Alpha Zero is comparable after throwing a lot of specialised compute power at it. Most importantly, neither system works at a symbolic level or does anything even close to thinking. They handle statistics and see searches extremely well, sometimes tactics, but most of it is still hardcoded - they are fed a high level symbolic representation of the game. When Alpha Zero can learn playing chess by experimenting with board physics and observing a few videos or at least reading a book, then we can say it can actually produce some form of symbolic thought. (OpenAI is closer but nowhere near close - it falls back on memory of play as a full state model.)

Isinlor 8y ago
There is ongoing open source, community based effort to replicate Alpha Zero Chess. The effort is lead by Gary Linscott. He is also one of the main developers of Stockfish (I believe the framework for testing Stockfish). https://github.com/LeelaChessZero/lczero https://github.com/LeelaChessZero/lczero Here is the current state of the effort: https://docs.google.com/spreadsheets/d/18UWR4FVhPi0vNwwPreu_avd9ycujGQ5ayR2LzJOWP4s/edit#gid=1045682900 https://docs.google.com/spreadsheets/d/18UWR4FVhPi0vNwwPreu_... Estimated date of surpassing Stockfish with settings and hardware reported in the spreadsheet is the end of this year. @edit I have missed your statements about symbolic reasoning. Yes, I'm referring to the same type of symbolic reasoning as the author of the article. I.e. system that manipulates high-level symbols without any grounded understanding of them in the surrounding us reality. Deliberating whether it is thinking or not is pointless. It's effective and that matters.

YeGoblynQueenne 8y ago

>> I'm curious why author decided to completely disregard reinforcement learning. Most probably because the vast majority of its applications is in game-playing and not human-level AI (as in general intelligence, capable of tackling free-form tasks in the real world), which is the subject of the article. To substantiate this intuition further: a) Agents playing arcade games, Go, chess, etc, are still constrained in "blocks worlds", even if they can beat humans in those worlds. b) To my knowledge, self-driving car systems actually deployed are not trained using reinforcement learning, or even deep learning (for anything other than vision) but instead make use of the SLAM approach discussed in the article. I might be wrong on this and would welcome a correction.