4 ms·

I'd say both characterizations are accurate. Neither the tree search nor the neural net could have accomplished this on their own. But the essential interface

• igravious 11y ago
  > The real world doesn't follow formal game rules. Really? Why not?
  • argonaut 11y ago
    Infinite state/belief/world space. Infinite action space. It's not so much that there aren't rules (there are - physics), it's that the complexity of the full set of rules is exponential or super-exponential.
    • ogrisel 11y ago
      You can do math with continuous and infinite dimensional spaces.
      • argonaut 11y ago
        And? This does not address my argument that the complexity is beyond-combinatorially explosive (infinite spaces). I'm not talking about the space of possible board states. I'm talking about merely the set of all possible actions. EDIT: clarified my language to address below reply.
        • eru 11y ago
          Go is combinatorially explosive, too.
          • deleted 11y ago
            [deleted]
        • ogrisel 11y ago
          ...and it's possible to train learning agents to sense and interact with a world described by high dimensional continuous vector spaces, for instance using conv nets (for sensing audio / video signals) and actor-critic to learn an continuous policy: http://arxiv.org/abs/1509.02971 http://arxiv.org/abs/1509.02971 The fact that the (reinforcement) learning problem is hard or not is not directly related to whether the observation and action spaces are discrete or continuous.
          • ogrisel 11y ago
            see also by the same team: http://arxiv.org/abs/1602.01783 http://arxiv.org/abs/1602.01783
          • argonaut 11y ago
            There is a near infinite number of such spaces.
            • ogrisel 11y ago
              I don't understand why would the "number of spaces" matters. What matters is can you design a learning algorithm that performs well in interesting spaces such as: - discrete spaces such as atari games and go, - continuous spaces such as driving a car, controlling a robot or bid on a ad exchange.
              • argonaut 11y ago
                A really really large number of distinct decisions that need to be made. A car only needs to control a small set of actions (wheels, engine, a couple others I'm missing). A game player only needs to choose from a small set of actions (e.g. place piece at position X, move left/right/up/down/jump).
                • ogrisel 11y ago
                  A human brain also has a limited number of muscles to control to interact with the world.
    • vlowther 11y ago
      Your assertion that the universe has infinite anything is a common mistake. A stupidly large number of states is not infinite.
• kqr 11y ago
  Sure, but when you say "tree search" people think of the traditional kind of expert system-based tree search. Traditional tree search methods prune trees by approximating subtrees according to some very specific rules decided on by human experts. This means in a tree search system, the computer cannot actually evaluate a position any better than the humans that designed it. The way it performs better is by running very many of these evaluations deep down in the tree. When you're talking about some other kind of evaluation function, such as Monte Carlo rollouts, you usually prefix that to the tree search (in the case of Monte Carlo rollouts, "Monte Carlo tree search" or MCTS) to indicate that besides the basic fundamental task common to almost all AIs (finding the optimal branches in a decision tree) it functions completely differently from the expert systems. So is the case with this program, which (in a first pass) approximates subtrees by a trained neural net, rather than Monte Carlo rollouts or an expert system. So using terminology that suggests classical expert system tree search is bound to cause confusion (as you noticed).