4 ms·

No, just means that someone needs to bother to make an AI for Ms. Pacman. Handling random ghost movements is only trivially confounding for AI learning. Low va

by haimez 10y ago

No, just means that someone needs to bother to make an AI for Ms. Pacman. Handling random ghost movements is only trivially confounding for AI learning. Low value, low yield.

antognini 10y ago
It's true that someone could pretty easily hand-craft an AI that plays Ms. Pac-Man pretty well. But that's not really the point. Because after all that work, you still only end up with a computer program that can play Ms. Pac-Man. It won't play Pong, it won't play Doom, it won't even be able to play regular Pac-Man! The goal is to build a general system that can learn how to play specific games on its own. So, there is potentially high value to developing a Ms. Pac-Man AI, but only if it's not really a Ms. Pac-Man AI, but actually a more general AI, for which playing Ms. Pac-Man is a specific instance. I agree with the parent comment that some sort of new approach is probably needed. The main issue with reinforcement learning is just that the algorithm gets too little feedback. This ends up discarding an enormous amount of data on each run (or, more accurately, learning from that data extremely inefficiently). There's a huge amount of data present, but the learning from that data has to be unsupervised. A child can do it easily --- they only have to play Ms. Pac-Man once or twice to get the hang of it. Computers currently cannot. Getting them to do that will definitely require something different, but no one is quite sure what.
- WhitneyLand 10y ago
  >someone could pretty easily hand-craft an AI that plays Ms. Pac-Man pretty well Not true. Many people have also tried handcrafting as well and only scored around 30,000 which is a low score.
- doikor 10y ago
  How much more complex is Ms. Pac-Man when compared to Atari 2600 games? Deep Minds DQN did master (as in better then any human can play them) a whole bunch (49) of 2600 games a while back https://deepmind.com/research/dqn/ https://deepmind.com/research/dqn/ edit: Checking wikipedia looks like it isn't playing all games it can play at superhuman levels > For most games (Space Invaders, Ms Pacman, Q*Bert for example), DeepMind plays below the current World Record https://en.wikipedia.org/wiki/DeepMind#Deep_reinforcement_learning https://en.wikipedia.org/wiki/DeepMind#Deep_reinforcement_le...
  - WhitneyLand 10y ago
    DM did terrible at Ms. Pac Man, let alone below the current world record is an understatement. DM scored less than 30,000. Human record is over 900,000.
WhitneyLand 10y ago
I think you are underestimating the difficulty. It's not like DM was the first to try. Multiple papers have been written on failed attempts. Dozens of other developers have tried. It's an open problem.
- haimez 10y ago
  It's a boring tautology. Arcade games require "clever" (read: not generalizable) solutions to optimize for speed in decision making. "Learning" AI requires the broadest feasible search space for any solution and successful designs require clusters of machines. Clusters of machines have network latency and coordination overhead. This is the same latency vs throughput design decision that every system must make, and its not impressive when a throughout oriented system struggles with being latency sensitive.
  - WhitneyLand 10y ago
    It may be boring because you don't realize it's not a tautology. You are wrong to separate video games from generalized AI. If you read the paper I posted above it explains part of this active area of research,

by haimez 10y ago

No, just means that someone needs to bother to make an AI for Ms. Pacman. Handling random ghost movements is only trivially confounding for AI learning. Low value, low yield.

antognini 10y ago

It's true that someone could pretty easily hand-craft an AI that plays Ms. Pac-Man pretty well. But that's not really the point. Because after all that work, you still only end up with a computer program that can play Ms. Pac-Man. It won't play Pong, it won't play Doom, it won't even be able to play regular Pac-Man! The goal is to build a general system that can learn how to play specific games on its own. So, there is potentially high value to developing a Ms. Pac-Man AI, but only if it's not really a Ms. Pac-Man AI, but actually a more general AI, for which playing Ms. Pac-Man is a specific instance. I agree with the parent comment that some sort of new approach is probably needed. The main issue with reinforcement learning is just that the algorithm gets too little feedback. This ends up discarding an enormous amount of data on each run (or, more accurately, learning from that data extremely inefficiently). There's a huge amount of data present, but the learning from that data has to be unsupervised. A child can do it easily --- they only have to play Ms. Pac-Man once or twice to get the hang of it. Computers currently cannot. Getting them to do that will definitely require something different, but no one is quite sure what.

WhitneyLand 10y ago
>someone could pretty easily hand-craft an AI that plays Ms. Pac-Man pretty well Not true. Many people have also tried handcrafting as well and only scored around 30,000 which is a low score.
doikor 10y ago
How much more complex is Ms. Pac-Man when compared to Atari 2600 games? Deep Minds DQN did master (as in better then any human can play them) a whole bunch (49) of 2600 games a while back https://deepmind.com/research/dqn/ https://deepmind.com/research/dqn/ edit: Checking wikipedia looks like it isn't playing all games it can play at superhuman levels > For most games (Space Invaders, Ms Pacman, Q*Bert for example), DeepMind plays below the current World Record https://en.wikipedia.org/wiki/DeepMind#Deep_reinforcement_learning https://en.wikipedia.org/wiki/DeepMind#Deep_reinforcement_le...
- WhitneyLand 10y ago
  DM did terrible at Ms. Pac Man, let alone below the current world record is an understatement. DM scored less than 30,000. Human record is over 900,000.

WhitneyLand 10y ago

I think you are underestimating the difficulty. It's not like DM was the first to try. Multiple papers have been written on failed attempts. Dozens of other developers have tried. It's an open problem.

haimez 10y ago
It's a boring tautology. Arcade games require "clever" (read: not generalizable) solutions to optimize for speed in decision making. "Learning" AI requires the broadest feasible search space for any solution and successful designs require clusters of machines. Clusters of machines have network latency and coordination overhead. This is the same latency vs throughput design decision that every system must make, and its not impressive when a throughout oriented system struggles with being latency sensitive.
- WhitneyLand 10y ago
  It may be boring because you don't realize it's not a tautology. You are wrong to separate video games from generalized AI. If you read the paper I posted above it explains part of this active area of research,