4 ms·

The AI Plays Pokemon project only made it to Mt. Moon (where coincidentially ClaudePlaysPokemon is stuck now) with many months of iteration and many many hours

The AI Plays Pokemon project only made it to Mt. Moon (where coincidentially ClaudePlaysPokemon is stuck now) with many months of iteration and many many hours of compute. The reason Claude 3.7's performance is interesting is that the LLM approach defeated Lt. Surge, far past Mt. Moon. (I wonder how Claude solved the infamous puzzle in Surge's gym) https://www.anthropic.com/research/visible-extended-thinking https://www.anthropic.com/research/visible-extended-thinking

deadbabe 2y ago
Not talking about Reinforcement learning type AI, I’m talking about classically programmed AI with standard pathfinders, GOAP, behavior trees, etc…
- adenta 2y ago
  Got a link handy?
- Philpax 2y ago
  But how much effort do you have to put in to build an agent that can play a specific game? Can you retarget that agent easily? How well will your agent deal with circumstances that it wasn't designed for?
  - deadbabe 2y ago
    A lot less effort than training a massive LLM. Also, there’s no point in designing for use cases it will never encounter. A Pokémon rpg AI is never going to have to go play GTA.
    - Philpax 2y ago
      A LLM can be reused for other use cases. Your agent can't.
      - deadbabe 2y ago
        The reusability is overrated. For every problem that isn’t natural language processing, there exists a far better solution that runs faster and more optimally than an LLM, at the expense of having to actually program the damn thing (for which you can use an LLM to help you anyway). Who can fight harder and better in a Pokémon battle, a programmed AI or an LLM? The programmed AI, because it has tactics and analysis built in. Even better, the AI’s difficulty can be scaled trivially where as an LLM you can tell it to “go easy” but it doesn’t actually know what that means? There’s no point in wasting time with an LLM for such an application.
gyomu 2y ago
The fact that these models can only play up to a certain point seems like an interesting indication as to the inherent limitation of their capabilities. After all, the game does not introduce any significant new mechanics beyond the first couple areas - any human player who has the reading/reasoning ability to make it to Mt Moon/Lt Surge would be able to complete the rest of the game. So why are these models getting stuck at arbitrary points in the game?
- minimaxir 2y ago
  There's one major mechanic that opens up shortly after Lt. Surge: nonlinearity. Once you get to Lavender Town, there are several options to go to, and I suspect that will be difficult for an AI to handle over a limited context window. And if the AI decides to attempt Seafoam Islands, all bets are off.

deadbabe 2y ago

Not talking about Reinforcement learning type AI, I’m talking about classically programmed AI with standard pathfinders, GOAP, behavior trees, etc…

adenta 2y ago
Got a link handy?
Philpax 2y ago
But how much effort do you have to put in to build an agent that can play a specific game? Can you retarget that agent easily? How well will your agent deal with circumstances that it wasn't designed for?
- deadbabe 2y ago
  A lot less effort than training a massive LLM. Also, there’s no point in designing for use cases it will never encounter. A Pokémon rpg AI is never going to have to go play GTA.
  - Philpax 2y ago
    A LLM can be reused for other use cases. Your agent can't.
    - deadbabe 2y ago
      The reusability is overrated. For every problem that isn’t natural language processing, there exists a far better solution that runs faster and more optimally than an LLM, at the expense of having to actually program the damn thing (for which you can use an LLM to help you anyway). Who can fight harder and better in a Pokémon battle, a programmed AI or an LLM? The programmed AI, because it has tactics and analysis built in. Even better, the AI’s difficulty can be scaled trivially where as an LLM you can tell it to “go easy” but it doesn’t actually know what that means? There’s no point in wasting time with an LLM for such an application.

gyomu 2y ago

The fact that these models can only play up to a certain point seems like an interesting indication as to the inherent limitation of their capabilities. After all, the game does not introduce any significant new mechanics beyond the first couple areas - any human player who has the reading/reasoning ability to make it to Mt Moon/Lt Surge would be able to complete the rest of the game. So why are these models getting stuck at arbitrary points in the game?

minimaxir 2y ago
There's one major mechanic that opens up shortly after Lt. Surge: nonlinearity. Once you get to Lavender Town, there are several options to go to, and I suspect that will be difficult for an AI to handle over a limited context window. And if the AI decides to attempt Seafoam Islands, all bets are off.