4 ms·
See also: the AI Plays Pokemon project that went megaviral a year or so ago, using CNNs and RL instead of LLMs: https://github.com/PWhiddy/PokemonRedExperiments
by minimaxir 2y ago
See also: the AI Plays Pokemon project that went megaviral a year or so ago, using CNNs and RL instead of LLMs: https://github.com/PWhiddy/PokemonRedExperiments https://github.com/PWhiddy/PokemonRedExperiments
> I believe that Claude Plays Pokemon isn't doing any of the memory parsing I spent a ton of time, they are just streaming the memory directly to Claude 3.7 and it is figuring it out
It is implied they are using structured Pokemon data from the LLM and saving it as a knowledge base. That is the only way they can get live Pokemon party data to display in the UI: https://www.twitch.tv/claudeplayspokemon https://www.twitch.tv/claudeplayspokemon
The AI Plays Pokemon project above does note some of the memory addresses where that data is contained, since it used that data to calculate the reward for the PPO.
- adenta 2y agoOn this page (https://excalidraw.com/#json=WrM9ViixPu2je5cVJZGCe,no_UoONhF6UxyMpTqltYkg https://excalidraw.com/#json=WrM9ViixPu2je5cVJZGCe,no_UoONhF...) linked from their twitch, it says: “ This info is all parsed directly from the RAM of the game, Claude Code is very good at this task”. I’m reading that as “we are pumping the RAM directly into the LLM”, but I could be mistaken.
- None4U 2y agoI suspect that means they wrote the memory parser using Claude (the Twitch description also mentions the LLM getting specific info)
- minimaxir 2y agoI agree that's ambigiously worded. For example, I'm not sure if Claude could identify "MT MOON B1F" from the RAM data alone since internally world map areas are only known by IDs, while AI Plays Pokemon did annotate the corresponding area with a human-readable name. https://github.com/PWhiddy/PokemonRedExperiments/blob/master/v2/map_data.json https://github.com/PWhiddy/PokemonRedExperiments/blob/master... Though this RAM data could be in Claude's training data.
- swyx 2y agohttps://www.latent.space/p/how-claude-plays-pokemon-was-made https://www.latent.space/p/how-claude-plays-pokemon-was-made more info here too