5 ms·

MarI/O – Machine Learning for Video Games

152 points by qtrain 11y ago

synthmeat 11y ago
I have reasonable suspicion that it wouldn't be nearly as easy to train without "spinning" move which basically gives invulnerability against all enemies sans their bullets, iirc.
- babuskov 11y ago
  As far as I can tell, it would only require longer time (more iterations) for the network to learn, but it would still work. The main problem I see is that the algorithm is learning from the same level (or set of levels). It would be interesting to see what happens if you give it some completely new level to play.
- nip 11y ago
  I see this videoclip as a great introduction to neural networks and genetic algorithms (something I didn't know existed in ML) rather than a simple "computer completes videogame level". You're probably right nonetheless (I can't remember the mechanics of this mario game) :)
  - jeanlucas 11y ago
    Actually for Genetic Algorithms and/or Evolutionary computation it is not. :P
- berberous 11y ago
  Wait, does it really give you invulnerability? In the video Mario never runs into a single enemy while spin jumping.
  - xahrepap 11y ago
    Not exactly. Some enemies that would kill you with a normal jump when you land on them, Mario will bounce off of with a spin jump.
- M4v3R 11y ago
  A similar work was done for Super Mario on NES (along with other NES games): https://www.youtube.com/watch?v=xOCurBYI_gY https://www.youtube.com/watch?v=xOCurBYI_gY
aaggarwal 11y ago
Thank you for sharing. The video explanation is really easy to follow. For interested people, here is an interview with K. Stanley, the author of NEAT: http://aigamedev.com/open/interviews/galactic-arms-race/ http://aigamedev.com/open/interviews/galactic-arms-race/ I was wondering if this learning technique was applied to any other games. I came across this: 1. https://www.cs.utexas.edu/~mhauskn/papers/atari.pdf https://www.cs.utexas.edu/~mhauskn/papers/atari.pdf 2. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.196&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93....
babuskov 11y ago
TLDR: a neural network playing Super Mario. A very cool demonstration showing how the network evolves.
dvirsky 11y ago
This is a really cool and intuitive introduction to this sort of algorithms, and the example and visualization make it very accessible. Great work, I'm sure it will inspire some kids to get into ML and neural networks.
qtheninja 11y ago
Awesomeness. How you explain machine learning principles/NEAT really shows a firm grasp. The fact it's done in lua...makes it even more eccentrically cool.
140am 11y ago
Veery nice visual introduction to the basics of neural networks
deepnet 11y ago
Bizhawk is a SNES emulator for Game Casters. Seth, the video author is a Super Mario World Speedrunner and Twitchcaster. http://www.twitch.tv/sethbling http://www.twitch.tv/sethbling Bizhawk has Lua Scripting built in : http://tasvideos.org/Bizhawk/Features.html http://tasvideos.org/Bizhawk/Features.html Seth implemented Neuro-Evolution in Lua and http://pastebin.com/ZZmSNaHX http://pastebin.com/ZZmSNaHX now machine learning casting is a thing...
scottfr 11y ago
This is very cool, but it isn't really "playing" the game. Instead it's "memorizing" how to complete a specific level through trial and error. If you took this neural network and tried it on another Mario level, it would fail miserably.
- comex 11y ago
  Well, for some definition of fail miserably: > Yeah, I think it wouldn't generalize super well, but I did try it on another easy level and it was able to get halfway through it. - the author, at http://www.reddit.com/r/videos/comments/39qel5/top_super_mario_speedrunner_teaches_computer_to/cs5l8k0 http://www.reddit.com/r/videos/comments/39qel5/top_super_mar...
- deepnet 11y ago
  The author has said that a net trained on one level can get about halfway through another level. http://www.reddit.com/r/videos/comments/39qel5/top_super_mario_speedrunner_teaches_computer_to/cs5l8k0 http://www.reddit.com/r/videos/comments/39qel5/top_super_mar... Super Mario World is very deterministic. The Author is streaming various learning tasks on his twitch. MarI/O has been trained on 3 levels so far, for 24 hours each.
JonnieCache 11y ago
Here's the author taking the world record (now superseded) at glitching mario world into the credits to "win" the game in 3:07: https://www.youtube.com/watch?v=HxFh1CJOrTU https://www.youtube.com/watch?v=HxFh1CJOrTU This is on a real console, not an emulator. They overflow the snes into the controller registers to achieve arbitrary code execution by hand. They do this by placing koopa shells at pixel-perfect locations to spell out ASM, overflow to the controller registers, and press a specific combination of buttons to jump to the sprite table and their literal shellcode. At the start of this year there was a crazy few weeks where different speedrunners were competing to execute the "credits warp" in the quickest and most efficient way. It started at around 6 minutes, now it's down to around 2 I think, most of which is the game's intro. Fun stuff. SethBling was the first person to ever execute it on a real console. Here's a video explaining it, from a different speedrunner: https://www.youtube.com/watch?v=vAHXK2wut_I https://www.youtube.com/watch?v=vAHXK2wut_I
- cheald 11y ago
  "literal shellcode" made me chortle. What a delightfully wonderful hack.
uxp100 11y ago
I apologize, this is pedantic, but I think people often overstate the relationship between neural networks and the brain. I'm far from an expert, but saying neural networks are based on the brain is a stretch. They're inspired by a theory of how the brain works that is not wholly in line with what most scientists believe. It's so tempting to say, this is based on the brain, when really it's more like, this was inspired by one psychologists model of the brain.

152 points by qtrain 11y ago

I have reasonable suspicion that it wouldn't be nearly as easy to train without "spinning" move which basically gives invulnerability against all enemies sans their bullets, iirc.

babuskov 11y ago
As far as I can tell, it would only require longer time (more iterations) for the network to learn, but it would still work. The main problem I see is that the algorithm is learning from the same level (or set of levels). It would be interesting to see what happens if you give it some completely new level to play.
nip 11y ago
I see this videoclip as a great introduction to neural networks and genetic algorithms (something I didn't know existed in ML) rather than a simple "computer completes videogame level". You're probably right nonetheless (I can't remember the mechanics of this mario game) :)
- jeanlucas 11y ago
  Actually for Genetic Algorithms and/or Evolutionary computation it is not. :P
berberous 11y ago
Wait, does it really give you invulnerability? In the video Mario never runs into a single enemy while spin jumping.
- xahrepap 11y ago
  Not exactly. Some enemies that would kill you with a normal jump when you land on them, Mario will bounce off of with a spin jump.
M4v3R 11y ago
A similar work was done for Super Mario on NES (along with other NES games): https://www.youtube.com/watch?v=xOCurBYI_gY https://www.youtube.com/watch?v=xOCurBYI_gY

aaggarwal 11y ago

Thank you for sharing. The video explanation is really easy to follow. For interested people, here is an interview with K. Stanley, the author of NEAT: http://aigamedev.com/open/interviews/galactic-arms-race/ http://aigamedev.com/open/interviews/galactic-arms-race/ I was wondering if this learning technique was applied to any other games. I came across this: 1. https://www.cs.utexas.edu/~mhauskn/papers/atari.pdf https://www.cs.utexas.edu/~mhauskn/papers/atari.pdf 2. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.196&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93....

babuskov 11y ago

TLDR: a neural network playing Super Mario. A very cool demonstration showing how the network evolves.

dvirsky 11y ago

This is a really cool and intuitive introduction to this sort of algorithms, and the example and visualization make it very accessible. Great work, I'm sure it will inspire some kids to get into ML and neural networks.

qtheninja 11y ago

Awesomeness. How you explain machine learning principles/NEAT really shows a firm grasp. The fact it's done in lua...makes it even more eccentrically cool.

140am 11y ago

Veery nice visual introduction to the basics of neural networks

deepnet 11y ago

Bizhawk is a SNES emulator for Game Casters. Seth, the video author is a Super Mario World Speedrunner and Twitchcaster. http://www.twitch.tv/sethbling http://www.twitch.tv/sethbling Bizhawk has Lua Scripting built in : http://tasvideos.org/Bizhawk/Features.html http://tasvideos.org/Bizhawk/Features.html Seth implemented Neuro-Evolution in Lua and http://pastebin.com/ZZmSNaHX http://pastebin.com/ZZmSNaHX now machine learning casting is a thing...

scottfr 11y ago

This is very cool, but it isn't really "playing" the game. Instead it's "memorizing" how to complete a specific level through trial and error. If you took this neural network and tried it on another Mario level, it would fail miserably.

comex 11y ago
Well, for some definition of fail miserably: > Yeah, I think it wouldn't generalize super well, but I did try it on another easy level and it was able to get halfway through it. - the author, at http://www.reddit.com/r/videos/comments/39qel5/top_super_mario_speedrunner_teaches_computer_to/cs5l8k0 http://www.reddit.com/r/videos/comments/39qel5/top_super_mar...
deepnet 11y ago
The author has said that a net trained on one level can get about halfway through another level. http://www.reddit.com/r/videos/comments/39qel5/top_super_mario_speedrunner_teaches_computer_to/cs5l8k0 http://www.reddit.com/r/videos/comments/39qel5/top_super_mar... Super Mario World is very deterministic. The Author is streaming various learning tasks on his twitch. MarI/O has been trained on 3 levels so far, for 24 hours each.

JonnieCache 11y ago

Here's the author taking the world record (now superseded) at glitching mario world into the credits to "win" the game in 3:07: https://www.youtube.com/watch?v=HxFh1CJOrTU https://www.youtube.com/watch?v=HxFh1CJOrTU This is on a real console, not an emulator. They overflow the snes into the controller registers to achieve arbitrary code execution by hand. They do this by placing koopa shells at pixel-perfect locations to spell out ASM, overflow to the controller registers, and press a specific combination of buttons to jump to the sprite table and their literal shellcode. At the start of this year there was a crazy few weeks where different speedrunners were competing to execute the "credits warp" in the quickest and most efficient way. It started at around 6 minutes, now it's down to around 2 I think, most of which is the game's intro. Fun stuff. SethBling was the first person to ever execute it on a real console. Here's a video explaining it, from a different speedrunner: https://www.youtube.com/watch?v=vAHXK2wut_I https://www.youtube.com/watch?v=vAHXK2wut_I

cheald 11y ago
"literal shellcode" made me chortle. What a delightfully wonderful hack.

uxp100 11y ago

I apologize, this is pedantic, but I think people often overstate the relationship between neural networks and the brain. I'm far from an expert, but saying neural networks are based on the brain is a stretch. They're inspired by a theory of how the brain works that is not wholly in line with what most scientists believe. It's so tempting to say, this is based on the brain, when really it's more like, this was inspired by one psychologists model of the brain.