5 ms·

From Scratch: AI Balancing Act in 50 Lines of Python

67 points by amasad 8y ago

X6S1x6Okd1st 8y ago
Initializing the policy to random and performing thousands of trials is just about the simplest implementation of "AI" you could possibly ask for. It's nice to see someone actually going for the MVP.
- mikeshi42 8y ago
  The mind blowing thing is that it actually works, and with rand(0,1), it'll get you results with only 10 guesses (though it's a bit cheating haha). Fun fact: a policy of [1,2,3,4] will solve the game. When I started off the article, I was thinking of doing some kind of evolutionary-based policy search but turns out none of that is even necessary. I hope to have a more technical follow up w/ deeper analysis and hopefully another one that goes into harder games where random guessing is okay, but there are more "sample efficient" methods.
  - X6S1x6Okd1st 8y ago
    Yeah I've been meaning to work my way up the complexity ladder on openai's gym. Seeing that it can be reasonably done by a random walk and the fact that it's only 4 dimensions kind of makes me want to do a uniform convering of the space and visualize the results.
    - mikeshi42 8y ago
      You should! I did some analysis myself a bit, have to dust it off to make it bit more presentable. There are definitely clear boundaries of "good" policies.
  - wholemoley 8y ago
    If you do decide to do an evolutionary-based policy, I highly recommend messing around with NEAT (https://github.com/CodeReclaimers/neat-python https://github.com/CodeReclaimers/neat-python).. I have successfully used it to play a number of SNES/Genesis games. I even made a tutorial series. https://www.youtube.com/watch?v=pClGmU1JEsM&list=PLTWFMbPFsvz3CeozHfeuJIXWAJMkPtAdS https://www.youtube.com/watch?v=pClGmU1JEsM&list=PLTWFMbPFsv... Apologies for the shameless plug.
skykooler 8y ago
Discussing "state": > In the game of pong, it might be the vertical position of each paddle and the x, y coordinate of the ball. The velocity vector of the ball is also a (very important!) part of the state.
- mikeshi42 8y ago
  Good point! Edited the post :) (Admittedly I've never built anything to play pong )
esaym 8y ago
Not sure why this has to be AI specific.. Terry Davis was doing this 15+ years ago: https://youtu.be/25hLohVZdME https://youtu.be/25hLohVZdME Seems to be more of an issue of knowing calculus and control system patterns.
- deleted 8y ago
  [deleted]
- X6S1x6Okd1st 8y ago
  It's not. It's a trivial problem to have something easy to do when starting out with reinforcement learning.

67 points by amasad 8y ago

Initializing the policy to random and performing thousands of trials is just about the simplest implementation of "AI" you could possibly ask for. It's nice to see someone actually going for the MVP.

mikeshi42 8y ago
The mind blowing thing is that it actually works, and with rand(0,1), it'll get you results with only 10 guesses (though it's a bit cheating haha). Fun fact: a policy of [1,2,3,4] will solve the game. When I started off the article, I was thinking of doing some kind of evolutionary-based policy search but turns out none of that is even necessary. I hope to have a more technical follow up w/ deeper analysis and hopefully another one that goes into harder games where random guessing is okay, but there are more "sample efficient" methods.
- X6S1x6Okd1st 8y ago
  Yeah I've been meaning to work my way up the complexity ladder on openai's gym. Seeing that it can be reasonably done by a random walk and the fact that it's only 4 dimensions kind of makes me want to do a uniform convering of the space and visualize the results.
  - mikeshi42 8y ago
    You should! I did some analysis myself a bit, have to dust it off to make it bit more presentable. There are definitely clear boundaries of "good" policies.
- wholemoley 8y ago
  If you do decide to do an evolutionary-based policy, I highly recommend messing around with NEAT (https://github.com/CodeReclaimers/neat-python https://github.com/CodeReclaimers/neat-python).. I have successfully used it to play a number of SNES/Genesis games. I even made a tutorial series. https://www.youtube.com/watch?v=pClGmU1JEsM&list=PLTWFMbPFsvz3CeozHfeuJIXWAJMkPtAdS https://www.youtube.com/watch?v=pClGmU1JEsM&list=PLTWFMbPFsv... Apologies for the shameless plug.

skykooler 8y ago

Discussing "state": > In the game of pong, it might be the vertical position of each paddle and the x, y coordinate of the ball. The velocity vector of the ball is also a (very important!) part of the state.

mikeshi42 8y ago
Good point! Edited the post :) (Admittedly I've never built anything to play pong )

esaym 8y ago

Not sure why this has to be AI specific.. Terry Davis was doing this 15+ years ago: https://youtu.be/25hLohVZdME https://youtu.be/25hLohVZdME Seems to be more of an issue of knowing calculus and control system patterns.

deleted 8y ago
[deleted]
X6S1x6Okd1st 8y ago
It's not. It's a trivial problem to have something easy to do when starting out with reinforcement learning.