5 ms·

The AI of the computer player is an RNN, which is a type of neural network. The objective of the game, what the controls do, and strategies were never hard cod

The AI of the computer player is an RNN, which is a type of neural network. The objective of the game, what the controls do, and strategies were never hard coded into the AI, it learned them through trial and error, the way a human would. It's a lightweight implementation too, done in JS. One neat feature is that you can see the real-time activations of the neurons which are connected to the control nodes in the background.

seanalltogether 11y ago
Is the "learned" behavior stored in the javascript somewhere, or is it coming from a socket in real time? Is it just a giant data array for handling possible decisions?
- cstrahan 11y ago
  My understanding is that the weights[1] were produced from a series of trials, and then persisted in the JS. 1: http://en.wikipedia.org/wiki/Synaptic_weight http://en.wikipedia.org/wiki/Synaptic_weight
  - ColinDabritz 11y ago
    It would be neat if you could have it know nothing, and learn from you over a series of games.
- deepnet 11y ago
  Line 60 of slimevolley_pro.js contains the "learned" behaviour in 140 floats: "gene":{"0":7.5719,"1":4.4285,"2":2.2716,"3":-0.3598,"4":-7.8189,"5":-2.5422,"6":-3.2034,"7":0.3935,"8":-6.7593,"9":-8.0551,"10":1.3679,"11":2.1859,"12":1.2202,"13":-0.49,"14":-0.0316,"15":0.5221,"16":0.7026,"17":0.4179,"18":-2.1689,"19":1.646,"20":-13.3639,"21":1.5151,"22":1.1175,"23":-5.3561,"24":5.0442,"25":0.8451,"26":0.3987,"27":-2.6158,"28":0.4318,"29":-0.7361,"30":0.5715,"31":-2.9501,"32":-3.7811,"33":-5.8994,"34":6.4167,"35":2.5014,"36":7.338,"37":-2.9887,"38":2.4586,"39":13.4191,"40":2.7395,"41":-3.9708,"42":1.6548,"43":-2.7554,"44":-1.5345,"45":-6.4708,"46":-4.4454,"47":-0.6224,"48":-1.0988,"49":4.4501,"50":9.2426,"51":-0.7392,"52":0.4452,"53":1.8828,"54":-2.6277,"55":-10.851,"56":-3.2353,"57":-4.4653,"58":-3.1153,"59":-1.3707,"60":7.318,"61":16.0902,"62":1.4686,"63":7.0391,"64":1.7765,"65":-4.9573,"66":-1.0578,"67":1.3668,"68":-1.4029,"69":-1.155,"70":2.6697,"71":-8.8877,"72":1.1958,"73":-3.2839,"74":-5.4425,"75":1.6809,"76":7.6812,"77":-2.4732,"78":1.738,"79":0.3781,"80":0.8718,"81":2.5886,"82":1.6911,"83":1.2953,"84":-5.5961,"85":2.174,"86":-3.5098,"87":-5.4715,"88":-9.0052,"89":-4.6038,"90":-6.7447,"91":-2.5528,"92":0.4391,"93":-4.9278,"94":-3.6695,"95":-4.8673,"96":-1.6035,"97":1.5011,"98":-5.6124,"99":4.9747,"100":1.8998,"101":3.0359,"102":6.2983,"103":-2.703,"104":1.5025,"105":6.1841,"106":-0.9357,"107":-4.8568,"108":-2.1888,"109":-4.1143,"110":-3.9874,"111":-0.0459,"112":4.7134,"113":2.8952,"114":-9.3627,"115":-4.685,"116":0.3601,"117":-1.3699,"118":9.7294,"119":11.5596,"120":0.1918,"121":3.0783,"122":-6.6828,"123":-5.4398,"124":-5.088,"125":3.6948,"126":0.0329,"127":-0.1362,"128":-0.1188,"129":-0.7579,"130":0.3278,"131":-0.977,"132":-0.9377,"133":2.2935,"134":-2.0353,"135":-1.7786,"136":5.4567,"137":-3.6368,"138":3.4996,"139":-0.0685} These are the weights on the connections between the inputs, neurons and outputs. Everything you need to know about how to win Slime Volleyball in 140 numbers !
  - _lce0 11y ago
    > Everything you need to know about how to win Slime Volleyball in 140 numbers ! I know kung fu!
    - hardmaru 11y ago
      haha, I like that. Yeah, there are a total of 140 weights/biases in the neural network. This stuff is not bleeding edge - just taking techniques popular in academia and industry many years ago to make a fun game mainly for my own educational purpose. Basically recently I saw a lot of work on AI's playing ATARI games and wanted to learn reinforcement learning techniques, and the best way to learn is to build a game! I put up a description of this sketch here: http://blog.otoro.net/2015/03/28/neural-slime-volleyball/ http://blog.otoro.net/2015/03/28/neural-slime-volleyball/ Basically, the agent's brain consists of 7 neurons (compared to 250k for an ant), which are connected to all the inputs. The inputs are the game states, like locations of agents and ball, and velocities. Each neuron is also connected to each other. The first 3 neurons control the neuron's output (it's motor function), like whether it hits the up, forward, or back button - the same function that a human game player can do. The other 4 neurons are hidden neurons responsible for any deeper computation or thought, and are fed back into the input with a time lag. I also graphed a slice of the neural network in real time on the screen just for effect. More for design effect if anything.
  - 0xdeadbeefbabe 11y ago
    How many numbers do you need? It isn't just a matter of trial and error is it?
    - deepnet 11y ago
      The number of weights depends on the network architecture. In this case fully connected , 1 weight per connection . 16 inputs to 7 neurons + 7 neurons to 3 outputs + 7 biases 16x7 + 7x3 + 7 = 140 http://blog.otoro.net/wp-content/uploads/sites/2/2015/03/slime_rnn.png http://blog.otoro.net/wp-content/uploads/sites/2/2015/03/sli... The number of neurons is a decidable.
  - 11y ago

seanalltogether 11y ago

Is the "learned" behavior stored in the javascript somewhere, or is it coming from a socket in real time? Is it just a giant data array for handling possible decisions?

cstrahan 11y ago
My understanding is that the weights[1] were produced from a series of trials, and then persisted in the JS. 1: http://en.wikipedia.org/wiki/Synaptic_weight http://en.wikipedia.org/wiki/Synaptic_weight
- ColinDabritz 11y ago
  It would be neat if you could have it know nothing, and learn from you over a series of games.
deepnet 11y ago
Line 60 of slimevolley_pro.js contains the "learned" behaviour in 140 floats: "gene":{"0":7.5719,"1":4.4285,"2":2.2716,"3":-0.3598,"4":-7.8189,"5":-2.5422,"6":-3.2034,"7":0.3935,"8":-6.7593,"9":-8.0551,"10":1.3679,"11":2.1859,"12":1.2202,"13":-0.49,"14":-0.0316,"15":0.5221,"16":0.7026,"17":0.4179,"18":-2.1689,"19":1.646,"20":-13.3639,"21":1.5151,"22":1.1175,"23":-5.3561,"24":5.0442,"25":0.8451,"26":0.3987,"27":-2.6158,"28":0.4318,"29":-0.7361,"30":0.5715,"31":-2.9501,"32":-3.7811,"33":-5.8994,"34":6.4167,"35":2.5014,"36":7.338,"37":-2.9887,"38":2.4586,"39":13.4191,"40":2.7395,"41":-3.9708,"42":1.6548,"43":-2.7554,"44":-1.5345,"45":-6.4708,"46":-4.4454,"47":-0.6224,"48":-1.0988,"49":4.4501,"50":9.2426,"51":-0.7392,"52":0.4452,"53":1.8828,"54":-2.6277,"55":-10.851,"56":-3.2353,"57":-4.4653,"58":-3.1153,"59":-1.3707,"60":7.318,"61":16.0902,"62":1.4686,"63":7.0391,"64":1.7765,"65":-4.9573,"66":-1.0578,"67":1.3668,"68":-1.4029,"69":-1.155,"70":2.6697,"71":-8.8877,"72":1.1958,"73":-3.2839,"74":-5.4425,"75":1.6809,"76":7.6812,"77":-2.4732,"78":1.738,"79":0.3781,"80":0.8718,"81":2.5886,"82":1.6911,"83":1.2953,"84":-5.5961,"85":2.174,"86":-3.5098,"87":-5.4715,"88":-9.0052,"89":-4.6038,"90":-6.7447,"91":-2.5528,"92":0.4391,"93":-4.9278,"94":-3.6695,"95":-4.8673,"96":-1.6035,"97":1.5011,"98":-5.6124,"99":4.9747,"100":1.8998,"101":3.0359,"102":6.2983,"103":-2.703,"104":1.5025,"105":6.1841,"106":-0.9357,"107":-4.8568,"108":-2.1888,"109":-4.1143,"110":-3.9874,"111":-0.0459,"112":4.7134,"113":2.8952,"114":-9.3627,"115":-4.685,"116":0.3601,"117":-1.3699,"118":9.7294,"119":11.5596,"120":0.1918,"121":3.0783,"122":-6.6828,"123":-5.4398,"124":-5.088,"125":3.6948,"126":0.0329,"127":-0.1362,"128":-0.1188,"129":-0.7579,"130":0.3278,"131":-0.977,"132":-0.9377,"133":2.2935,"134":-2.0353,"135":-1.7786,"136":5.4567,"137":-3.6368,"138":3.4996,"139":-0.0685} These are the weights on the connections between the inputs, neurons and outputs. Everything you need to know about how to win Slime Volleyball in 140 numbers !
- _lce0 11y ago
  > Everything you need to know about how to win Slime Volleyball in 140 numbers ! I know kung fu!
  - hardmaru 11y ago
    haha, I like that. Yeah, there are a total of 140 weights/biases in the neural network. This stuff is not bleeding edge - just taking techniques popular in academia and industry many years ago to make a fun game mainly for my own educational purpose. Basically recently I saw a lot of work on AI's playing ATARI games and wanted to learn reinforcement learning techniques, and the best way to learn is to build a game! I put up a description of this sketch here: http://blog.otoro.net/2015/03/28/neural-slime-volleyball/ http://blog.otoro.net/2015/03/28/neural-slime-volleyball/ Basically, the agent's brain consists of 7 neurons (compared to 250k for an ant), which are connected to all the inputs. The inputs are the game states, like locations of agents and ball, and velocities. Each neuron is also connected to each other. The first 3 neurons control the neuron's output (it's motor function), like whether it hits the up, forward, or back button - the same function that a human game player can do. The other 4 neurons are hidden neurons responsible for any deeper computation or thought, and are fed back into the input with a time lag. I also graphed a slice of the neural network in real time on the screen just for effect. More for design effect if anything.
- 0xdeadbeefbabe 11y ago
  How many numbers do you need? It isn't just a matter of trial and error is it?
  - deepnet 11y ago
    The number of weights depends on the network architecture. In this case fully connected , 1 weight per connection . 16 inputs to 7 neurons + 7 neurons to 3 outputs + 7 biases 16x7 + 7x3 + 7 = 140 http://blog.otoro.net/wp-content/uploads/sites/2/2015/03/slime_rnn.png http://blog.otoro.net/wp-content/uploads/sites/2/2015/03/sli... The number of neurons is a decidable.
- 11y ago