5 ms·

Probably leftovers from development phase.

hardmaru 11y ago
lol - apologies about my messy code guys. It's just hacked together as a rough sketch to see if it'll work. as you can see my web coding skillz still needs work. I put it up on github though (be warned about messy hackish code): https://github.com/hardmaru/neuralslimevolley https://github.com/hardmaru/neuralslimevolley
- deepnet 11y ago
  I found the code clear and comprehensible - your genetic library for convnet.js looks useful - have you seen Karpathy's reinforce.js ? I notice you left a training flag - I would like to watch the nets train so definitely going to have a go. And thanks for introducing me to CNE, I am reading John Gomez's thesis right now - very very interesting, looks to have a big advantage over Q-learning. In your blog post you compare the 'DNA' of multiple training episodes, the similar weights kind of suggests that you have exhaustively searched the problem space ?
  - hardmaru 11y ago
    Thanks - thinking of putting up a version that has not been pretrained, so users can see the training over several generations. I think I have one lying around - remind me on twitter if I forget. I'm also trying to learn 'deep q learning' (q-learning with a neural net as the q function) and other cool stuff that has recently been developed. btw, I think your previous post with the 'brain' of 140 numbers is messing up this thread a bit on my browser for some reason and the comments not wrap, wonder if there's a way to fix it.
    - deepnet 11y ago
      Many Thanks for the untrained version - I am working on visualising the evolving genome based on your RGBA illustration. I read Sutton and Barto on Q-Learning but only 'got' it when I saw it hand-done with matrix math. http://mnemstudio.org/path-finding-q-learning-tutorial.htm http://mnemstudio.org/path-finding-q-learning-tutorial.htm Sutton & Barto suggest Eligibility Traces seem necessary to make Q learn fast enough. http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node79.html http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node79.html Gomez's thesis suggests that genetic methods have an advantage over backprop for learning sequences of actions. http://www.cs.utexas.edu/users/nn/downloads/papers/gomez.phdtr03.pdf http://www.cs.utexas.edu/users/nn/downloads/papers/gomez.phd... Deep Q Learning is brilliant - evolving the Q function would certainly be very interesting - currently trying to get the ALE sending frames to my convnet :) mods have helped fix the errant brain code - thanks for the heads up.
      - hardmaru 11y ago
        cool! Let's talk further offline later- I would be interested to know what you end up doing with it, and pick your (biological) brain for ideas. Ping me on Twitter message later-
- squeaky-clean 11y ago
  Hope it didn't seem like I was trying to criticize. The app is really cool, and thanks for open-sourcing it. I'd be really interested in seeing the version that doesn't start pre-trained.
  - hardmaru 11y ago
    Hi This post outlines a version that doesn't start pre-trained http://blog.otoro.net/2015/05/13/neural-slime-volleyball-evolution-demo/ http://blog.otoro.net/2015/05/13/neural-slime-volleyball-evo...

lol - apologies about my messy code guys. It's just hacked together as a rough sketch to see if it'll work. as you can see my web coding skillz still needs work. I put it up on github though (be warned about messy hackish code): https://github.com/hardmaru/neuralslimevolley https://github.com/hardmaru/neuralslimevolley

deepnet 11y ago
I found the code clear and comprehensible - your genetic library for convnet.js looks useful - have you seen Karpathy's reinforce.js ? I notice you left a training flag - I would like to watch the nets train so definitely going to have a go. And thanks for introducing me to CNE, I am reading John Gomez's thesis right now - very very interesting, looks to have a big advantage over Q-learning. In your blog post you compare the 'DNA' of multiple training episodes, the similar weights kind of suggests that you have exhaustively searched the problem space ?
- hardmaru 11y ago
  Thanks - thinking of putting up a version that has not been pretrained, so users can see the training over several generations. I think I have one lying around - remind me on twitter if I forget. I'm also trying to learn 'deep q learning' (q-learning with a neural net as the q function) and other cool stuff that has recently been developed. btw, I think your previous post with the 'brain' of 140 numbers is messing up this thread a bit on my browser for some reason and the comments not wrap, wonder if there's a way to fix it.
  - deepnet 11y ago
    Many Thanks for the untrained version - I am working on visualising the evolving genome based on your RGBA illustration. I read Sutton and Barto on Q-Learning but only 'got' it when I saw it hand-done with matrix math. http://mnemstudio.org/path-finding-q-learning-tutorial.htm http://mnemstudio.org/path-finding-q-learning-tutorial.htm Sutton & Barto suggest Eligibility Traces seem necessary to make Q learn fast enough. http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node79.html http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node79.html Gomez's thesis suggests that genetic methods have an advantage over backprop for learning sequences of actions. http://www.cs.utexas.edu/users/nn/downloads/papers/gomez.phdtr03.pdf http://www.cs.utexas.edu/users/nn/downloads/papers/gomez.phd... Deep Q Learning is brilliant - evolving the Q function would certainly be very interesting - currently trying to get the ALE sending frames to my convnet :) mods have helped fix the errant brain code - thanks for the heads up.
    - hardmaru 11y ago
      cool! Let's talk further offline later- I would be interested to know what you end up doing with it, and pick your (biological) brain for ideas. Ping me on Twitter message later-
squeaky-clean 11y ago
Hope it didn't seem like I was trying to criticize. The app is really cool, and thanks for open-sourcing it. I'd be really interested in seeing the version that doesn't start pre-trained.
- hardmaru 11y ago
  Hi This post outlines a version that doesn't start pre-trained http://blog.otoro.net/2015/05/13/neural-slime-volleyball-evolution-demo/ http://blog.otoro.net/2015/05/13/neural-slime-volleyball-evo...