4 ms·

Shameless self-promotion post: if you're interested in AI based on Markov, I maintain an open-source library here https://github.com/Svalorzen/AI-Toolbox https:

Shameless self-promotion post: if you're interested in AI based on Markov, I maintain an open-source library here https://github.com/Svalorzen/AI-Toolbox https://github.com/Svalorzen/AI-Toolbox. It can be used for general autonomous agents which need to act in response to environmental stimuli. I've used it to learn to play StarCraft and to control cameras to predict people's movements around buildings. Any comments and suggestions are welcome =)

detaro 11y ago
> I've used it to learn to play StarCraft and to control cameras to predict people's movements around buildings. If you've described / published either of these examples somewhere, you should submit them to HN!
- svalorzen 11y ago
  I actually have a video [0] and code [1] of the first (which was a one-month project), while the second is my thesis which is going to be finished soon. Maybe I'll post them as a separate entry one day, but I always think these things are too simple and trivial to be interesting.. [0]: https://www.youtube.com/watch?v=TrKMBIR82Qw https://www.youtube.com/watch?v=TrKMBIR82Qw [1]: https://github.com/Svalorzen/DESOLATOR https://github.com/Svalorzen/DESOLATOR
  - nickpsecurity 11y ago
    Loved watching the Dragoons dance around their opponents while taking almost no damage. They seemed to have slipped a few times where they could've doubled up on fire but just one was firing instead. I might have looked at it wrong though.
denniskane 11y ago
Thank you for this, I will look into it :). I was actually going to tell these people to give me a github link or go away, haha. I don't need any kind of license page that looks like http://alchemy.cs.washington.edu/license.php?action=2 http://alchemy.cs.washington.edu/license.php?action=2, thank you very much! By the way, you can talk to my AI guy if you have a recent Chrome/Chromium browser. It is actually an entire OS in a browser, and the AI guy is an app called Bertie. My current implementation of the OS is called "The Native Client Proving Ground", and can be found here: https://nacl-pg.appspot.com/desk https://nacl-pg.appspot.com/desk. The OS should install in less than 2 seconds after clicking the link. Then you just open the Applications folder and click on Bertie's face. He will then introduce himself, and you can start talking to him.
Dn_Ab 11y ago
Ah, I was confused for a second—I'd thought Markov was a library, but you meant the markov assumption—the topics are actually loosely related and orthogonal. Your excellent looking library deals with reinforcement learning agents that model environment/agent interactions as a (PO)Markov Decision Process, where as the Alchemy library combines FOL with network representations of particular (satisfying certain markov properties) probability distributions to perform inference. More pertinent to your post, Sutton's working on an updated RL book here: http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.p... If anyou have the time, Chapter 15 (pdf pg 273) of the above link is a fascinating read. In particular, TD-Gammon had already achieved impressive results using NNs in the early 90s; reaching world class levels in Backgammon with zero specialized knowledge.
marmaduke 11y ago
Shameless naive question: are these techniques suitable for time series models?
- svalorzen 11y ago
  You mean data-fitting? Possibly (every problem can be modeled as an MDP) but it's definitely not the most efficient nor the most intuitive way to go about it, so I wouldn't recommend it.

detaro 11y ago

> I've used it to learn to play StarCraft and to control cameras to predict people's movements around buildings. If you've described / published either of these examples somewhere, you should submit them to HN!

svalorzen 11y ago
I actually have a video [0] and code [1] of the first (which was a one-month project), while the second is my thesis which is going to be finished soon. Maybe I'll post them as a separate entry one day, but I always think these things are too simple and trivial to be interesting.. [0]: https://www.youtube.com/watch?v=TrKMBIR82Qw https://www.youtube.com/watch?v=TrKMBIR82Qw [1]: https://github.com/Svalorzen/DESOLATOR https://github.com/Svalorzen/DESOLATOR
- nickpsecurity 11y ago
  Loved watching the Dragoons dance around their opponents while taking almost no damage. They seemed to have slipped a few times where they could've doubled up on fire but just one was firing instead. I might have looked at it wrong though.

denniskane 11y ago

Thank you for this, I will look into it :). I was actually going to tell these people to give me a github link or go away, haha. I don't need any kind of license page that looks like http://alchemy.cs.washington.edu/license.php?action=2 http://alchemy.cs.washington.edu/license.php?action=2, thank you very much! By the way, you can talk to my AI guy if you have a recent Chrome/Chromium browser. It is actually an entire OS in a browser, and the AI guy is an app called Bertie. My current implementation of the OS is called "The Native Client Proving Ground", and can be found here: https://nacl-pg.appspot.com/desk https://nacl-pg.appspot.com/desk. The OS should install in less than 2 seconds after clicking the link. Then you just open the Applications folder and click on Bertie's face. He will then introduce himself, and you can start talking to him.

Dn_Ab 11y ago

Ah, I was confused for a second—I'd thought Markov was a library, but you meant the markov assumption—the topics are actually loosely related and orthogonal. Your excellent looking library deals with reinforcement learning agents that model environment/agent interactions as a (PO)Markov Decision Process, where as the Alchemy library combines FOL with network representations of particular (satisfying certain markov properties) probability distributions to perform inference. More pertinent to your post, Sutton's working on an updated RL book here: http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.p... If anyou have the time, Chapter 15 (pdf pg 273) of the above link is a fascinating read. In particular, TD-Gammon had already achieved impressive results using NNs in the early 90s; reaching world class levels in Backgammon with zero specialized knowledge.

marmaduke 11y ago

Shameless naive question: are these techniques suitable for time series models?

svalorzen 11y ago
You mean data-fitting? Possibly (every problem can be modeled as an MDP) but it's definitely not the most efficient nor the most intuitive way to go about it, so I wouldn't recommend it.