4 ms·
One interesting result from this challenge that could stoke debate is that symbolic (rules-based) agents are clearly achieving better results than neural (machi
by jka 4y ago
One interesting result from this challenge that could stoke debate is that symbolic (rules-based) agents are clearly achieving better results than neural (machine-learning-based) agents.
Maybe that's related to the nature of NetHack itself, compared to other games where ML has seen (often overwhelming) success?
Will rules-based systems continue to succeed for NetHack, or is this a temporary condition until the state-of-the-art in machine learning improves?
- erikerikson 4y agoConsider that the existing MVP for ascension is a neutral network.
- jka 4y agoI wasn't able to find out about that, from a quick scout around - could you share a link to it? (I did find BotHack[1] and a video of its' first ascension[2], both of which are cool) [1] - https://github.com/krajj7/BotHack https://github.com/krajj7/BotHack [2] - https://www.youtube.com/watch?v=unCQHAbGsAA https://www.youtube.com/watch?v=unCQHAbGsAA
- dunefox 4y agoI think neurosymbolic ai could do better here. It seems to be an interesting field and this might be a cool application.
- vintermann 4y agoNethack is notorious for being utterly unwinnable without spoilers. There are a few claims of people who have done it, they don't hold up very plausibly: at the very least those people got indirect spoilers from talking with people who were spoiled. So it's not really surprising that hard-coded rules based bots incorporating spoiler knowledge beat learning bots trying to learn the game directly from the interface.
- linspace 4y agoBut we have neural networks for NLP, who could read the spoilers.
- moomin 4y agoDCSS might be a better target, in that it doesn’t reward hanging around on one level, and has far fewer things you need to be spoiled on (you need to know that picking up runes is good, and how to kill hydras, that’s about it) But it has a steady stream of qualitatively different challenges on different floors, forcing a lot of contextual behaviour. And there’s a little bit of long term strategy that can really mess up an AI.
- pmoriarty 4y agoDCSS is just a much simpler game than Nethack, so a bot would likely do well.
- vintermann 4y agoIt's questionable if it's simpler (notwithstanding all the things they've cut from dcss over the years). The main obstacle for bots is that to win nethack you need to know lots of things you're unlikely to find out by trial and error. Like how eating a lizard corpse can prevent stoning from a cockatrice's hissing, or hallucination can prevent a Lich's touch of death, how you can apply a towel to wipe your face if you've been blinded by a thrown cream pie etc. Know all the tricks, and the game becomes much easier. You need a lot of knowledge to win DCSS, too, but it's more about wise use of limited resources, playing the hand you're dealt, and the dangers of various locations. It's much harder to codify as spoilers, but you will learn by trial and error.
- kryptiskt 4y agoI imagine you'd at least have to inject domain knowledge into the model, the ruleset isn't as simple as chess or go, and all the places where a bunch of actions have to be strung together just so will lead to a very spiky parameter space where gradient descent struggles.