7 ms·

The NetHack Challenge: Dungeons, Dragons, and Tourists (2021)

122 points by cpeterso 4y ago

mensetmanusman 4y ago
Does this website somehow prevent smooth scrolling on the iPhone?
robonerd 4y ago
Tangentially related, but I wish the Tourist class would let me talk my way out of adversarial encounters by explaining that I'm merely a tourist and unfamiliar with local customs. Like Twoflower in The Color of Magic.
- pmontra 4y ago
  That could be counterbalanced by shopkeepers making them pay inflated prices and some cop fining them for nothing.
newobj 4y ago
Is symbolic just a fancy way of saying hand coded rules, or is it “trained” in some way, just not via neural net?
haolez 4y ago
I used to play NetHack a lot and the only thing that bothered me in the gameplay was the monsters' AI. They were basically a huge heap of mindless creatures that lived with a single objective in mind: to destroy you. I think there is a lot of opportunity to make NetHack even more amazing by having smarter monsters that come up with new strategies, often as a group, to beat the player.
CydeWeys 4y ago
Oh man, this takes me waaay back to my time spent on rec.games.roguelike.nethack about two decades ago. It literally took me a couple years of off-and-on effort (and hundreds of deaths) until I ascended for the first time. But once you put that much actual effort into getting good at it, you stay good, and I ended up doing multiple back-to-back ascensions, ascending with every class, and doing some high conduct runs, including one that was vegetarian, weaponless, genocideless, self-polymorphless, and artifact-wishless. It's definitely a hard nut to crack, and I can see why the machine learning attempts would've struggled so much, but honestly I'm surprised that the best symbolic logic bots only achieved a few thousand points on average. That's really not even getting very far into the game. I guess NetHack is a harder game than StarCraft II, as with SC, the best bots are competitive with the best humans (albeit they have advantages in inputs/attention and disadvantages in grand strategy). NetHack though is literally turn-based (like Chess), so there's no advantages to be had here, and clearly they're struggling for it.
- philsnow 4y ago
  > it literally took me a couple years of off-and-on effort (and hundreds of deaths) until I ascended for the first time. > But once you put that much actual effort into getting good at it, you stay good I had the same experience, but I don't think it's a matter of being "good" at the game so much as having memorized all of the stupid ways to die (many of them insta-deaths ) plus how to avoid them, and knowing the mechanics of how to win (I think you're supposed to figure out the invocation ritual by giving tons of money to the Oracle, but it's been decades and I can't recall if I learned it that way or just read spoilers). edit: just remembered the whole vibrating square thing and that there are a few ways of making a run unwinnable, which is a kick in the teeth when you have a promising @ going. Maybe that's just splitting hairs (is that just becoming "good"?) but in retrospect, I don't think that getting good at nethack was worth the investment in time and frustration. I will say that playing nethack improved my vocabulary.
  - CydeWeys 4y ago
    Yeah, I do think that's splitting hairs. A large part of that is what being good at the game actually entails. Also, you're underselling it -- it's not memorizing specific ways to die per se, it's learning a large number of different strategies to deal with a large number of different problematic situations. If it were just simple memorization the programs would be better at playing it. Ironically, the vibrating square is one thing that a program would do much better at than a person (also Sokoban) -- it's mind-numbingly boring for a person to do, and you can easily mess it up and frustrate yourself to no end, but a program will execute a perfect sweep to find it in no time at all. A large part of becoming good at NetHack ends up being learning to be patient, playing conservatively, and taking lots of time to think before acting. And there are plenty of mini-games within the game (such as Sokoban, potion/scroll IDing, pudding farming setup) that you need to learn well too. What helps most of all is being resourceful. There are so many different systems in the game that can all be used in the right situations to give you an advantage (e.g. Elbereth, wielding cockatrice corpses), so you just need to play slowly and cautiously and continuously think about every single possible action of the dozens available at any one point that you can use to maximal effect in any specific hairy situation.
jka 4y ago
One interesting result from this challenge that could stoke debate is that symbolic (rules-based) agents are clearly achieving better results than neural (machine-learning-based) agents. Maybe that's related to the nature of NetHack itself, compared to other games where ML has seen (often overwhelming) success? Will rules-based systems continue to succeed for NetHack, or is this a temporary condition until the state-of-the-art in machine learning improves?
- erikerikson 4y ago
  Consider that the existing MVP for ascension is a neutral network.
  - jka 4y ago
    I wasn't able to find out about that, from a quick scout around - could you share a link to it? (I did find BotHack[1] and a video of its' first ascension[2], both of which are cool) [1] - https://github.com/krajj7/BotHack https://github.com/krajj7/BotHack [2] - https://www.youtube.com/watch?v=unCQHAbGsAA https://www.youtube.com/watch?v=unCQHAbGsAA
    - tubs 4y ago
      Human brain.
      - jka 4y ago
        Hehe, ah, of course :)
- dunefox 4y ago
  I think neurosymbolic ai could do better here. It seems to be an interesting field and this might be a cool application.
- vintermann 4y ago
  Nethack is notorious for being utterly unwinnable without spoilers. There are a few claims of people who have done it, they don't hold up very plausibly: at the very least those people got indirect spoilers from talking with people who were spoiled. So it's not really surprising that hard-coded rules based bots incorporating spoiler knowledge beat learning bots trying to learn the game directly from the interface.
seren 4y ago
I just checked on nethack.alt.org high score in the last 60 days https://alt.org/nethack/top60d.html https://alt.org/nethack/top60d.html 5000 points is still a relatively low score, it corresponds to reach Dungeon level between 5 and 7, which is quite early in the game.
- CydeWeys 4y ago
  To those not that familiar with NetHack, it's possible to run up arbitrarily high scores by doing activities within the game such as pudding farming. This is essentially a risk-free way (when done correctly) to get an unbounded number of kills, and as each kill is worth points, an unbounded score as well. I believe the score maxes out at 2 billion something, i.e. the maximum value of a signed int. So you may as well consider all 7 digit and up scores for ascended characters to belong to one single group, as any of these characters could have arbitrarily inflated their score, they just chose not to, as it's relatively uninteresting/boring to do. What's more interesting is a low score ascension. That is insanely hard to pull off as it means you're taking a very fragile character into the endgame, and yet somehow pulling through.
  - ntrz 4y ago
    Various farming strategies have been severely restricted in recent versions of the game; puddings don't even drop real corpses now. There are definitely still plenty of ways to loiter around and boost your score, though.
    - CydeWeys 4y ago
      I'm showing my age here then. I mostly played version 3.3.1, then some of the 3.4.* versions. I haven't played 3.6.*.
moomin 4y ago
There’s a good reason for putting computation restrictions on the bots. Turns out without it, it’s already pretty solved: https://pellsson.github.io/ https://pellsson.github.io/
- devit 4y ago
  You can easily avoid that by replacing the RNG with a proper secure one.
  - deleted 4y ago
    [deleted]
  - dsr_ 4y ago
    Then it wouldn't be Nethack. Nethack is not a game where the dice are fair. Nethack is a game where the phase of the moon has real influence. Nethack is, though not originally designed for that purpose, a game which encourages players to learn to read the source code and familiarize themselves with text files, editing, and searching. You could build a nice first-year CS class around Nethack, taking people from "this is a fun game" to "I have a tenuous grasp on programming" in a semester.
    - CydeWeys 4y ago
      > Then it wouldn't be Nethack. Hard disagree. No human player would remotely detect a difference if the exploitable pseudorandom seed were replaced with a truly random source. > Nethack is not a game where the dice are fair. Nethack is a game where the phase of the moon has real influence. The phase of the Moon _is not random_! It is deterministic! You're arguing against yourself here. > Nethack is, though not originally designed for that purpose, a game which encourages players to learn to read the source code and familiarize themselves with text files, editing, and searching. Entirely orthogonal to the issue of using a better random number source. Indeed, a good class project might be improving the source of randomness used by NetHack.
      - HelloNurse 4y ago
dzdt 4y ago
No human has ever beaten NetHack without being "spoiled". That is, in NetHack culture, reading how-to guides or the game's source code to understand what process to follow to identify unknown objects or to navigate tricky situations. It is unsurprising that with training rules that prohibit AI's from similarly being spoiled no AI has beaten the game.
- pjlegato 4y ago
  It's worth noting that even people who have been spoiled still usually find it _extremely hard_ to beat NetHack.
- generaljelly 4y ago
  I started playing when I was 12 and have gone relatively spoiler free. I have ascended once. (I was 25 at the time) I got stuck on the vibrating square many times, the castle many times, and have died on the elemental planes many times. Because I had no idea to do what was expected to me. The first I got to Medusa I died instantly. Yes, I think some of the final puzzles are a bit too obscure... But I think you can beat it without spoilers. But I would spend entire days to learn mechanics... IE: day of kicking sinks is a day I remember fondly in my childhood. I am sure there is tons I don't know. But I have never used explore mode, and I have never looked up spoilers before ascending. Note, I had some word of mouth friends, and it took me 3 years alone to figure out altars.
- ollybee 4y ago
  Where can you see "training rules that prohibit AI's from similarly being spoiled"? It says "We imposed no limits upon how teams would design or train (if relevant) their agents" so seems they do allow spoiled AI. Even spoiled, when it's achieved, an ability to complete a task like ascending in NetHack seems more world changing to me than the LaMDA conversations.
  - failedengineer 4y ago
    I'd be completely flabbergasted if they trained an AI to parse source code while playing a game. I think it's the "lack of addition" rather than the "expressly disallowed" training rules that are being referred to.
bhaak 4y ago
Right now, Junethack, the NetHack Cross-Variant Summer Tournament (aka the tournament that is abusing players for finding bugs in nethack forks tournament) is running. This year, I don't think there are any obvious bots running. The only time a bot clan was participating AFAIK was 2015. https://junethack.net/archive/2015/scoreboard.html https://junethack.net/archive/2015/scoreboard.html And it didn't do badly (it did score 2 of the 5 clan trophies) but also not particularly well. That's mostly due to the tournament including lots of NetHack forks and the clan trophies being special side achievements that aren't necessarily needed for winning the game. So a generic bot that is trained to win the game isn't best at getting those. And for today, I think there is no bot that can win the game for the newer versions or any forks. The bots from the NetHack Challenge also can't participate as they need a custom NetHack binary that outputs the game data in a machine parsable way whereas the tournament requires you to play on existing public servers.
Buttons840 4y ago
On the topic of what's next, I'd really like to see an AI tackle a 4X strategy game like Civilization. AlphaStar is the closest we've seen, but I think it relied on superior control ("micro") more than good decision making, and the Dota AI was similar. I want to see an AI succeed in a strategy game via pure decision making in a turn-based game.
- urthor 4y ago
  I'd like to see an AI tackle 4X because Firaxis' AI is famously miserable at its own game. Firaxis iirc has never shipped an iteration of civilization where the AI could compete on a level playing field with even quite mediocre humans.
- pmoriarty 4y ago
  I'd be much more interested in watching an AI play the board game Diplomacy, charades, or see one LARP.
ArtWomb 4y ago
>>> NetHack benefits ‘strategic’ play — good play often involves executing a series of actions with a well-defined, expressible sub-objective, eg: “Find Sokoban” or “Apply a Unicorn Horn to Cure Poison”. Symbolic bots found it easy to define ‘strategy’-like subroutines and to decide when to deploy them based on rich, human-legible representations of the game state. This feels akin to Classic Game AI vs Modern AI debates that happen all the time. And even in 2022, with desktop GPU capabilities nearing supercomputing levels, it feels like rules-based, goal-oriented planning still dominates NPC & Enemy AI in games. The question really becomes, why did symbolic ai research die when it's so effective at specialization? Rather, the research obsession is solving generalization. And the problems always seem to stem around the black box itself, the lack of "human-legible representations" of its latent spaces ;)
- jerf 4y ago
  "The question really becomes, why did symbolic ai research die when it's so effective at specialization?" I think we want "AI" that scales faster than humans can implement. Game "AI" looks nothing like machine learning and looks a lot more like some programmer just smashing out some code and trying it out for a while. Some autotuning can be done, of course, but it's largely a program being written. Who has time for that?
- gwern 4y ago
  Because Nethack and video games remove most of the reasons that GOFAI failed; for example, everything hard about perception from pixels is removed when you have a little parsed text grid of cells presented to you or are hooked into the game engine and query objects directly. (The real world is not made of a clean little grid of discrete high-level objects like 'dragon' or 'black jelly'.) Moravec's paradox again: humans find perception easy but things like Sokoban hard, while GOFAI approaches find Sokoban so trivial that it's common to use it (or Sudoku) as a toy problem introduction to constraint solving. Nethack literally has levels which are just Sokoban and which are important to solve; GOFAI can push the boulders around perfectly, even though it would be unable to recognize a photograph of 'a boulder' or use the word 'boulder' in a story. Then you have the extensive hand-engineering of expert knowledge which goes into ascension agents and the symbolic winners, above and beyond merely plugging in a Sokoban solver. There are increasing experiments in making DRL agents exploit or initialize from pretrained language models (https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2201.12122 https://arxiv.org/abs/2201.12122 https://arxiv.org/abs/2204.01691#google https://arxiv.org/abs/2204.01691#google https://ai.stanford.edu/blog/DrRepair/ https://ai.stanford.edu/blog/DrRepair/ https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2009.03393 https://arxiv.org/abs/2009.03393 https://arxiv.org/abs/2204.00598 https://arxiv.org/abs/2204.00598 come to mind) or reading manuals (https://arxiv.org/abs/1401.5390 https://arxiv.org/abs/1401.5390 all the way back in 2012!), and of course, a DRL agent can learn a tremendous amount without actually doing any playing by offline and off-policy and imitation learning, but while it is exciting and things like Gato look like the future, there is a long way to go from feeding in a dump of the Nethack wiki which mentions offhandedly "you can do X" to an agent recognizing an opportunity for X in the wild and executing it. (Which is something that symbolic approaches also don't come anywhere close to doing, because they just cheat by the capability being given to them by hand-engineering rather than having to autonomously read, understand, and apply.)
eljimmy 4y ago
This reminds me of Zebulon (sp?) which I think was run by Packetstorm Security - although it was more geared towards infosec than AI. Does anyone else remember that?

122 points by cpeterso 4y ago

Does this website somehow prevent smooth scrolling on the iPhone?

Tangentially related, but I wish the Tourist class would let me talk my way out of adversarial encounters by explaining that I'm merely a tourist and unfamiliar with local customs. Like Twoflower in The Color of Magic.

pmontra 4y ago
That could be counterbalanced by shopkeepers making them pay inflated prices and some cop fining them for nothing.

newobj 4y ago

Is symbolic just a fancy way of saying hand coded rules, or is it “trained” in some way, just not via neural net?

haolez 4y ago

I used to play NetHack a lot and the only thing that bothered me in the gameplay was the monsters' AI. They were basically a huge heap of mindless creatures that lived with a single objective in mind: to destroy you. I think there is a lot of opportunity to make NetHack even more amazing by having smarter monsters that come up with new strategies, often as a group, to beat the player.

CydeWeys 4y ago

Oh man, this takes me waaay back to my time spent on rec.games.roguelike.nethack about two decades ago. It literally took me a couple years of off-and-on effort (and hundreds of deaths) until I ascended for the first time. But once you put that much actual effort into getting good at it, you stay good, and I ended up doing multiple back-to-back ascensions, ascending with every class, and doing some high conduct runs, including one that was vegetarian, weaponless, genocideless, self-polymorphless, and artifact-wishless. It's definitely a hard nut to crack, and I can see why the machine learning attempts would've struggled so much, but honestly I'm surprised that the best symbolic logic bots only achieved a few thousand points on average. That's really not even getting very far into the game. I guess NetHack is a harder game than StarCraft II, as with SC, the best bots are competitive with the best humans (albeit they have advantages in inputs/attention and disadvantages in grand strategy). NetHack though is literally turn-based (like Chess), so there's no advantages to be had here, and clearly they're struggling for it.

philsnow 4y ago
> it literally took me a couple years of off-and-on effort (and hundreds of deaths) until I ascended for the first time. > But once you put that much actual effort into getting good at it, you stay good I had the same experience, but I don't think it's a matter of being "good" at the game so much as having memorized all of the stupid ways to die (many of them insta-deaths ) plus how to avoid them, and knowing the mechanics of how to win (I think you're supposed to figure out the invocation ritual by giving tons of money to the Oracle, but it's been decades and I can't recall if I learned it that way or just read spoilers). edit: just remembered the whole vibrating square thing and that there are a few ways of making a run unwinnable, which is a kick in the teeth when you have a promising @ going. Maybe that's just splitting hairs (is that just becoming "good"?) but in retrospect, I don't think that getting good at nethack was worth the investment in time and frustration. I will say that playing nethack improved my vocabulary.
- CydeWeys 4y ago
  Yeah, I do think that's splitting hairs. A large part of that is what being good at the game actually entails. Also, you're underselling it -- it's not memorizing specific ways to die per se, it's learning a large number of different strategies to deal with a large number of different problematic situations. If it were just simple memorization the programs would be better at playing it. Ironically, the vibrating square is one thing that a program would do much better at than a person (also Sokoban) -- it's mind-numbingly boring for a person to do, and you can easily mess it up and frustrate yourself to no end, but a program will execute a perfect sweep to find it in no time at all. A large part of becoming good at NetHack ends up being learning to be patient, playing conservatively, and taking lots of time to think before acting. And there are plenty of mini-games within the game (such as Sokoban, potion/scroll IDing, pudding farming setup) that you need to learn well too. What helps most of all is being resourceful. There are so many different systems in the game that can all be used in the right situations to give you an advantage (e.g. Elbereth, wielding cockatrice corpses), so you just need to play slowly and cautiously and continuously think about every single possible action of the dozens available at any one point that you can use to maximal effect in any specific hairy situation.

jka 4y ago

One interesting result from this challenge that could stoke debate is that symbolic (rules-based) agents are clearly achieving better results than neural (machine-learning-based) agents. Maybe that's related to the nature of NetHack itself, compared to other games where ML has seen (often overwhelming) success? Will rules-based systems continue to succeed for NetHack, or is this a temporary condition until the state-of-the-art in machine learning improves?

erikerikson 4y ago
Consider that the existing MVP for ascension is a neutral network.
- jka 4y ago
  I wasn't able to find out about that, from a quick scout around - could you share a link to it? (I did find BotHack[1] and a video of its' first ascension[2], both of which are cool) [1] - https://github.com/krajj7/BotHack https://github.com/krajj7/BotHack [2] - https://www.youtube.com/watch?v=unCQHAbGsAA https://www.youtube.com/watch?v=unCQHAbGsAA
  - tubs 4y ago
    Human brain.
    - jka 4y ago
      Hehe, ah, of course :)
dunefox 4y ago
I think neurosymbolic ai could do better here. It seems to be an interesting field and this might be a cool application.
vintermann 4y ago
Nethack is notorious for being utterly unwinnable without spoilers. There are a few claims of people who have done it, they don't hold up very plausibly: at the very least those people got indirect spoilers from talking with people who were spoiled. So it's not really surprising that hard-coded rules based bots incorporating spoiler knowledge beat learning bots trying to learn the game directly from the interface.

seren 4y ago

I just checked on nethack.alt.org high score in the last 60 days https://alt.org/nethack/top60d.html https://alt.org/nethack/top60d.html 5000 points is still a relatively low score, it corresponds to reach Dungeon level between 5 and 7, which is quite early in the game.

CydeWeys 4y ago
To those not that familiar with NetHack, it's possible to run up arbitrarily high scores by doing activities within the game such as pudding farming. This is essentially a risk-free way (when done correctly) to get an unbounded number of kills, and as each kill is worth points, an unbounded score as well. I believe the score maxes out at 2 billion something, i.e. the maximum value of a signed int. So you may as well consider all 7 digit and up scores for ascended characters to belong to one single group, as any of these characters could have arbitrarily inflated their score, they just chose not to, as it's relatively uninteresting/boring to do. What's more interesting is a low score ascension. That is insanely hard to pull off as it means you're taking a very fragile character into the endgame, and yet somehow pulling through.
- ntrz 4y ago
  Various farming strategies have been severely restricted in recent versions of the game; puddings don't even drop real corpses now. There are definitely still plenty of ways to loiter around and boost your score, though.
  - CydeWeys 4y ago
    I'm showing my age here then. I mostly played version 3.3.1, then some of the 3.4.* versions. I haven't played 3.6.*.

moomin 4y ago

There’s a good reason for putting computation restrictions on the bots. Turns out without it, it’s already pretty solved: https://pellsson.github.io/ https://pellsson.github.io/

devit 4y ago
You can easily avoid that by replacing the RNG with a proper secure one.
- deleted 4y ago
  [deleted]
- dsr_ 4y ago
  Then it wouldn't be Nethack. Nethack is not a game where the dice are fair. Nethack is a game where the phase of the moon has real influence. Nethack is, though not originally designed for that purpose, a game which encourages players to learn to read the source code and familiarize themselves with text files, editing, and searching. You could build a nice first-year CS class around Nethack, taking people from "this is a fun game" to "I have a tenuous grasp on programming" in a semester.
  - CydeWeys 4y ago
    > Then it wouldn't be Nethack. Hard disagree. No human player would remotely detect a difference if the exploitable pseudorandom seed were replaced with a truly random source. > Nethack is not a game where the dice are fair. Nethack is a game where the phase of the moon has real influence. The phase of the Moon _is not random_! It is deterministic! You're arguing against yourself here. > Nethack is, though not originally designed for that purpose, a game which encourages players to learn to read the source code and familiarize themselves with text files, editing, and searching. Entirely orthogonal to the issue of using a better random number source. Indeed, a good class project might be improving the source of randomness used by NetHack.
    - HelloNurse 4y ago

dzdt 4y ago

No human has ever beaten NetHack without being "spoiled". That is, in NetHack culture, reading how-to guides or the game's source code to understand what process to follow to identify unknown objects or to navigate tricky situations. It is unsurprising that with training rules that prohibit AI's from similarly being spoiled no AI has beaten the game.

pjlegato 4y ago
It's worth noting that even people who have been spoiled still usually find it _extremely hard_ to beat NetHack.
generaljelly 4y ago
I started playing when I was 12 and have gone relatively spoiler free. I have ascended once. (I was 25 at the time) I got stuck on the vibrating square many times, the castle many times, and have died on the elemental planes many times. Because I had no idea to do what was expected to me. The first I got to Medusa I died instantly. Yes, I think some of the final puzzles are a bit too obscure... But I think you can beat it without spoilers. But I would spend entire days to learn mechanics... IE: day of kicking sinks is a day I remember fondly in my childhood. I am sure there is tons I don't know. But I have never used explore mode, and I have never looked up spoilers before ascending. Note, I had some word of mouth friends, and it took me 3 years alone to figure out altars.
ollybee 4y ago
Where can you see "training rules that prohibit AI's from similarly being spoiled"? It says "We imposed no limits upon how teams would design or train (if relevant) their agents" so seems they do allow spoiled AI. Even spoiled, when it's achieved, an ability to complete a task like ascending in NetHack seems more world changing to me than the LaMDA conversations.
- failedengineer 4y ago
  I'd be completely flabbergasted if they trained an AI to parse source code while playing a game. I think it's the "lack of addition" rather than the "expressly disallowed" training rules that are being referred to.

bhaak 4y ago

Right now, Junethack, the NetHack Cross-Variant Summer Tournament (aka the tournament that is abusing players for finding bugs in nethack forks tournament) is running. This year, I don't think there are any obvious bots running. The only time a bot clan was participating AFAIK was 2015. https://junethack.net/archive/2015/scoreboard.html https://junethack.net/archive/2015/scoreboard.html And it didn't do badly (it did score 2 of the 5 clan trophies) but also not particularly well. That's mostly due to the tournament including lots of NetHack forks and the clan trophies being special side achievements that aren't necessarily needed for winning the game. So a generic bot that is trained to win the game isn't best at getting those. And for today, I think there is no bot that can win the game for the newer versions or any forks. The bots from the NetHack Challenge also can't participate as they need a custom NetHack binary that outputs the game data in a machine parsable way whereas the tournament requires you to play on existing public servers.

Buttons840 4y ago

On the topic of what's next, I'd really like to see an AI tackle a 4X strategy game like Civilization. AlphaStar is the closest we've seen, but I think it relied on superior control ("micro") more than good decision making, and the Dota AI was similar. I want to see an AI succeed in a strategy game via pure decision making in a turn-based game.

urthor 4y ago
I'd like to see an AI tackle 4X because Firaxis' AI is famously miserable at its own game. Firaxis iirc has never shipped an iteration of civilization where the AI could compete on a level playing field with even quite mediocre humans.
pmoriarty 4y ago
I'd be much more interested in watching an AI play the board game Diplomacy, charades, or see one LARP.

ArtWomb 4y ago

>>> NetHack benefits ‘strategic’ play — good play often involves executing a series of actions with a well-defined, expressible sub-objective, eg: “Find Sokoban” or “Apply a Unicorn Horn to Cure Poison”. Symbolic bots found it easy to define ‘strategy’-like subroutines and to decide when to deploy them based on rich, human-legible representations of the game state. This feels akin to Classic Game AI vs Modern AI debates that happen all the time. And even in 2022, with desktop GPU capabilities nearing supercomputing levels, it feels like rules-based, goal-oriented planning still dominates NPC & Enemy AI in games. The question really becomes, why did symbolic ai research die when it's so effective at specialization? Rather, the research obsession is solving generalization. And the problems always seem to stem around the black box itself, the lack of "human-legible representations" of its latent spaces ;)

jerf 4y ago
"The question really becomes, why did symbolic ai research die when it's so effective at specialization?" I think we want "AI" that scales faster than humans can implement. Game "AI" looks nothing like machine learning and looks a lot more like some programmer just smashing out some code and trying it out for a while. Some autotuning can be done, of course, but it's largely a program being written. Who has time for that?
gwern 4y ago
Because Nethack and video games remove most of the reasons that GOFAI failed; for example, everything hard about perception from pixels is removed when you have a little parsed text grid of cells presented to you or are hooked into the game engine and query objects directly. (The real world is not made of a clean little grid of discrete high-level objects like 'dragon' or 'black jelly'.) Moravec's paradox again: humans find perception easy but things like Sokoban hard, while GOFAI approaches find Sokoban so trivial that it's common to use it (or Sudoku) as a toy problem introduction to constraint solving. Nethack literally has levels which are just Sokoban and which are important to solve; GOFAI can push the boulders around perfectly, even though it would be unable to recognize a photograph of 'a boulder' or use the word 'boulder' in a story. Then you have the extensive hand-engineering of expert knowledge which goes into ascension agents and the symbolic winners, above and beyond merely plugging in a Sokoban solver. There are increasing experiments in making DRL agents exploit or initialize from pretrained language models (https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2201.12122 https://arxiv.org/abs/2201.12122 https://arxiv.org/abs/2204.01691#google https://arxiv.org/abs/2204.01691#google https://ai.stanford.edu/blog/DrRepair/ https://ai.stanford.edu/blog/DrRepair/ https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2005.07648#google https://arxiv.org/abs/2009.03393 https://arxiv.org/abs/2009.03393 https://arxiv.org/abs/2204.00598 https://arxiv.org/abs/2204.00598 come to mind) or reading manuals (https://arxiv.org/abs/1401.5390 https://arxiv.org/abs/1401.5390 all the way back in 2012!), and of course, a DRL agent can learn a tremendous amount without actually doing any playing by offline and off-policy and imitation learning, but while it is exciting and things like Gato look like the future, there is a long way to go from feeding in a dump of the Nethack wiki which mentions offhandedly "you can do X" to an agent recognizing an opportunity for X in the wild and executing it. (Which is something that symbolic approaches also don't come anywhere close to doing, because they just cheat by the capability being given to them by hand-engineering rather than having to autonomously read, understand, and apply.)

eljimmy 4y ago

This reminds me of Zebulon (sp?) which I think was run by Packetstorm Security - although it was more geared towards infosec than AI. Does anyone else remember that?