4 ms·
What kind of bug would make machine learning suddenly 40% worse at NetHack?
- tokai 2y agoRL agents can't ascent? That's something other nethack bots solved a decade ago. They seem to have a really bad grasp of the game. In any case a badly fitted model is not a bug.
- sebstefan 2y ago>Members of the Legendary Computer Bugs Tribunal, honored guests, if I may have your attention? I would, humbly, submit a new contender for your esteemed judgment. You may or may not find it novel, you may even deign to call it a "bug," but I assure you, you will find it entertaining. It gets a pass but my top 1 remains "I can't send emails further than 500 miles" http://web.mit.edu/jemorris/humor/500-miles http://web.mit.edu/jemorris/humor/500-miles
- viraptor 2y agoI'd go with "can't print in Tuesdays" https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161/comments/28 https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
- ffhhj 2y agoAt least it's no allergic to vanilla icecream: https://news.ycombinator.com/item?id=21779857 https://news.ycombinator.com/item?id=21779857
- david422 2y agoHadn't seen that one before. Imagine trying to troubleshoot that "can't print" bug with someone the next day. I guess once you had somehow narrowed it down to Tues it would be reproducible.
- junon 2y agoI hadn't seen this one, that's some awesome debugging by both parties.
- eythian 2y agoMore like that: https://500mile.email/ https://500mile.email/
- sebstefan 2y agoThat's a gold mine! https://predr.ag/blog/wifi-only-works-when-its-raining/ https://predr.ag/blog/wifi-only-works-when-its-raining/ https://www.linkedin.com/pulse/application-crashes-when-i-drink-coke-kaushik-raghavan/ https://www.linkedin.com/pulse/application-crashes-when-i-dr... https://gyrovague.com/2015/07/29/crashes-only-on-wednesdays/ https://gyrovague.com/2015/07/29/crashes-only-on-wednesdays/
- OptionOfT 2y agoThere is an older story out there about WiFi failing during a storm, even though all the cables and antennas are fine. I'll see if I can find it. Edit: it was the other way around. https://predr.ag/blog/wifi-only-works-when-its-raining/ https://predr.ag/blog/wifi-only-works-when-its-raining/
- samsartor 2y agoMy favorite has always been "I can't log in when I stand up" https://www.reddit.com/r/talesfromtechsupport/comments/3v52pw/i_cant_log_in_when_i_stand_up/ https://www.reddit.com/r/talesfromtechsupport/comments/3v52p...
- smokel 2y agoFor some more information on solving Nethack with AI, check out the (outstanding) TalkRL podcast: https://www.talkrl.com/episodes/pierluca-doro-and-martin-klissarov https://www.talkrl.com/episodes/pierluca-doro-and-martin-kli...
- jsjohnst 2y agoInteresting story, if maybe a bit oversold. Since the game apparently announces this (didn’t know that), shouldn’t the model have detected a significant difference during the gameplay?
- elijaht 2y agoThe article mentions that “It simply doesn't have data about full moon variables in its training data, so a branching series of decisions likely leads to lesser outcomes, or just confusion.”
- thaumasiotes 2y agoIt does say that, but I'd like to know more about the mechanisms involved. The problem is that every action you take generates better results than you'd expect. I can see why prediction would be worse while that's going on, but why would performance be reduced?
- bravetraveler 2y agoI believe games like NetHack have monkey paw situations where fortune may not actually be that fortunate My memory has not lasted well
- andersource 2y agoI can imagine it's a bit like if gravity suddenly changed to 0.9g. Everything(?) is easier but a lot of people would probably stumble around a bit before muscle memory, coordination etc. get used to the change.
- seanhunter 2y agoNethack is weird. It's literally a game comprised of a huge number of special cases all welded together. A lot of the crucial info for playing the game is not really discoverable through the interface, so Humans used to mostly learn the game by reading the source code and now they mostly learn the game by reading guides. The bug here is not in nethack but just in the training, which meant some of the special cases (full moons, fri 13ths etc) weren't in the training data. They should have been running the training in VMs with the clock set to include these cases. Honestly a lot of the reporting of this "bug" seems wildly overblown.
- Tepix 2y agoI'm surprised that the researches had played so little nethack that they didn't know about this. By the way, this story is several weeks old, ars is late in covering it.
- eru 2y agoI've played quite a lot of NetHack, but I can imagine not thinking about this, when in the middle of debugging. NetHack does warn you when you (re)-start a session that it's full moon (or new moon, which also has effects).
- deleted 2y ago[deleted]
- TeMPOraL 2y agoInteresting case, but the article itself really feels like a submarine ad for Singularity, whatever that is.
- tantalor 2y agoIf anything it gave them a false sense of confidence: > our whole environment is in a single, self-contained file Except the parts of the environment which aren't in that file, like the current date.
- jmclnx 2y agoA quick note, junethack has started, join the fun: https://junethack.net/ https://junethack.net/ Just started a game, I got: "Be careful! New moon tonight." I wonder if that will affect this learning test ?
- nottorp 2y agoIt trained itself on its own gameplay? :)
- tantalor 2y agoThat's how people get better at video games too.
- nottorp 2y agoFrom TFA: Of course, "score" is not a real metric for success in NetHack, as Cupiał himself noted. Ask a model to get the best score, and it will farm the heck out of early-stage monsters because it never gets bored. "Finding items required for [ascension] or even [just] doing a quest is too much for pure RL agent," Cupiał wrote. Another neural network, AutoAscend, does a better job of progressing through the game, but "even it can only solve sokoban and reach mines end," Cupiał notes. The NN seems to be good at grinding. They should make some for those free to play games.
- PhasmaFelis 2y agoThis is why idle games are the wave of the future. They play themselves with minimal input, so you can spend your time doing other things, like playing better games.
- qduy88 2y ago[flagged]
- fractal618 2y agodiff (40% worse code, pre-40% worse code)
- nrclark 2y agothis kind: https://tron.fandom.com/wiki/Gridbug https://tron.fandom.com/wiki/Gridbug
- bravetraveler 2y agoWithout the AI veil this is: my 'expect' implementation broke because it didn't see what I thought it was supposed to Or, something that has been going on regularly for decades. This isn't even close to the quality of the 500 mile email IMO, yet they seem to be doing everything possible to ride those coattails
- surfingdino 2y agoIt's not technically a bug, but a "political compromise" by a self-professed "not a technical manager" who had aversion to including version numbers in API endpoints. He asked us to come up with a way of enabling tests of a new feature in the API (new response payload schema). We submitted two proposals, one with v1/v2 endpoints returning payloads using old and new schema, or a URL param ?v=1 or ?v-2... He was not happy and asked one of the senior technical architects of the project to weight in and the guy said we should use the URL param. That was not good enough and we were told to go away and wait for a decision. Three months later we were told to implement a new cron job to deploy v2 into the integration test environment. Another cron job would then deploy v1 into that environment two hours after deploying v2. Yes, you read that right, v2 would be available for two hours per week, because a non-technical manager would not trust software devs to do the right thing for their colleagues testing their implementations of the API. I learned recently that almost a decade later those cron jobs are still there. If your tests are only working on day X of the week within a two-hour window, that's why. Sorry, cannot name names due to NDA.
- TrainedMonkey 2y ago> It's not technically a bug, It is absolutely an anti-feature. Most things in life function on some level of predictability. Computers are pretty high on predictability scale, do the least surprising thing and all that. I sincerely hope there is some kind of feedback mechanism for the "not a technical manager" to either learn or get less decision making authority.
- surfingdino 2y agoNo, the manager is still there having skilfully removed anyone who has any clue how to build, test, or deploy software from the organisation. And he does not accept points of view other than his own.
- tojaprice 2y agoWhat exactly is his aversion? Did he ever say? It seems like an inane hill for him to choose to die on.
- zer00eyz 2y agoThis is the older article: https://news.ycombinator.com/item?id=28686016 https://news.ycombinator.com/item?id=28686016
- laurencei 2y agoI'm not a machine learning person - so I'm confused about this. As someone who doesnt understand ML - I have always assumed the whole point of ML is to try different things in the game, almost randomly, and over (long) periods of time the AI gets better and better at the game. If having a single unexpected event causes such a large swing in outcome, and the AI cant "explain" what is different to cause the swing, then what exactly is the ML doing for it to fail on such a seemingly simple change? Doesnt that defeat the whole purpose of this? I'm obviously missing something obvious - because I would assume the real goal of ML is that it can teach itself the game, even if that involves unexpected situations, as a human does?
- stetrain 2y agoHumans train simultaneously as they operate, and humans can see the message about the full moon. If nobody includes the full moon message as input to the ML model, and tries to operate the ML model with the training it has achieved running in non-full-moon mode, its operating score in full-moon-mode may be lower. Even if it had proportional training time against full-moon-mode to incorporate that into the model, if you don't tell it when full-moon-mode is active wouldn't the optimal behavior be to optimize the score for 27/28 days vs 1/28 days of the month? If full-moon-mode is an input to the model, then it can trained to optimize for both scenarios.
- laurencei 2y agoSo for ML to work, it has to know all permutations of a game? Does that mean ML is useless for non-deterministic games with random outcomes or procedural generation?
- atrus 2y agoIt still worked, just not as well. The program was trained, built it's intuition for the game. It had a set script and assumptions. Some of those assumptions were different, and since it's not learning/training it couldn't adjust for those new assumptions, so it didn't do as well If you a human, were forced to follow a set script/assumptions the same would happen to you.
- sva_ 2y agoMeh, RL really lacks robustness against changes in the environment if they weren't in the training.
- flobosg 2y ago> What a terrible night to have a learning model. They blew it: It’s “What a horrible night…”
- Ladsko 2y agoI have no knowledge of the game. In one tweet @CupiaBart writes: > Maximizing the score means that you will just farm monsters. Finding items required for ascention or even Just doing a quest is too much for pure RL agent. So what is the stop condition then? Elapsed time? Does it run out of monsters sooner because the fullmoon makes "werecreatures mostly kept to their animal forms" and there are simply less easily farmable points in early levels?
- tux3 2y agoThe stop condition is that the bots just die miserably after sitting around, because they run out of items, or the monsters scale faster than them, or they run into one of the thousands of funny/stupid ways you can die in NetHack. There is a LOT of knowledge and strategy that is VERY FAR from obvious in this game. "Unspoiled" players who haven't read the wiki only have a very faint chance of winning the game. If you sit around in early levels without trying to make progress, you eventually run out of food, your equipment will not improve and may even degrade, and worst of all you level up, which means monsters start scaling faster than you. You have to rely on prayers to survive, but prayers have a random cooldown, and if you pray too early, your god will make sure you regret it.
- deleted 2y ago[deleted]
- flaminHotSpeedo 2y agoThe literal whole point of the game (as described by the opening text) is to retrieve the Amulet of Yendor and offer it to your chosen "god" (called ascension). If score is not tied to progress in the game, I'd say the agent's scoring system is, by definition, incorrect.
- picometer 2y agoQuestion for ML/AI people: is it still common to use the term “overfitting” for these cases, where a model is overtrained on one thing, to the detriment of another? Or is that term only used for literal curve fitting?
- oriel 2y agoUnfortunately pithy response (in abstract, not specific to this case) boils down to: Because they fired the people who how the system worked.
- deleted 2y ago[deleted]
- abetusk 2y agoOne of the best pieces of advice I ever got was "look at your data". Sometimes it's hard, sometimes you need to come up with novel visualizations, sometimes it can only give partial insight or just be noise, but I always strive, where I can, to have some type of method to look at data, in as close of a form as it was generated as possible. I think in this instance, look at an actual run might have caught this issue earlier. They might have seen the "it's a full moon" message and might think that's odd, or see were-creatures keeping their form or seeing agents being extra lucky, or whatever, but running it headless and just looking at the score means they're cutting off a huge signal vector (and, admittedly, a huge noise vector).
- DerekL 2y agoHere's the previous discussion of the original Twitter thread: https://news.ycombinator.com/item?id=40472226 https://news.ycombinator.com/item?id=40472226.
- flaminHotSpeedo 2y agoThe title here is simply wrong, there's no nuance that could possibly make it correct. Nethack, by design, changes the gameplay (slightly for some characters, greatly for others) based on the phase of the moon. It tells you this, the same way the game tells you everything else. There is no sane justification to call this a bug on either side, it's just a poorly trained model responding poorly to a feature it hadn't seen in training