11 ms·
I replaced Animal Crossing's dialogue with a live LLM by hacking GameCube memory
https://github.com/vuciv/animal-crossing-llm-mod https://github.com/vuciv/animal-crossing-llm-mod
- galacticaactual 1y agoVery cool, nice work!
- sardonyx001 1y agoThis is so cool. It opens up a whole new dimension to messing with old video games where AI can change things like how characters move, loot box contents etc.
- Master_Odin 1y agoIn terms of changing loot box contents, that has been going on for at least the past like 7+ years through "randomizers", though I think mostly used by speed runners.
- wincy 1y agoThat's hilarious that you give the villagers thought and the first thing they want to do is overthrow Tom Nook. If this works in Gamecube on an emulator, surely it'd be possible to make it work on the Switch emulators as well?
- amilios 1y agoWould require decompilation of the Animal Crossing game code for the Switch. I believe DRM has gotten a lot better since the Gamecube days as well. Hypothetically possible maybe but good luck haha
- vuciv 1y agoI actually think now that I've gone through the process, memory scanning and writing will be enough... Except, they probably have different control codes that I'd need to reverse engineer.
- bigyabai 1y agoYou should be able to run Cheat Engine on your emul*tor of choice to tweak New Leaf "and newer" titles. And if you're a stickler for pissing Nintendo off in very specific ways, LayeredFS + Atmosphere opens up some modding opportunities right on the console itself. Not sure how easy it would be to pull something like this off though...
- sterlind 1y agoyou really don't have to self-censor "emulator" here. HN moderation is not like social media platforms.
- lazide 1y agoSocial media platforms censor emulator?!?
- Gigachad 1y agoProbably not. But silent deranking of censored terms has everyone paranoid that anything even slightly controversial will get hidden.
- bigyabai 1y agoI'm just covering my ass. Life is good right now and I don't want to meet any Nintendo ninjas, even for insinuated infractions. It's not so much a condemnation of HN, but the way IP is in the US. The only website I want hosting my comments on Nintendo modding is my own.
- 0points 1y agoThat's hilarious. That modder who had to pay 2M sold drm circumvention kits for the Switch. That's a pretty clear case. You pretending that saying "emulator" on a forum qualifies just makes you a extra special snowflake.
- looperhacks 1y agoSwitch DRM, running custom code and interacting with RAM is a solved problem on the Switch (1). There are some really impressive mods, like a multiplayer implementation for Super Mario Odyssey
- lawlessone 1y agoI am curious. Does the kind of DRM used to prevent decompilation come with any performance costs?
- vuciv 1y agoyes, i'd like to try out Animal Crossing New Horizons next! :)
- smt88 1y agoLLMs are heavily trained on reddit, and "Tom Nook is a capitalist dictator if you think about it" is a reddit meme. Like many LLM reactions, it's less "thought" and more "regurgitating reddit jokes/opinions".
- dahcryn 1y agoIt would be great if this evolved slowly during the progression of the game lol
- Topfi 1y agoThat's essentially what it does, being prompted with a progression from: > "Stay normal unless prompted. Avoid overt references to debt unless it comes up naturally." all the way to: > "Openly agitate for change. Use fiery language (still PG) and talk about reclaiming fairness from Nook's shop and loans." Source: https://github.com/vuciv/animal-crossing-llm-mod/blob/cc9b6b571da1be062d979d50aa86e2ac1dce7a44/dialogue_prompt.py#L165-L182 https://github.com/vuciv/animal-crossing-llm-mod/blob/cc9b6b...
- ameliaquining 1y agoActually, it turns out that this particular plot arc was hardcoded by the developer and does not spontaneously arise from the LLM. https://github.com/vuciv/animal-crossing-llm-mod/blob/cc9b6b571da1be062d979d50aa86e2ac1dce7a44/dialogue_prompt.py#L143-L184 https://github.com/vuciv/animal-crossing-llm-mod/blob/cc9b6b...
- Topfi 1y agoThere is something to be said for a comment talking about "regurgitating reddit jokes/opinions", whilst falling into a major bug bear many have with Reddit, commenting before reading the source...
- smt88 1y agoI read the Github landing page. I have no interest in reading the entire source of the project, especially since I don't plan to run it.
- RajT88 1y agoI also want to overthrow Tom Nook. He traps you (and one presumes) all of the rest of the villagers in a vicious cycle of debt, and can only be satiated by millions of bells.
- deleted 1y ago[deleted]
- all2 1y agoThis sounds vaguely, achingly familiar. I can't put my finger on it, but I think I've had this experience.
- johnzim 1y agoTo be fair to Tom Nook, the interest rate on those loans is 0% and there's no risk of being foreclosed upon!
- isk517 1y agoTom Nook didn't need a down payment either.
- RajT88 1y agoBut also - you cannot turn down his loans. Pay off one, just showing up in his proximity saddles you with another one. That's more protection racket than bank loan. Also in real life, you make 0 dollars catching beetles and fish in Hawaii.
- ericzawo 1y agoCan you blame them when he has an iron grip on the whole town like a mafia crime lord but somehow even more evil.
- maxwelljoslyn 1y agoThis rules!
- amilios 1y agoThere's something about taking old games and injecting new life into them that just seems so fun and exciting! Also very interesting to know that the Animal Crossing codebase has been decompiled into readable C code. Fascinating! So many opportunities to mess with it.
- hrdwdmrbl 1y agoIt always felt both "a cheap shot" and "valid" to express dismay that characters in video games don't react when you do things like jump up and down on their table. While it's impossible for game developers to write code to cover every situation, AI could make general reactions possible. It's surprising that really simple things like this haven't been tried yet (AFAIK!). Like, even if it's just the dialogue, it can still go a long way.
- lmm 1y agoMany games have tried for more "realistic" simulated NPCs, but usually it turns out they don't make the game any more fun, quite the opposite.
- ehnto 1y agoThe cost benefit is really poor, but I also wonder if it's just never been done well. Old text adventures honestly did this heaps better than modern games do, but the reality is there was a more finite action space for them and it wasn't surprising when something wasn't catered for.
- theshackleford 1y agoWhich games are these? I’d be interested in checking them out. I’m only aware of experimentation in making more “difficult” NPC AI which was found less enjoyable for obvious reasons, so would be interested to see why similar but different attempts down another path also failed.
- anon7000 1y agoKingdom Come Deliverance 2 is one of the closer attempts, NPCs will react to lots of things about you and behavior, like if you smell bad or stare at them for too long
- dclowd9901 1y agoI was so surprised in BotW and TotK to see NPCs duck, huddle, gasp and otherwise react to odd shit you might be doing. Also in dialogue, do contextual things like talk about the weather and time. I would love to see a Zelda game implement LLM dialogue for random inconsequential dialogue or relating dialogue to the game context.
- adammenges 1y agoGreat work!! Very clever.
- nurettin 1y agoApart from the memory hacking, I also appreciate how he fully typed his python code. (as in foo: Optional[Dict[str, int]])
- vunderba 1y agoNice job! Seems like a good use-case for the random Mii avatars milling about in the Mii Plaza on the original Nintendo Wii.
- rybosome 1y agoThe idea of giving every character this sort of agency and seeing what opinion builds up about the world is incredibly fascinating. Depending on how well we assume an LLM would do at this task, it’s an interesting way to see what “real people” would think about a very hypothetical situation.
- deleted 1y ago[deleted]
- dolebirchwood 1y agoThis is awesome. LLM-powered NPCs is one thing I'm most excited about in the future of gaming. Characters repeating the same scripted dialog over and over again is one of the biggest immersion breakers.
- malfist 1y agoI used to care about repeat scripted dialogue like you, until I took an arrow to the knee
- Arisaka1 1y agoYep. I can see how relying on non-repetitive dialogue generated by LLM's will inevitably make us lose common ground scenarios that end up being memorable moments, like the arrow to the knee from Skyrim or "such devastation! this was not my intention" from Final Fantasy 14. A way to bypass this problem is to keep the important dialogue fixed, but this only one of the problems. Another issue would be emphasizing the meaninglessness of the dialogue. For example, playing Trails in the Sky has lots of NPC dialogue that's repetitive, but at least the dialogue is relevant with how the NPC's life progresses in the grander scheme of things, such as having difficulty with her entrance exams, or having an argument with his fiancé. It's not main dialogue but adds flavor for anyone who cares about the world enough to interact with the citizens. I don't think I'd like to interact with characters that I know whatever it is they have to say is generated on the fly and adds nothing other than random tidbits. The novelty would quickly wear off.
- cubefox 1y agoA better application of LLMs could be merely using them as a text parser in text adventures. NPCs could continue to have some hidden, hard coded information or abilities, here stored in the context window, and the LLM is used to provide that information to the the player he (or she) puts in roughly the right words. Rather than exactly the right words.
- maplethorpe 1y agoInstead of generating truly random dialogue, you could choose to only generate random seeds within a certain number range, so that the probability that two players seeing the same line of dialogue increases, while still having enough variety for it to feel "random" to a solo user.
- foota 1y agoI wonder if it supports Resetti :-) But also, why couldn't you look at the code to find the addresses used for dialogue? If it's already disassembled I would think you could just look at the addresses used by the relevant functions/set a breakpoint etc.,?
- Nition 1y agoI've thought for a while that the ideal old game for this kind of conversion would be Starship Titanic.
- b3lvedere 1y agoThat's a game name i haven't heard in ages. :)
- famahar 1y agoExtremely interesting use case. LLMs as a modding tool to recontexualize virtual spaces. I can see this being a tool used for artistic intervention in the same vein as plunderludic tools like Unity Hawk which allows you to run emulator save states in Unity3D. https://plunderludics.github.io/tools/unityhawk.html https://plunderludics.github.io/tools/unityhawk.html
- aledalgrande 1y agoI love this. Great skill and also hilarious!
- shomp 1y agoThis is amazing. Well done figuring out the shared memory hack. That's a huge time saver and really makes sense from an engineering perspective. Ten dollars for the hammer and five thousand for knowing where to strike the machine, to paraphrase a trope on expertise applied correctly.
- eric-burel 1y agoTechnical question: what guarantees that the memory address is stable? It means that allocations always happen in the same order in the game ?
- mschuster91 1y agoIt needs to be a global variable in C. A variable in a function scope or runtime malloc'd spot? No chance of finding a stable spot. Thankfully, a lot of old games love to use global variables because you can't run out of stack space or allocatable memory. Modern games shy away from that because even the tiniest consoles these days come with gigabytes worth of RAM, even a memory leak has to be a gigantic firehose to bring the system to a halt.
- Retr0id 1y ago> A variable in a function scope or runtime malloc'd spot? No chance of finding a stable spot. It depends, on a system with no ASLR it's potentially still deterministic, but there are a lot of potential sources of non-determinism.
- zero_shift 1y agoConsole games of that era don't usually malloc; they have globals. Entities go into pools / buffers that are allocated ahead of time
- a_t48 1y agoThis is a fairly "common" technique for doing multiplayer randomizers. Typically on a smaller region, but you end up with some sort of scratch memory being used as a buffer for messages sent back and forth between the emulator and the network layer (or your flash cart's RAM and the network layer, in the case of FXPakPro!). It's super useful.
- bryanhogan 1y agoThis is amazing! Would have loved to see more gameplay!
- adt 1y agoThe kind of thing we were doing many years ago. Here's the big one that made the rounds in Feb/2021: OpenAI GPT-3 Powered NPCs: A Must-Watch Glimpse Of The Future (Modbox) https://youtu.be/jH-6-ZIgmKY https://youtu.be/jH-6-ZIgmKY
- b3lvedere 1y agoThis is awesome! I'd love lots of screenshots with more funny dialogs. :)
- rgovostes 1y agoMany years ago there was a project in which researchers swapped localization strings to turn The Sims and Grim Fandango into language practice experiences. Generative dialog that is contextual and skill-appropriate dialog could also be a killer application for language practice.
- KeplerBoy 1y agoDuolingo has already gone all in on AI.
- momojo 1y agoThis excites me. Everyone wants to learn a new language, no one wants to slog. But I'd pay money to play Grim Fandango, and get the chance to ask "What does Morte mean?" and have the LLM, in real-time, tutor me. And if I get bored, I can game on, business as usual. But just give the AI a list of words I currently know, and it can slowly slip in more and more vocabulary, and maybe full sentences.
- brookman64k 1y ago> The game runs on a Nintendo GameCube, a 24-year-old console with a 485 MHz PowerPC processor, 24MB of RAM, and absolutely no internet connectivity. In fact, Nintendo did release an official add-on called the Broadband Adapter, which plugged into the bottom expansion port and provided an Ethernet jack. Only a handful of games supported it, one was Phantasy Star Online. I also used it to stream games/roms from a PC. This worked by exploiting a memory vulnerability in Phantasy Star Online to load arbitrary code over the network, though with slower load times compared to running from disc.
- b3lvedere 1y agoThis is mentioned in the article: "What About the GameCube Broadband Adapter? Yes, the GameCube had an official Broadband Adapter (BBA). But Animal Crossing shipped without networking primitives, sockets, or any game-layer protocol to use it. Using the BBA here would have required building a tiny networking stack and patching the game to call it. That means: hooking engine callsites, scheduling async I/O, and handling retries/timeouts, all inside a codebase that never expected the network to exist."
- LeoPanthera 1y agoI wonder if you can bitbang an ethernet interface to abuse it into being a serial port.
- mrheosuper 1y agoAt that point why not bit bang any spare GPIO ? like GPIO for 2nd player controller ?
- brookman64k 1y agoI didn't finish reading the article before commenting. Mea culpa! Maybe it would be possible to use Phantasy Star Online's network stack via the streaming exploit. But that would leave the hooking part.
- deleted 1y ago[deleted]
- loliver666 1y ago[flagged]
- ameliaquining 1y agoPresumably the author encountered it by accident and then thought it was an amusing contrast with the Animal Crossing setting.
- loliver666 1y agoYes I get it the LLM is trained on current news. But it's in very poor taste for the author to post that screenshot.
- zamadatix 1y agoIt looks like the news feed data is external, not part of training, same as the character sheet data (at least according to the diagram). This makes sense from a design perspective as then you don't need to retrain the model each time the news changes. When an unplanned result is worth qualifying as poor taste instead of humorous is a debate unlikely to be resolved on HN, but it doesn't need to be to explain why the author bothered to include it when documenting their experience.
- endymion-light 1y agoI love this project.
- VagabundoP 1y agoI love projects like this. And I love my OG Game Cube - time to dust it off from the attic and put it pride of place. LLMs in games is something I excited about. The evil subtext in Animal crossing: https://lparchive.org/Animal-Crossing/ https://lparchive.org/Animal-Crossing/
- thrance 1y agoThat's super cool! Really funny how quickly they turned into stereotypical country dwellers: mainly spending their times gossiping about each others or getting all worked-up on right-wing propaganda about places they'll never go to.
- wodenokoto 1y agoWhat is the in-game context that decides what dialogue is used? Or do they just choose at random from a dialogue bank?
- Cyphus 1y agoOutside of holidays and special events, they do indeed choose randomly from a dialogue bank. There’s actually multiple dialogue banks, one for each villager personality type.
- tehmillhouse 1y agoInteresting and well-executed. But why does the blog post read like it was written by fucking chatgpt?!
- _joel 1y agoMaybe it was, given the context of the post.
- adenta 1y agoDoesn’t the GameCube already have a network interface? Like something called phantasy star online used it for internet, and you can connect multiple GameCubes for 16 player Mario kart I believe.
- simonw 1y agoThat's mentioned half way down the article - search for "What About the GameCube Broadband Adapter?".
- simonw 1y agoHere's the code: https://github.com/vuciv/animal-crossing-llm-mod https://github.com/vuciv/animal-crossing-llm-mod I was intrigued as to how it would intercept a conversation and then pause the game for long enough for the LLM to return a response, so I used https://gitingest.com/vuciv/animal-crossing-llm-mod https://gitingest.com/vuciv/animal-crossing-llm-mod to dump the 40,000 tokens into Claude Opus 4.1 and asked it: https://claude.ai/share/66c52dc8-9ebd-4db7-8159-8f694e06b381 https://claude.ai/share/66c52dc8-9ebd-4db7-8159-8f694e06b381 The trick is the watch_dialogue() function which polls every 0.1 seconds and then answers with placeholder text: https://github.com/vuciv/animal-crossing-llm-mod/blob/cc9b6b571da1be062d979d50aa86e2ac1dce7a44/ac_parser_encoder.py#L496-L515 https://github.com/vuciv/animal-crossing-llm-mod/blob/cc9b6b... loading_text = ".<Pause [0A]>.<Pause [0A]>.<Pause [0A]><Press A><Clear Text>" write_dialogue_to_address(loading_text, addr) So the user gets a "press A to continue" button and hopefully the LLM has finished by the time they press that button.
- diamondlove 1y ago[flagged]
- snet0 1y agoDo you try trick your calculator by entering large sums that it might not have been programmed to answer?
- deleted 1y ago[deleted]
- moffkalast 1y ago
- larodi 1y agoThe memory box part is the real hackers delight here. Thanks for taking the time to share this with us:)
- klaussilveira 1y agoSomewhat related. Quake 3 bot chat with LLMs: https://www.youtube.com/watch?v=BeyvvQOPlhM https://www.youtube.com/watch?v=BeyvvQOPlhM https://github.com/jmarshall23/Quake3LLM https://github.com/jmarshall23/Quake3LLM jmarshall23 is a beast, with tons of interesting id tech-based projects.
- Razengan 1y agoThis is what I and I'm sure many other gamers/gamedevs thought when LLMs/ChatGPT first came on the scene: Dynamically-generated dialogue. Device-local models designed for games could be trained on the game's lore and given a different style for each NPC, and react to the player's random actions instead of blurting out canned responses each time. I'm sure if the genre-starting veterans like Richard Garriott or Roberta Williams were still active in the industry they'd definitely want to use something like this. Imagine a King's Quest or Monkey Island where you could literally type or try anything and get a relevant response! instead of just "nope"
- standardly 1y agoI've floated that idea around in various internet comments and am usually met with the typical AI=BAD response. It would work so well in a CRPG type game, but I think a lot of folks would review bomb a game like that just because
- rkomorn 1y agoMy main issue would be that, especially with dialog, I'd never trust whether it's something the game's writers would think is canon, correct, etc. It would probably creep into my suspension of disbelief and mess with my enjoyment.
- Razengan 1y ago> It would probably creep into my suspension of disbelief and mess with my enjoyment. You mean trying some interaction that should be plausible in a point-&-click adventure game but getting told "I can't do that" because the devs didn't think of it, or typing some slightly different grammar in a text adventure game and the parser not understanding common words, or getting the same responses over and over in an AAA+ RPG once the prewritten dialog tree runs out, does NOT creep into your suspension of disbelief?
- rkomorn 1y agoHonestly? No, it doesn't, but maybe suspension of disbelief isn't the right wording. Typically, when I play games, I build up some sort of mental model of how the game's meant to function, and how the game's creators want me to experience it. eg: running into useless/limited dialog trees with an NPC tells me it's not important. Not being able to do something tells me that thing isn't relevant (although sometimes, I guess it is fun to try something fruitless for a while). It doesn't have to be perfect but it helps if it's consistent. What I understand about programming and game design leads me to accept there are going to be limitations encountered and tradeoffs made, so they rarely bother me beyond the first time I experience it. With a more open-ended, AI-driven experience, I think I'd just end up wondering if the interactions I'm getting make sense, if they should be taken seriously, etc. I do think games have a lot of artistry (design, assets, writing, etc) behind them, and I suspect having "live" AI-generated stuff would distract me by making me wonder if I'm actually getting the right vision. It's all speculative, of course. Maybe I'd try it out and find it cool. It's certainly not something I think shouldn't be attempted. :)
- magic_hamster 1y agoOne thing I didn't quite understand from the writeup is how do you how much space you have in the memory "mailbox" for text. What is your maximum text length?
- brianf0 1y agoThis brings me such joy, thanks for sharing.
- robbingtherob 1y ago[flagged]
- tantalor 1y agoAm I missing something, or does this not react to any in-game events? So the generated text could have been done offline or using some batch process. The online generation is neat... but not necessary or important? Replacing the polling with a lookup table would significantly improve the performance, and be much simpler.
- Nursie 1y agoIn the video he mentions that he also feeds in a screenshot so that it could include live location clues.
- iam_saurabh 1y agoThis feels like the future of gaming: community-driven mods where AI brings infinite new dialogue and quests. Imagine if Nintendo leaned into this instead of fighting mods.
- deleted 1y ago[deleted]
- jd3 1y agohttps://www.youtube.com/watch?v=7AyEzA5ziE0 https://www.youtube.com/watch?v=7AyEzA5ziE0
- n1b0m 1y agoFull video: https://youtu.be/7AyEzA5ziE0?si=Ve3R7gG2KjoWCmV7 https://youtu.be/7AyEzA5ziE0?si=Ve3R7gG2KjoWCmV7
- AfterHIA 1y agoIt'd be cooler if someone hacked the game so parents could remotely/secretly play as villagers like marionettes.
- aucisson_masque 1y agoTo me LLM are one of the greatest advancement in gaming. I don’t understand how there isn’t already any big game that’s using it to have ‘live’ npc. It would make single player game so much more alive, even railroaded one like red dead redemption because the npc could adapt more easily to what you just did, how you behave in game and so on. Games are already demanding on gpu, running a very tiny local LLM and only when people interact with npc wouldn’t require so much power. I’m sure there are issues which explain why it’s not already being used but the first one to do it will be remembered.
- netruk44 1y agoI wanted the same, so I hacked/bolted on an LLM to Morrowind (the open source recreation OpenMW). The biggest problem I faced at the time (during ChatGPT 3 era) was that, without a good context, LLMs are the most vanilla roleplayers you’ve ever seen. By themselves, LLMs are just not interesting enough for a player to choose to talk to in-game. If you want them to be “interesting” to talk to, you must provide (or generate and keep track of): a backstory, chat history, the scene, NPC inventory, the NPC’s current emotional state, the weather, literally everything needs to be given to the model before it generates messages for the player. At which point you’ve got a big task. You need a way to automatically get the relevant data to the model for the specific conversation you’re having. There might be tools to pick appropriate text documents from a db given a conversation topic, but I didn’t/don’t know how to make that work for games. I’m sure there’s a way to accomplish this with more modern tools and models. (Maybe instead of providing all that data up front, you would now give the model tools to call to retrieve that data on-demand?) But that’s what made me give up in 2022.
- aucisson_masque 1y agoOf course it would require the game engine to provide context to the LLM, it certainly is quite a bit of work but nothing technically impossible and the result are endless possibilities and endless replayabilities. You could very well let the LLM decide if an npc should agree to give an object to the player or not, even let the player try to convince the npc. I mean it looks like to me the next big step in gaming after 3D and yet this is being ignored.