11 ms·
Street Fighter II's AI Engine
- radarsat1 10y agoThat's pretty fun, thanks for sharing ;) Actually with a machine learning approach it's not immediately obvious to me how to incorporate it. I suppose an RNN or reinforcement learning could be used, in principle, to learn good reactions to a given sequence from the opponent, but it would take so many failures to train it, which would have to be generated more or less manually. I don't see how it could be done easily, and certainly not in an "online" way without delivering a pretty terrible play experience. Are there any good examples of computer-controlled players using an online (or offline) machine learning-based approach to player control, that doesn't suffer from these problems? The image that comes to mind for me is a computer player that gets better and better at beating you, but the problem is that with reinforcement learning there is no guarantee that this happens reliably and in an amount of time and gradient that would provide a good playing experience -- and then you'd be faced with the problem of it getting "too good", equally not a particularly good play experience. Perhaps something in between? A machine learning approach that helps select and maybe parameterize these "scripts"?
- NikolaeVarius 10y agoThey're fighting games. Why do we need to incorporate machine learning at all? If the computer wanted to they could always play perfectly because it could react to any input perfectly.
- viewer5 10y ago'learning to fight against you, sort of like a human would' is a lot more fun than 'literally cheating and perfectly countering you'
- eyko 10y agoSince humans are not infallible, let's leave it at 'learning to fight against you, sort of like a superhuman would'. It would make for a formidable opponent since you can still defeat it (with a good dose of luck).
- wmil 10y agoIf you forced the AI to work with a substantial input lag, it could be quite complex yet still possible to defeat.
- ilitirit 10y agoAI's are useful as training dummies. I believe Killer Instinct has quite a good AI called "Shadow Mode" that learns from your behaviour. http://www.polygon.com/2015/5/30/8691219/killer-instinct-teabag-yourself-video http://www.polygon.com/2015/5/30/8691219/killer-instinct-tea... It probably uses a hybrid or AI and scripting techniques. As for things like human reaction limits etc, that can built into the AI as part of parameters it needs to operate within.
- Kurtz79 10y agoWell, that's the challenge, right ? Not making an AI that has perfect response, but a "human-like" one.
- FroshKiller 10y agoThis is just me bullshitting, but I imagine you'd want to continue shipping a simple interpreter and some kind of script for the computer opponents and use a hypothetical learning system to generate the scripts separate from the game you ship. The AI code in the game would be simpler, the scripts could be easily patched, users could generate their own scripts, etc. I think you could probably generate and hand-tune a number of situational scripts and let a learning system decide how best to chain them for particular circumstances (e.g. if the opponent is Character A, the computer is Character B, the computer's relative health is much lower, and Character A has a full super combo gauge, then it learns a particular sequence of scripts that proves effective in replays).
- 60654 10y agoThere have been some attempts to apply ML to NPC control: sometimes to learn how to beat the player, sometimes how to mimic them (Forza), sometimes to learn what player wants done and assist them (B&W). But none of them have crossed the chasm from novelty to killer app. The main difficulty with combat AI is that players enjoy mastering a game and beating it. And if the game keeps getting better and beating them instead, they will end up unhappy.
- radarsat1 10y agoYeah I think that's a very interesting aspect. If you want, I think this speaks to a more general point about AI that I have thought about sometimes. I suspect that we enjoy interacting with machines and using machines when we understand them, and that this understanding of how a machine operates, its capacity and its behaviour, is similar to the "theory of mind" idea that we employ a model of another person when interacting with them. That is why it is so frustrating to work with a machine that changes as you use it -- it's like trying to shoot a moving target. I really suspect that ML will find its place in helping design machines that are then static, but ML that adapts as we use it will be frustrating to work with. At least until we develop, as a society, some kind of "theory of mind" of AI, where we learn how to expect a machine to adapt as we use it. But we are not there yet. For now, machines are still better when they act like machines. That is why I think, for example, society adapting to the presence of things like ubiquity of self-driving cars is going to be a long and scary road. As for video games, it seems like a computer-controlled character should be somewhat predictable, even if complicated. When the ML algorithms adapt, it should present a new character to fight, not adapt an existing one that the player has already "theorized" (i.e. figured out its pattern.)
- meheleventyone 10y agoColin McRae Rally 2.0 was released back in 2000 and used offline learning to train a driving model. Here's an interview with the engineer that made it: http://www.ai-junkie.com/misc/hannan/hannan.html http://www.ai-junkie.com/misc/hannan/hannan.html In this modern day and age grabbing telemetry from multiplayer matches might be a good way of getting training data. Particularly for games that already measure player skill. For example creating AI for Hearthstone that played like each of the skill tiers. Periodic retraining on up to date data could then keep up with things like changes in the metagame.
- lloeki 10y agoThis is how Drivatar works in Forza Motorsport. It was first trained and tuned offline internally at Turn 10 to give opponents personality traits and driving style (Forza 2, ah, that damn M. Rossi!), then trained from the player as well and used offline (IIRC Forza 3 had AI progress vanish after the race while 4 persisted it across races. Anyway I loved how the opponents would ostensibly start to use my own tricks and lines against me, making each lap harder than the previous one), and finally starting with Forza 5 and 6 you could have your own AI that would play as you online with your own driving style while you were away, and if it was winning, you'd get some cash.
- ivan_ah 10y agoI wonder if knowing the different strategies would make pro-humans better at playing the game. I feel like they might already have internalized the different strategies the computer uses, but still pretty surprising to see how simple the source code is. Also cool to think about would be AI-vs-AI street fighter competitions. Or SF AI that learns from live matches currently being played online.
- coldpie 10y ago> Also cool to think about would be AI-vs-AI street fighter competitions. That's a fun idea. Provide an editor for the AI script files so you can program your own AI, then inject that into the ROM and pit AIs against each other.
- ilitirit 10y agoYou can do it using the techniques employed here: https://github.com/aikorea/strikersii_ai/tree/master/genetic https://github.com/aikorea/strikersii_ai/tree/master/genetic Demo: https://www.youtube.com/watch?v=k6Ir8yd9iOk https://www.youtube.com/watch?v=k6Ir8yd9iOk In terms of FGs, you would store the relevant game state as inputs and build a reward function around health, time, damage done etc. A relatively simple (and useful!) SFII AI would be one that tries to win by only using crouching medium kick, sweep and throw.
- coldpie 10y agoI just meant straight up using the scripting language described in the article to hand-code an AI.
- ilitirit 10y agoAah in that case if you're willing to forego the limitation of using the original AI instruction scheme, you can use the technique I recently used to build a training dummy. It's just a state machine that reacts differently depending on the state of various game variables (the same idea used by the original SFII code). The scripting language is Lua, and it's basically lines of code that go: if (distanceBetweenChars() < THRESHOLD) then chooseAppropriateMove(distanceBetweenChars()) end etc... Here's a demo of one that reads the distance between characters and either dashes in and does a low attack, or attempts to throw you. https://www.youtube.com/watch?v=R9THKU63viM https://www.youtube.com/watch?v=R9THKU63viM It's pretty simple to have this script run for Player 1 and Player 2. In a competitive environment (ie. when you have AI vs AI) the players involved would need to agree on certain limitations, otherwise it would be pretty simple to build an AI that reacts "instantly" to stimuli.
- kensai 10y agoI wonder if one can create an invincible AI for Street Fighter II. One that obviously makes the right choice always and can counter every possible human attack. (kind of think that it was also possible even back in the 90s, but never implemented; what would have been the point?)
- emodendroket 10y agoPeople have kind of tried this with Mugen, I think.
- IWillScoop 10y agoYep. SpriteClub.tv, a mugen stream, showcases a bunch of these kinds of characters with hilarious results.
- kstenerud 10y agoMortal Kombat II effectively did this with really shitty AI. If you jumped towards an opponent, they would jump straight up and hit you with a projectile with perfect accuracy. Every time. It was impossible to throw a computer opponent because the AI had better timing then you. Every time. In fact, the only way to beat the computer opponent was to take advantage of weaknesses in the AI script, the biggest one of which is jumping backwards when there's a specific distance between you. The computer would jump towards you, leaving them open to you jumping forwards with a kick. Every time. Just don't get caught in the corner.
- zerr 10y agoExactly. That's why I've never enjoyed playing MK with a computer. Game AI should be about fun and enjoyment for the player.
- draw_down 10y agoThey also would let you leg sweep a lot more than a normal player would. I'd win whole rounds with just leg sweeps. Not the ideal kind of "fun" when playing a video game, but neither is getting perfectly, precisely owned every single time you jump.
- snickmy 10y agoAI != IF STATEMENT + RANDOM
- ghusbands 10y agoYou're confusing GAI and AI. AI covers anything that's making a system seem intelligent, essentially. Expert systems and game AIs are both very condition-based.
- luhn 10y agoSure it is. AI is a computer reacting to its environment to solve a problem. It doesn't necessitate machine learning, neural networks, or what-have-you.
- nischay 10y agoseems like it is down, 502 error https://usafacts.org/ https://usafacts.org/
- redm 10y ago> Anyone looking for some insight into how to write an AI engine for a game today will be disappointed. And yet, it was really effective and fun. This is like a magician revealing how a trick was done, it's always a "letdown".
- EGreg 10y agoWell you are in luck because in a major sense, deep learning AI cannot reveal how the trick was done unless you are willing to speak in terms of a model with trillions of variables and feedback loops.
- indspenceable 10y ago> These days, when most people talk about AI they’re talking about machine learning. There’s not any of that in SF2. Actually, AI for games is pretty much never equivalent to AI for non-games. The end goal is different - games want to provide a non-optimal set of instructions, so that it's challenging but not impossible to win. The goals are entirely different. If you're making a game, this is probably at least a useful example to look at, even if I don't agree with some of the decisions they made (uninterruptible moves, for instance)
- stcredzero 10y ago(uninterruptible moves, for instance) I bet this was done to decrease the complexity of the script flow. Otherwise, you'd have to provide for the interruption cases.
- psykotic 10y agoI asked the blog's author and he said hits do interrupt scripts. Aside from that, more complex scripts intended for higher difficulty have explicit conditional branches after moves to allow for blocking and counters. The triple fireball script example is for "easy Ryu", so it's intentionally dumb/non-reactive.
- echan00 10y agoCompletely agree. Games are typically only "fun" when you feel challenged. Most people do not find "fun" in losing.
- vvanders 10y agoIt depends. With a well balanced fighting game I don't mind losing to a good human opponent because it means that my game improves. Playing against someone who's good is a great way to find out what moves/patterns you thought might be safe aren't. I agree that if you have no way to measure that progress/improvement then the fun is lost.
- pmarreck 10y ago
- CocaKoala 10y ago"Charge moves such as blade kicks are simply executed as instructions, so they cannot fail. Guile can do a bladekick from a standing position simply because that’s what’s in the script." That's a pretty big cheat, to be honest.
- ben174 10y agoWelcome to the frustrations of my childhood. I sort of neat trick is during a dump you can be holding down to charge your "blade kick"* and immediately when you hit the ground, press up and kick to execute the move, and it gives the appearance of one of performing the move without charging. Sometimes gets a surprised look by your opponent. * we called it a flash kick, but whatever.
- faragon 10y agoGaming AI must be "beatable". "Good" AI is not good for having fun games.
- RealNeatoDude 10y agoIt's funny how much SF slang hints at you and your opponent being lackluster AI scripts: - Downloaded: when you've identified your opponents patterns and know what they're going to do next. Ex. they always jump after they jab. - Conditioning: when you've trained your opponent to react the same way consistently to something you do. Ex. they always jump when you throw a fireball because you've let them do it successfully. - "time to guess": situations where, if executed correctly, your opponent must choose one defense from many defenses randomly. Ex. your jump trajectory is such that whether you end up on the left or right side of your opponent is determined by a pixel. A phrase that novice players say a lot is "HOW DID HE KNOW?" because they're in shock that their opponent is "guessing" right every time when really their opponent just knows what the novice is going to do in every situation. What's impressive is that "every situation" is really the more skilled player abstracting over similar situations. Ex. they recognize kick->jump = punch->jump for their particular opponent. I don't think an AI will be able to make complicated abstractions like that on an offline game console for a while.
- vvanders 10y agoYeah, Sirlin covers that really well in his Layers of Yomi on Playing to Win: http://www.sirlin.net/ptw-book/7-spies-of-the-mind http://www.sirlin.net/ptw-book/7-spies-of-the-mind
- billmalarky 10y ago"When reacting to an attack the scripts are chosen based on something called a yoke. Each frame of animation for both avatars and projectiles contains a value for the yoke in the metadata, which the AI peeks at to select a script suitable for responding to that attack. The computer sees the yoke of your move as soon as you have input it, before the first animation frame has even displayed. As such it gets one more frame of advantage on top of your reaction time." I KNEW IT!
- the_denial 10y agoWhy aren't there longer bytecode instructions?
- sf2platinum 10y ago1. Because it's not necessary, there are a lot of them instead, so it runs through a short script, picks another one at random, rinse, lather, repeat. 2. Longer scripts would be recognisable and easy to beat.
- c0achmcguirk 10y agoI'm going to find Vega's script and replace it with NOOPs. I hated playing Vega. He would climb that wall like a spider and then jump with a shrill yell and then body-slam me. I think that's the most rage I've ever experienced in my life....just repeatedly getting beat by Vega. The only thing that comes close is getting hit by the blue shell in Mario Kart.
- madshiva 10y agolol, vega is easy to beat come on! There's a lot of trick in SF2, with almost every character. Thanks for the info, I'm currently building my SF2 game and was wondering how they made their AI, then much thanks for the sharing.
- sf2platinum 10y agoWhen he climbs on the fence, get as far away from him as possible. When he jumps off, jump away and hit roundhouse, this usually beats him.
- michaelbuddy 10y agoLosing is not fun but almost winning IS fun. Progress through refinement, strategy and speed is fun.