9 ms·
Ask HN: Game devs, what is the hardest part about designing an AI for your game?
I've been thinking a lot about different machine learning techniques applied to video game artificial intelligence. I'm wondering if any video game devs would like to share their experience about what parts of making a game AI are challenging, things that are difficult to get right, time consuming, or tedious.
Thanks!
- slimsag 8y agoI've been thinking about this a lot, too! Honestly, I think the most challenging factor is that when you are working on machine learning for your game.... you aren't really working on your game. i.e., you're not writing storylines, creating visuals, audio, programming general interaction, etc. Machine learning (if you don't have a ton of experience) can be super time consuming and unless your game is 100% based around the sole idea that it is ML.. all of that time is taken away from the rest of your game.
- sgillen 8y agoYes I agree, I think the idea here is to provide applied machine learning as a service to game devs.
- sjg007 8y agoMy guess is you want it challenging but not too hard.
- springogeek 8y agoThe risk of overcomplicating. You can put a lot of effort into detailed game AI which just makes for a hugely unfun experience. Something simple or predictable often leads to be better gameplay.
- mattnewport 8y agoI've never specialized in AI but from my experience on game teams most of the challenges revolve around making the AI 'fun' to play against, which is not generally very well defined. It's also about taking feedback from designers and playtesters and figuring out how to adjust the AI to address that feedback. Machine learning techniques aren't really used much in game AI, partly due to these aspects. It's not necessarily obvious how to adjust some black box ML model in response to feedback from playtesters on balance issues or from designers on making the AI more or less 'agressive' or other qualitative feedback. Usually it's more a collection of rules and heuristics with various parameters that can be adjusted to tune the behavior and allow for different difficulty levels etc.
- Jasper_ 8y agoGame AI is mostly pathfinding (which basically falls down to creating navigation meshes for levels), and very basic decision making skills, which is usually a weighted decision tree (e.g. if they're getting shot at, cover might be weighted at 80%, shooting back at 20%). You don't want to actually create something actually smart to fight, you want something with understandable, predictable logic for the player and something they feel clever outsmarting. Anything too complex and players will get discouraged by the difficulty. Machine learning isn't really a great fit for this, as it is neither predictable nor understandable, and thus the player can't plan around it or try to outsmart it.
- xg15 8y agoI'm curious about that. Keeping it dumb of course makes sense for adversarial uses - if the AI is you enemy. But I could imagine there are scenarios where the AI is on your side - e.g., as a support player, NPC or pet. Would a smarter approach be more useful in such a game?
- nkrisc 8y agoEven if the AI is friendly being able to predict their behavior is still desirable. Nothing worse than friendly AI ruining your game because it went off and did something you couldn't anticipate.
- c0nducktr 8y agoWhat if the friendly AI called out it's plan, and the player then could have an agree/disagree option with the proposed strategy? If done well, it could feel much more like you're playing with a human partner.
- nkrisc 8y agoThen, sure, because it is now predictable. Alternatively, I don't think it would be an issue if the AI was good enough that it could do unpredictable things that were actually intelligent (which would pleasantly surprise the player) or if the unpredictability and occasional awfulness of the AI were somehow an intentional challenge of the game.
- azhenley 8y agoMaking an AI that isn't "perfect", but rather it is fun and challenging to play against. My friend was working on a chess variant for years, and it was very easy for him to make the AI be very, very difficult but that isn't fun at all! It would always win. He then tried adding an element of randomness to it to dumb it down. But then the AI just looked perfect most of the time while occasionally making bizarrely dumb moves. He later added "personalities" to the AI, where each one would try their own sets of strategies and attributes. He continued to make it better and better.
- duggalr2 8y agoI'm curious if your friend's project is open-source at all, would love to see the implementation details of this?
- azhenley 8y agoIt isn't open source, but you can still play it if you have Flash installed: http://www.kongregate.com/games/thegrandestine/chess-evolved-online http://www.kongregate.com/games/thegrandestine/chess-evolved... Imagine chess but with 100+ different pieces and an element of collecting/upgrading pieces.
- duggalr2 8y agoCool, thanks!
- xg15 8y ago> My friend was working on a chess variant for years, and it was very easy for him to make the AI be very, very good but that isn't fun at all! It would always win. Oh yeah, I can relate to that experience. My chess app had an "undo turn" button, where you could jump back an arbitrary number of turns. The most frustrating thing wasn't the AI winning, but realising that the AI had already won several turns ago and you're just a dead man walking, with all remaining options leading to failure. I never hated A* as much as during those games.
- 8y ago
- jamesgeck0 8y agoIIRC, the developers of Final Fantasy XII said in an interview that the Gambit system that the player uses to set the behavior of their computer controlled party members is basically just a GUI representing the same logical structure used behind the scenes for enemy behavior.
- thedevindevops 8y agoI love that Gambit system and wish more games would adopt similar systems. The Rules based approach allows complex behavior to emerge from a relatively simple set of rules. Although this is a per-unit approach, in combination with an overarching management AI, this could be very engaging.
- adamnemecek 8y agoA friend is working on a solution for this. Check it out https://yuri.ai https://yuri.ai
- coolreader18 8y agoWhy is it named Yuri, if I may ask?
- adamnemecek 8y agoAfter Yuri from command and conquer.
- dejaime 8y agoLooks cool, if not overkill for the vast majority of games. I hope I remember this if I ever get to make a moba or a hardcore strategy game.
- setr 8y agoOne big issue I can imagine with applying ML is that most games are buggy and poorly balanced, but the majority of your players arent optimizing machines (outside of speedrunners), and will follow “natural” paths, so its fine. The only kinds of games that see enough time and resources to the same issues that would trap an AI in some optimal but horribly boring strategy (or rather, “unfair” strategy) would be competitive multiplayer games — and even then, only those that can classify for e-sports. That is, those games who already treat players (or rather, the wetware community optimization machine) the same as they would treat your ML AI.
- deleted 8y ago[deleted]
- cpeterso 8y agoGDC (Game Developers Conference) publishes a lot of great talks on YouTube about game design. Here are some of their popular AI talks: https://www.youtube.com/results?search_query=GDC+AI https://www.youtube.com/results?search_query=GDC+AI
- ggggtez 8y agoWatch the talk about Civilization's AI. Seriously, game AI does not need to use advanced ML techniques, and probably shouldn't if you want players to have fun. The trick is coming up with fast algorithms, with a semblance of personality to make it interesting. Difficulty is not required for the AI to be fun in most cases.
- bryondowd 8y agoWhat talk is that? Last time I tried a Civ game, I was underwhelmed. That AI just isn't fun to play against, to me. And the games run too long to play against humans in a reasonable timeframe on normal settings. I'd love to play Civ with competitive AIs.
- ggggtez 8y ago"Playing to Lose" : https://www.youtube.com/watch?v=IJcuQQ1eWWI https://www.youtube.com/watch?v=IJcuQQ1eWWI Personal preferences aside, they specifically talk about how to make the game "fun". For example, they try to account for things like real-world leader's personalities. "Gandhi" shouldn't just attack his neighbors unprovoked, but maybe "Caesar" should. And why do the AIs even bother to ally/trade/help the player at all, since the player is the only enemy that really matters anyway? Because the player is also the only person that matters in terms of having fun, and making the game too hard isn't any fun.
- bryondowd 8y agoInteresting talk, but it really pretty much confirmed a lot of my assumptions about Civ AI. He pretty much admits that they are constrained on developer resources and AI code only makes of a small percentage of the game. Because of that, they have to make do with very naive assumptions about strategy. And because of that, they have to cheat to get the AI up to a level that's even considered fun. At first I thought he seemed to be making a lot of false dichotomies, but really, he was saying, these are my only choices, given the overarching constraint on how much effort we can put into our AI. Basically, all of the AI decisions he talked about were small scale heuristics, with no ability to really look at the bigger picture, which is what the player is always doing when making small decisions. And of course, personalities are a great idea, and not mutually exclusive with a strong AI. One thing I very much disagreed with was when he said an AI can roleplay, but a player wouldn't do that, the player just plays to win at all costs. But earlier on he even talked about one of the player types is people who play for a narrative. Those are roleplayers. So, especially at lower difficulty levels, it makes a lot of sense for the AI personalities to trump optimal play. But that doesn't mean you can't have a strong AI in general, that makes some limited sacrifices for the sake of fun. Likewise with diplomacy. It isn't about making the game as difficult as possible. If that were the goal, you could just make the player lose when he loaded the game. It's about making it feel like you're on roughly even footing with your opponents, so that success has meaning. To that end, I would expect each empire to act in its own best interest, and therefore treat the player the same as it would any other opponent. It's also why diplomatic options need game mechanics to enforce them. As he pointed out, if there's no mechanic to stop a player from accepting money for peace, and then declaring war again, then there is no reason to every pay for peace. You should design these mechanics with player behavior in mind, and the AI should use them the same way players do. Think about how players behave with one another, make that fun, and then make the AI behave similarly. This wasn't possible given their constraints, but it's certainly possible in the more general sense. As far as "making the game too hard isn't any fun" that's the problem. Too hard is subjective. What's too hard for one person is perfect for another, and way too easy for yet another. If anything, to please the maximum number of people, you'd have to write (or train) a very skilled AI that can compete with the brightest players. If you can do that, it should be much easier to tone it down or penalize it to make easier difficulties for the average and casual players.
- dleslie 8y agoF.E.A.R. remains at the top of its class for the feel of the NPC behaviour, and it's worth understanding why. https://aiandgames.com/facing-your-fear/ https://aiandgames.com/facing-your-fear/
- z3t4 8y agoWhat I understand from machine learning is that the actual learning takes a lot of compute power and learning data. So you can't have and AI that adapts during gameplay. The AI will "just know the optimal move for any given play state.
- bryondowd 8y agoIf your game doesn't give competitors perfect knowledge of the game state, it would make sense for the AI to track previous activity of its current opponent as part of the 'play state' for making decisions. For instance, if it doesn't know where the enemy is, but the enemy has attacked from this direction every time so far, it would prepare for another attack from that direction. Or if the last time the AI made a given offensive move, the player countered effectively a certain way, it may modify its next offensive to overcome that counter. The longer the game, the more it could appear to be learning to counter its opponent, even though, as you say, it's just making the optimal move given the current state of play.
- krm01 8y agoI’ve been designing/coding games since I was 11yo and have a degree in gamedesign & dev. The hardest part isn’t the technical side of the game’s AI. It’s finding the balance of an AI that makes the gamer feel powerful enough to be fun- but not too powerful to become bored - and not find it too hard that he/she becomes demotivated. Obviously every player is different so finding a balance that can keep the gamer excited enough to continue is extremely hard. Getting the emotional-experience side of the player into the sweet zone is hard. One would say ‘why not make the AI such that it adapts to the user’. Which is 50% of the answer. But the challenge then becomes that you don’t want good players to be at the same level as bad players when the AI creates a similar experience for both. You want both players to enjoy the game, but not in the same way.
- Anil_Uttani 8y agoThe Solution here would be rather than weighing the AI Difficulty for good players and bad players.You must have different sets of AI.
- mrfusion 8y agoI’d love to do this as a career or sell Premade AI modules for games. Any ideas? Would game developers consider contracting out the AI to a freelancer?
- tareqak 8y agoI actually think there would be value in having machine-learning / AI in a game that would dynamically try to match the human player's skill without overwhelming the human player. The goal would be to maintain parity such that the human's player response would always be "I was this close, and I could have done X, Y, and Z" as opposed to "I had no chance".
- bladedtoys 8y agoLike others have said, balancing play-ability is hard. Another problem is users may perceive the AI cheats which hurts fun. But I'd say another hard thing is that every game needs a different approach to AI. Some need path finding, which is well documented. Others need resource balancing which is less so. The newer and rarer the game mechanic, the more novel the AI. I would say Machine Learning is not useful except in a handful of games and if yours is not in that category, you risk spending a lot of time for little pay off. And even if in those categories, ML will be working to win not to maximize fun so they could end up being too good or to cheaty looking.
- ionwake 8y agoStopping it from making a base an Antarctica which players can’t invade due to a bug
- formalsystem 8y agoI think any game with strategy is going to have to use reinforcement learning to have any hopes of being understood by its players. Concerns around cheesing strategies can be mitigated for real time games using constraints and cheesing in turn based strategy games is a symptom that maybe the game you're working on isn't that strategic. Ultimately Chess, Go, Dota, Starcraft are compelling games to watch and play because the skill cap is so high. No hand coded AI for such a game could ever prepare you to enjoy playing any of those games professionally or get to a point where you can understand professional play. High level strategy in those games would never even be discovered had we not had a large population of players playing those games for long hours. Odds are most strategy games won't be as successful as any of the above games but the newcomers could still have compelling strategic elements that gamers could explore had they been able to find other players that are slightly better than they are so they learn but at the same time aren't frustrated by too big a skill gap. A small multiplayer community is why the strategy game market is pretty bleak except for a few large players. Reinforcement learning can provide a better experience to your players because you the game developer don't need be the world expert at the strategies of your game to create a top level AI. You just make sure the rules of the game are there and self play will take care of the rest. You can constrain the AI such that it's enjoyable by both your newest and most devoted players. You can discover OP strategies and better guide your design process. Excel is great for balancing but if you want to make a game complex enough then balancing will be a tremendous challenge without a larger team. If you're interested in leveraging reinforcement learning for your game, I run yuri.ai which helps game developers do exactly that. So reach out to me at mark@yuri.ai if I can help!
- sgillen 8y agoThanks for your response, I had actually heard about Yuri before and was hoping you would respond to this post! I have a lot of questions for you if you don't mind, maybe you don't want to answer any/all of them that's fine. 1. How do you handle constraining the AI such that it's enjoyable? If you read through the comments here you'll notice just about everyone is saying the hard part is not making a good AI but in making a fun AI. Do you know of any research in this area? most examples I see are all about making the best AI possible. It seems like it's really hard to find a good reward signal to use for "fun". Have you seen this [0,1] paper on learning from human preferences? does Yuri attempt to do something similar? It seems like it would be labor intensive for the play testers. 2. Does Yuri make use of imitation learning at all? I.E learning from lots of human data when it's available to bootstrap the learning process? 3. Do you let the game designers impose any structure on the AI? I.E. "I want three phases for the boss where he progressively gets more aggressive" or use RL just for certain small parts of the behavior? A well defined path finding algorithm with the RL deciding where the agent tries to go? stuff like that? I have a ton of questions about the technical details side of Yuri too, what frameworks are you using, which RL algorithms etc etc, not sure if you are willing to share those or not. [0] https://arxiv.org/abs/1706.03741 https://arxiv.org/abs/1706.03741 [1] https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum...
- cableshaft 8y agoNo matter how dumb you make the A.I., you will have players complaining that it's too hard or unfair. I literally had the computer make a list of all possible moves and pick one out of the list at random for the default A.I. of one of my games and I got accusations that the computer was cheating on that difficulty level on a fairly regular basis.
- expopinions 8y agoNone of those (e.g. lighting, physics, graphics) are truly difficult today with the advent of modern game tools (e.g Unity). Assuming you know basic vector math and can think in 3 dimensions, most problems related to sound, physics, graphics etc can be solved by some Googling.
- remoteworkerJ 8y ago"Working overtime in office and not getting paid in time" No technical challenge is above this
- alexsandro 8y agoFor me the hardest part was planning and creating of starting design. It is easy when you have basic code and design but not in any other case
- ezekg 8y agoI always thought it would be an interesting to write an AI for a shooter like Call of Duty or Halo, which is trained on real player data, according to your skill level. Most FPS games already record match data, so the dataset is available. What might happen is that the AI might evolve as the player meta evolves, and would learn what types of maneuvers work in certain situations, it would know average accuracy for a skill level to adjust itself, etc. It would also allow the game to fill in low player population, e.g. during a low traffic time or for a player with a ping that is too high, with bots that act human-like. Maybe wishful thinking, but I always thought it was an interesting thought.
- danbolt 8y agoI programmed some new API functions for the AI scripting language in a popular RTS game once. I think the most tedious part is exposing the game data in an intuitive/helpful way for someone in a design role. It's a bit of an odd mix of optimization, computational geometry, and finding the right level of expressiveness. For instance, a designer might say "It'd be great if I could put a tower between myself and the enemy town, but kind of at the border of mine, but only if I don't have enough money for a cool castle and I'm not overwhelmed." There's a lot of fuzzy logic in statements like those. The API between the game and the "AI" usually has to be general enough not to need spatial information, but also specific enough not to do things that feel unintuitive or silly to the player. It's worth noting that machine learning helps avoid a lot of these issues, but I find AI tends to be designed for an feeling or experience, so I'm not sure if things like the Starcraft AI competition bots tend to make better products. Perhaps if ML could make the building blocks for designers to design AI?
- EtherDynamics 8y agoMixing Concepts The biggest surprise I have found on this subject is this strange entanglement of ideas between "AI designed to kill the player" ("Military" AI), and "AI designed to make the game fun ("Game" AI). This is like comparing a F-22 stealth fighter to a 747 commercial airliner -- sure, both are technically "jets", but they serve completely different functions. The only AI we should be talking about here is AI designed to make a game more engaging and enjoyable for the player. That may include emulating a tiny bit of behavior that resembles military tactics, but only where games use that as a topical facade -- unless you play The Sims waaaay differently than I do. Easy Mode There are several posts here where people assert that "they don't want a game to be hard, that would be bad". As a counterexample, I would invite these individuals to view the awards, testimonials, and revenue from the ["Souls"](https://en.wikipedia.org/wiki/Souls_(series) https://en.wikipedia.org/wiki/Souls_(series)) series (Demon's Souls / Dark Souls). I have spoken to psychology experts at game development conferences, as well as professors that explicitly study psychology in games -- and they are still struggling to define and quantify the exact elements that came together to make this series such a hit. But there is no denying that it is an absolute favorite, with a huge following, and tremendous profits to prove the efficacy of the model. Obviously, I'm not advocating that "every game should have a Souls-style difficulty curve". That series caters to a specific audience -- and there are an ocean of subtle differences between fans of the series. The point is: broad statements like that go directly against a mountain of evidence. We need to think about this issue with its full (tremendous!) complexity, and how to leverage some immense computing power to address it. Speaking of which... Based on your Purchase History... Right now, companies like Amazon and Google are mashing up the entire history of everything you've purchased, every web page you've ever visited, along with demographic and social media data about you. They can then deliver a product recommendation / advertisement / etc. within milliseconds that targets your specific needs, even for things you didn't realize you wanted / needed yet. If that can be done in milliseconds, then the same principles can be applied to tweaking the difficulty for players in real time. You don't even need to maintain a hot internet connection to constantly feed some huge cloud implementation at all times -- some local processing combined with daily or weekly micro-updates can do the trick. For a perfect example of local / cloud balance, check out SnapChat -- that thing uses the local computing power on your phone to identify your face, and then attach various 3D objects and effects to it in real time. That's done at 20-30 FPS on a smartphone without any special dots or markers to help "anchor" the models. That's amazing. And it's all possible because the [Convolutional Neural Network](https://en.wikipedia.org/wiki/Convolutional_neural_network https://en.wikipedia.org/wiki/Convolutional_neural_network) models that make that possible get trained on the cloud and downloaded every now and then. So, yes -- you can code a game to dynamically customize itself for every single player. This is the [exact point of my video series](https://www.youtube.com/watch?v=hTDSGU2vAow https://www.youtube.com/watch?v=hTDSGU2vAow) about this subject. Thus far I've made over a half-dozen Skyrim mods that demonstrate some of these concepts, and our team is expanding into other games that are finally opening up their engines to 3rd parties. If folks would like to talk about the nitty-gritty of all that, I'm happy to chat.