13 ms·
Pac-Man recreated with a GAN trained on 50k game episodes
- ginko 7y agoSo how large is the generated model compared to the Pac-Man binary? (edit: which was 16KB)
- Kenji 7y agoI'd love to see more gameplay. In fact, I'd love to play it myself. I just can't believe it replicates the original gameplay exactly.
- caust1c 7y agoFor those confused, this is just a recreation of an animated image of pacman, not a playable recreation of the actual game. Impressive still, but not exactly what's lead to be believed by the title.
- CShorten 7y agoI made a video explaining the details of this paper if you are interested! https://youtu.be/H8F6J7mYyz0 https://youtu.be/H8F6J7mYyz0
- zeroxfe 7y agoThanks for doing this. I stumbled upon one of your previous videos a while ago and really liked it. I'm super indebted to YouTubers like yourself that take the time to distill and explain complex topics, papers, and new research.
- CShorten 7y agoThank you so much!
- bjourne 7y agoVery interesting video! How does the GameGAN image discriminator work? I don't know so much about GANs but it seem to me that classifying PacMan images as real or fake would be a very hard problem because of the low resolution. Suppose PacMan is in a corridor and there is a dot to the left of him and three to the right: ".P..." That is plausible because PacMan hasn't eaten any dots yet. But if the second dot to the right is missing: ".P._." something is wrong because the first dot to the right is still there and PacMan can't jump over dots. It must be very hard to get the discriminator to understand that the second image is fake.
- CShorten 7y agoThanks for watching bjourne! There are three different discriminators, the first does realism for the game, the second does action-conditioned discrimination (making sure the generator takes in the action for its next frame prediction), and the third is a temporal realism with a 3D CNN. Interesting scenario, it would be a cool experiment to walk through the latent space and try to find that kind of frame in the GAN latent space. I'm sure you could find it, but the question might be how far off the generated data's manifold is it.
- deleted 7y ago[deleted]
- itchyjunk 7y agoThought experiment: Is my brain also generating reality without a physics engine and is filling in details that it thinks should be there? If true, what are the advantages/disadvantages of it?
- coldpie 7y agoYes, absolutely. You can catch a baseball because your brain predicts where it will go before your eyes see its actual movement. This may be a good place to start learning more: https://en.wikipedia.org/wiki/Direct_and_indirect_realism https://en.wikipedia.org/wiki/Direct_and_indirect_realism
- lonelappde 7y agoHow is that "not a physics engine"?
- jcims 7y agoIt's more general than a physics engine in that it's predicting an expected outcome...it just so happens that the outcome here is largely dictated by physics. (At some level they all are but some more obviously than others)
- contravariant 7y agoIt's ridiculously bad at modelling quickly rotating objects.
- the_af 7y agoAnother fun example of how your brain predicts physics stuff is when you misjudge what something weighs. For example, if you subconsciously judge a glass is full of water but it's actually empty, as you try to lift it your arm will fly up awkwardly. This all happens without conscious thought. Don't tell me it never happened to you!
- HeWhoLurksLate 7y ago
- flixic 7y agoInteresting to explore how this can be used. You still need to develop a game with an actual engine first, to generate all the samples for training the AI how to generate frames without an engine. But maybe if training sample size can be considerably reduced, game making can open up to far less technical people. Instead of developing with an engine, game makers could show the AI "when I press this button, change the scene like this", and the AI could try to figure out the rules from given samples.
- sitkack 7y agoProgramming then becomes teaching.
- jcims 7y agoSeems like it's basically simulating an environment which it has agency to affect. To the extent that games are simulating some universe, it seems like this could also be applicable for simulating at least one universe that has a game engine we can't quite reproduce yet.
- drcode 7y agoThe design of their GAN looks super complex, and Pacman is pretty simple... I kinda wonder if they meta-overfit the game (ie the Pacman game engine which they say is missing is sorta hidden in the structure of the GAN architecture that they start off with)
- ekianjo 7y ago> Trained on 50,000 episodes of the game Meanwhile an actual human will probably need only about 5 games to fully understand how the game works and reacts.
- ttul 7y agoFundamentally, this is because we have already been trained on years of living in and reacting to the real world, which, to some abstract degree, the game mimicks. The Nvidia model starts with nothing.
- Ericson2314 7y agoHumans intelligence basically evolved (as far as we know) by retarding childhood development so we had neural plasticity longer to learn more. Crazy that we're giving our already-dumb AIs the extra challenge of learning things from nothing and doing so quickly.
- amelius 7y agoSo you are saying that if you first trained it on a different game with similar real-world mechanics, then training it on this game would require far fewer iterations?
- cyberbanjo 7y agoNo, in humans the cycles involved in walking and 3d orienting carry over, so several thousand iterations of pre-game and a few of game.
- mcphage 7y agoIt might. My understanding is, it is common to take trained models from one problem and use parts of them on other problems in a similar domain to speed up training.
- saystupidthings 7y agoI don't think that's what he is saying, but that is true. transfer learning for domain adaption is quite common, not sure how it works for behaviors, but if you were using say... a common controller for both the games then yes, I imagine it would take significantly less iterations to train.
- QuixoticQuibit 7y agoThis research reminds me of one of their April Fool’s videos [1] with a gaming USB stick that you plug into your computer to offer various gaming enhancements. One of which is a “Ghostplay” feature where an AI program can mimic you after training on your gameplay. I remember that video because it always struck me as not being far off from reality. It’ll make the current set of online game cheats/hacks seem like child’s play if it can be done effectively (read: human-like and undetectable). Maybe then a server-running AI will be needed to detect AI-like gameplay mimicry. [1] https://youtu.be/smM-Wdk2RLQ https://youtu.be/smM-Wdk2RLQ
- Ajedi32 7y agoBut if the AI is functionally indistinguishable from a human player (human-like and undetectable), does it even matter really? https://xkcd.com/810/ https://xkcd.com/810/ (Conversely, if they're _not_ functionally indistinguishable from a human player, then that means it's possible to write an AI or other algorithm to detect them.)
- leetcrew 7y agoAI-based detection systems have to be fairly conservative about banning accounts. so there will probably be a transition period where the clientside AI mimics a human well enough not to get banned, but makes enough "dumb computer" type mistakes to get noticed by other human players. this would be very annoying for the humans. if the AI is truly indistinguishable from a human player, then it sort of depends on the human player we are comparing to. if the AI plays like the best human players in the world, this would be a big problem for games without a good ELO system.
- sandworm101 7y ago>> does it even matter really? Because, even if acting like a human, AI can play/fight 24/7. Many online games still have leader boards that can be climbed through steady play rather than short-term bursts of skill.
- indeyets 7y ago
- pbhjpbhj 7y agoWhat're the total number of possible states in Pacman?
- simias 7y agoThe number of possible pellet configurations on the grid is astronomical on its own, then you have pacman positions and ghost positions at least. I don't know the exact value but my estimate is "a very big number".
- eru 7y agoIt's not so much the total number, but the dimensionality. Eg if you have a blob that can move in x and y coordinates, it doesn't get more complicated just because you move from 32-bit floating point numbers to 64-bit floating point numbers. Even though the number of states squares.
- tokai 7y agoIf the old patterns don't work its not really pacman.
- gallerdude 7y agoI wonder what the copyright implications of this are. Obviously it's Pacman, but none of the source is the same.
- lonelappde 7y agoGame rules aren't copyrightable. The visuals are a copy but obviously fair use, and it's obviously trademark use, and Nvidia probably has a marketing partnership with NAMCO for this
- laumars 7y agoI'm not convinced the visuals would be fair use. However it was certainly done in partnership with Namco. It's celebrating Pac-Man's 40th anniversary and says at the bottom of the article: PAC-MAN™ & ©BANDAI NAMCO Entertainment Inc. Plus there is a massive give away that Namco approved this within the article: We were blown away when we saw the results, in disbelief that AI could recreate the iconic PAC-MAN experience without a game engine,” said Koichiro Tsutsumi from BANDAI NAMCO Research Inc., the research development company of the game’s publisher BANDAI NAMCO Entertainment Inc., which provided the PAC-MAN data to train GameGAN. “This research presents exciting possibilities to help game developers accelerate the creative process of developing new level layouts, characters and even games.”
- jerf 7y ago"“We were blown away when we saw the results, in disbelief that AI could recreate the iconic PAC-MAN experience without a game engine,” said Koichiro Tsutsumi from BANDAI NAMCO Research Inc., the research development company of the game’s publisher BANDAI NAMCO Entertainment Inc., which provided the PAC-MAN data to train GameGAN. “This research presents exciting possibilities to help game developers accelerate the creative process of developing new level layouts, characters and even games.”" Emphasis mine. Also, while I disagree that this would be real "fair use" (rather than Internet Pundit fair use, which is virtually indistinguishable from "I want this, so it must be legal") without permission, it wouldn't matter anyhow. If nVidia did this without permission and tried to distribute exactly what we saw, it would also be a trademark violation. Pacman is almost certainly in the top 1% of defended trademarks, if not .1% or .01%... not a great thing to try to use without permission.
- hola123456789 7y agohola
- andr 7y agoIt would be super interesting to see if this knowledge generalizes with transfer learning. For example, after seeing 50,000 episodes of PacMan, would the GAN be able to recreate Space Invaders with just 5,000 extra episodes?
- arkanciscan 7y agoI've never had a harder time explaining something this impressive to my gf.
- spacephysics 7y agoHow long until this method evolves to solve business requirements? (Argument could be made that the game is a set of business requirements) Imagine visual programming but instead of connecting systems and dealing with high level schemas, the stake holder just defines the data flow, and the model creates a (hopefully) more extendable system than current visual programming systems. But the complexities/black box nature of some neural nets (and the CPU designed by AI that was enormously complex) may transcend to this use case. Meaning we can create systems that are more efficient, more easily, but we ourselves can’t interpret the entire system, let alone each piece sufficiently.
- nradov 7y agoIn most real world business situations we can't even define the requirements and constraints in a manner amenable to AI solutions. It only works for the most trivial issues.
- dontreact 7y agoI predict methods like this will be used to create a simulation environment that approximates the real world. Reinforcement learning folks are running out of game engines win inside of. I believe the best way to transcend this coming limitation will be to take a learning based approach similar to this.
- justicezyx 7y agoI mean, it's pretty clear that we are already living in such simulation, right?
- eru 7y agoWhat evidence do you have? How would the world look different, if we weren't in a simulation?
- ngneer 7y agoPlease help me understand the point. Is it to show that nVidia chips can be used to create game AI using GANs? Or is it to aid in game design? Wouldn't a human be in a better position to do both?
- Teknoman117 7y agoIf I'm understanding correctly, the machine learning model was able to create a game executable from information about a game.
- Ajedi32 7y agoFrom the article, it sounds to me more like the AI itself is rendering individual frames of the game. It takes the last frame and the current state of the gamepad as input, and outputs the next frame. They've effectively replaced the entire game code with a neural network.
- lumost 7y agoI've observed an increasing trend with modern AI research that A) It's no longer practical to incorporate into real products given the hardware costs for training and inference. In NLP, large transformer based models can easily occupy a 5 thousand dollar GPU at a paltry 50-60 words per second processing pace. B) The Research often demonstrates things that existing non-ML toolchains were already reasonable at doing with a better result, and a faster iteration time. This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place! While this is fascinating from the technical and academic perspective, it's hard to be convinced that any gains we see in the field won't be offset by increased hardware/latency costs from a product standpoint.
- lostmsu 7y agoA) is just a sign, that further engineering, both model and framework, is required. Also, 50-60 words per second does not sound too bad. It amounts to 4M words per day. I imagine most people only type a thousand words per day. So with a single GPU you can handle 4000 users (assuming batch-style processing). B) This specific example actually shows ML toolchains are orders of magnitude better, than existing ones. Imagine you wanted to make a football game from scratch, and this technique would let you do that by just feeding a lot of football game recordings to a program, as opposed to hiring software devs and graphics designers, and iterating for a year.
- lumost 7y agoOn A: Using a p3.2xl for that task leaves you with a minimum ec2 cost of goods sold of 50 cents per user per month for the target task. To make a reasonable SaaS business out of this you would need to be charging a minimum of $5.00 per user month for the service that this GPU is providing, assuming all other tasks have a negligible impact on Cost of Goods Sold, and that you are able to spread the load optimally throughout the day - and thus don't have to purchase higher capacity during peak. This isn't too bad of a situation if this GPU provides 100% of your value proposition. but, if you needed your product to perform 5 such tasks - or competition/customer demand forces you to move to a bigger model with a 10 words per second inference rate then you may find it hard to balance customer value vs. price vs. margin.
- ge96 7y agodumb question but could ML be made to copy an OS? since this has a learning/input ability with rules.
- danans 7y agoSo AI can learn to play the games, but can it learn to create them? That is, can it create compelling cultural novelty? The most successful examples I've heard are computer generated classical music, but generating abstract art like classical music is simpler than generating art that relies on coherent language and narrative, like opera, or the blues, or a good novel.
- mettamage 7y agoThere has been research on it. I forgot the source and can’t find it easily, but if you email Julian Togelius, he might know. He doesn’t focus on game AI research and is the only name I can remember at the moment. Edit: oh, Togelius also seemed to have researched it. https://www.google.com/amp/s/www.technologyreview.com/2017/11/29/147421/ai-is-dreaming-up-new-kinds-of-video-games/amp/ https://www.google.com/amp/s/www.technologyreview.com/2017/1...
- danans 7y agoCool. Thanks for that link!
- YeGoblynQueenne 7y agoFull article is here: https://cdn.arstechnica.net/wp-content/uploads/2020/05/Nvidia_GameGAN_Research.pdf https://cdn.arstechnica.net/wp-content/uploads/2020/05/Nvidi...
- huffmsa 7y agoSo wait, is this playable, or is it just capable of reproducing video of gameplay?
- dougabug 7y agoThe point is this brings us closer to demonstrating AI which is in some sense grounded in reality, as opposed to some kind of “Clever Hans” or Searle’s “Chinese Box.” The network has learned how to “imagine playing out a scenario” much like a human player might use his own mental model of the game to consider various strategies to attempt, perhaps with the real game. That the network apparently has a mental model which is rich enough to simulate actual game play implies that it hasn’t just fit a static caricurature of the game which immediately breaks on an unseen move. That is, it has a non-trivial “conceptualization” of its environment which goes beyond simple pattern matching or nearest neighbor matching.
- kabes 7y agoHow do they know it's fully functional without trying out how it reacts in every possible situation (which isn't really feasible)?
- SirYandi 7y agoI guess they should train an AI to play it too
- vagab0nd 7y agoDoes anyone know if we can combine knowledge from different domains when training to play games? Example: have a language model read pieces of text about the game, then feed the output of that into a model that's doing normal reinforcement learning. Maybe train them both at the same time? The hope is that the model can combine inputs from different domains and generalize better.
- ookblah 7y agohmm, train a bunch of UI elements and interactions and feed it underlying codebases and see what comes back... lol.
- dougabug 7y agoThat’s actually been done at least a couple years back. https://towardsdatascience.com/code2pix-deep-learning-compiler-for-graphical-user-interfaces-1256c346950b https://towardsdatascience.com/code2pix-deep-learning-compil...
- polivka2 7y agoFor anyone interested on how the ghosts AI works: https://youtu.be/ataGotQ7ir8 https://youtu.be/ataGotQ7ir8 I doubt the AI actually recreated this.
- tripzilch 6y agoI wonder, does it also reproduce the four unique behaviours of the ghosts? (their movement patterns are all programmed differently, one is random, one goes away from you, and I'm not sure about the rest)
- ccvannorman 6y agoThis seems less impressive than the Mario recreation from a few years ago[0] -- I must be missing something because this is a simpler game that was trained on significantly more data, yet the result is comparatively worse. [0] https://www.theverge.com/2017/9/10/16276528/ai-video-games-game-engine https://www.theverge.com/2017/9/10/16276528/ai-video-games-g...