4 ms·

Deep Learning for Procedural Content Generation – a survey

118 points by homarp 6y ago

minimaxir 6y ago
Interestingly, there's very little discussion of the Transformer architecture aside from the known text implementations like AI Dungeon 2 (using GPT-2/GPT-3). There's no mechanical restriction from using it on text only (with OpenAI's ImageGPT being an extreme example), and it will be interesting to see how/if it competes with LSTMs for nontext domains.
- Der_Einzige 6y ago
  It seems to not support going in the reverse though, e.g. giving a vector in a low dimensional space and inverse transforming to a hypothetical generated document. Closest to this is the context starting text but it's not the same. To be clear, I'm describing using transformers as autoencoders
aaron-santos 6y ago
There have been a few attempts at communicating deep learning methodology for procedural generation specifically within the roguelike community to use a specific example. While there is typically some high-level interest, the interest quickly fades and developers go back to using traditional procgen methods. One of the hangups that I've observed is that many of the introductions for procgen deep learning (and deep learning in general) use SL, USL, and AL which require datasets containing many examples. The (time) cost associated with gathering or creating these examples is not appealing to procgen devs. Procgen devs continue using the procgen equivalent of symbolic AI. RL in procgen is largely avoided for the same reasons that RL is rare in other domains. Despite this, I believe there is a connection between the fields through the lens of optimization problems. Typically procgen practitioners have a handful of parameters which they hand-tune to arrive at their desired results. The number of parameters is kept low so as not to exceed the cognitive capabilities of the practitioner. I'm a believer that by turning many of the current discrete procgen algorithms into continuous-valued generators, the number of generator parameters can be increased a thousand or million-fold so long as an appropriate loss function can be crafted (in practice this isn't very hard and proxies can even be used in tricky cases). In a lot of ways this becomes a reparameterization in a way that makes for more salient generator parameters. For me one path forward is crafting continuous versions of existing discrete generators and using autodiff tools like JAX to optimize procgen parameters. This whole rant is pretty specific to the roguelike domain and probably doesn't carry over well to other spaces. Huge YMMV disclaimer.
- mam2 6y ago
  Yes I think the whole problem for procgen is that it's hard to find an optimisation problem to optimise related to it. if everything goes well with a "symbolic programming" there no reason to change. Its almost
  - aaron-santos 6y ago
    > if everything goes well with a "symbolic programming" there no reason to change. Yes and no. The elephant in the procgen room is the "1000 bowls of oatmean" problem[1] ie: perceptual uniqueness. Many procgen practitioners want to maximize serendipity but don't see a path toward that. I expect that methods explored in academia will filter down to practitioners, but the speed at which this occurs is largely modulated by the number of people with a foot in each world and willing to communicate with both sides. The same kind of gap exists within programming language theory and design, though because it is a bigger field, it benefits from network effects to a greater extent. [1] https://galaxykate0.tumblr.com/post/139774965871/so-you-want-to-build-a-generator https://galaxykate0.tumblr.com/post/139774965871/so-you-want...
    - mam2 6y ago
      Well even the real world looks like oatmeal after 50+ countries...
- munificent 6y ago
  I'm skeptical that machine learning has much to offer the typical roguelike developer. Most roguelikes approach a player/developer ratio of zero. For every successful shipped roguelike with actual players, there are probably a thousand hobby roguelikes with no users but a happy developer or two wiling their time away. This is not a particularly lucrative corner of the gamedev world, so most are in it for the intrinsic motivation. Handcrafting a level generator is a lot of fun. Training an ML model might be fun too, but it seems like a very different sort of activity, and not necessarily one that appeals to roguelike devs. > so long as an appropriate loss function can be crafted (in practice this isn't very hard and proxies can even be used in tricky cases). I'm skeptical that this is as easy as you suggest. Everyone thinks making something "fun" is easy until they sit down to try. Fun is the knife-edge between familiarity and surprise. Optimizing for it is probably in the same order of difficulty as predicting the stock market: If you train your generator to be particularly fun in a certain way, players will quickly grow tired of it, which isn't fun.
grawprog 6y ago
If it produces interesting content as good as or better than traditional methods or has the same level of ease as traditional methods, then sure why not, otherwise why? In the end, the results are what's important. It's cool to have fancy, new ways of doing things, but if something works and the new way isn't significantly better or easier than traditional ways and ends up being out of reach or significantly different than traditional wqys for little to no gain then, yeah, it's not likely to be widely adopted. That all being said, it wouldn't surprise me if some day, some enterprising deep learning fan does come up with some new interesting or more efficient thing that sparks a new line of research and innovation. It's the way with a lot of things, the early adopters and enthusiasts pave the way for the others. I tend to regard a lot of machine learning, deep learning, ai kind of stuff as overblown, buzz wordy, euphemisms for fancy algorithms with internal memory and data analysis and manipulation capabilities. But, that doesn't mean there aren't still innovative things that can be done with it. Likely, someone will come along at some point with something easily accessible made with deep learning that revolutionizes procedural generation and helps in creating more realistic random environments. We're not there yet though, hence the general disregard for it.
- swiley 6y ago
  I'd completely disagree. For me the most exciting thing about procedural content generation is that it can give you some insight into yourself or the content by examining the algorithm used to generate it. Deep learning looses this.

118 points by homarp 6y ago

Interestingly, there's very little discussion of the Transformer architecture aside from the known text implementations like AI Dungeon 2 (using GPT-2/GPT-3). There's no mechanical restriction from using it on text only (with OpenAI's ImageGPT being an extreme example), and it will be interesting to see how/if it competes with LSTMs for nontext domains.

Der_Einzige 6y ago
It seems to not support going in the reverse though, e.g. giving a vector in a low dimensional space and inverse transforming to a hypothetical generated document. Closest to this is the context starting text but it's not the same. To be clear, I'm describing using transformers as autoencoders

aaron-santos 6y ago

There have been a few attempts at communicating deep learning methodology for procedural generation specifically within the roguelike community to use a specific example. While there is typically some high-level interest, the interest quickly fades and developers go back to using traditional procgen methods. One of the hangups that I've observed is that many of the introductions for procgen deep learning (and deep learning in general) use SL, USL, and AL which require datasets containing many examples. The (time) cost associated with gathering or creating these examples is not appealing to procgen devs. Procgen devs continue using the procgen equivalent of symbolic AI. RL in procgen is largely avoided for the same reasons that RL is rare in other domains. Despite this, I believe there is a connection between the fields through the lens of optimization problems. Typically procgen practitioners have a handful of parameters which they hand-tune to arrive at their desired results. The number of parameters is kept low so as not to exceed the cognitive capabilities of the practitioner. I'm a believer that by turning many of the current discrete procgen algorithms into continuous-valued generators, the number of generator parameters can be increased a thousand or million-fold so long as an appropriate loss function can be crafted (in practice this isn't very hard and proxies can even be used in tricky cases). In a lot of ways this becomes a reparameterization in a way that makes for more salient generator parameters. For me one path forward is crafting continuous versions of existing discrete generators and using autodiff tools like JAX to optimize procgen parameters. This whole rant is pretty specific to the roguelike domain and probably doesn't carry over well to other spaces. Huge YMMV disclaimer.

mam2 6y ago
Yes I think the whole problem for procgen is that it's hard to find an optimisation problem to optimise related to it. if everything goes well with a "symbolic programming" there no reason to change. Its almost
- aaron-santos 6y ago
  > if everything goes well with a "symbolic programming" there no reason to change. Yes and no. The elephant in the procgen room is the "1000 bowls of oatmean" problem[1] ie: perceptual uniqueness. Many procgen practitioners want to maximize serendipity but don't see a path toward that. I expect that methods explored in academia will filter down to practitioners, but the speed at which this occurs is largely modulated by the number of people with a foot in each world and willing to communicate with both sides. The same kind of gap exists within programming language theory and design, though because it is a bigger field, it benefits from network effects to a greater extent. [1] https://galaxykate0.tumblr.com/post/139774965871/so-you-want-to-build-a-generator https://galaxykate0.tumblr.com/post/139774965871/so-you-want...
  - mam2 6y ago
    Well even the real world looks like oatmeal after 50+ countries...
munificent 6y ago
I'm skeptical that machine learning has much to offer the typical roguelike developer. Most roguelikes approach a player/developer ratio of zero. For every successful shipped roguelike with actual players, there are probably a thousand hobby roguelikes with no users but a happy developer or two wiling their time away. This is not a particularly lucrative corner of the gamedev world, so most are in it for the intrinsic motivation. Handcrafting a level generator is a lot of fun. Training an ML model might be fun too, but it seems like a very different sort of activity, and not necessarily one that appeals to roguelike devs. > so long as an appropriate loss function can be crafted (in practice this isn't very hard and proxies can even be used in tricky cases). I'm skeptical that this is as easy as you suggest. Everyone thinks making something "fun" is easy until they sit down to try. Fun is the knife-edge between familiarity and surprise. Optimizing for it is probably in the same order of difficulty as predicting the stock market: If you train your generator to be particularly fun in a certain way, players will quickly grow tired of it, which isn't fun.

grawprog 6y ago

If it produces interesting content as good as or better than traditional methods or has the same level of ease as traditional methods, then sure why not, otherwise why? In the end, the results are what's important. It's cool to have fancy, new ways of doing things, but if something works and the new way isn't significantly better or easier than traditional ways and ends up being out of reach or significantly different than traditional wqys for little to no gain then, yeah, it's not likely to be widely adopted. That all being said, it wouldn't surprise me if some day, some enterprising deep learning fan does come up with some new interesting or more efficient thing that sparks a new line of research and innovation. It's the way with a lot of things, the early adopters and enthusiasts pave the way for the others. I tend to regard a lot of machine learning, deep learning, ai kind of stuff as overblown, buzz wordy, euphemisms for fancy algorithms with internal memory and data analysis and manipulation capabilities. But, that doesn't mean there aren't still innovative things that can be done with it. Likely, someone will come along at some point with something easily accessible made with deep learning that revolutionizes procedural generation and helps in creating more realistic random environments. We're not there yet though, hence the general disregard for it.

swiley 6y ago
I'd completely disagree. For me the most exciting thing about procedural content generation is that it can give you some insight into yourself or the content by examining the algorithm used to generate it. Deep learning looses this.