25 ms·

Creating Isometric RPG Game Backgrounds

339 points by flimsythoughts 4y ago

rkagerer 4y ago
Thank you to everyone who still works on isometric and 2D games. I hated it when my favorite titles like Civ went to cheesy 3D interfaces with assets that are indistinguishable until you're zoomed in so close its useless as a map.
patriciaaa 4y ago
[dead]
kelseyfrog 4y ago
I've proof of concept-ed the same workflow and it just doesn't hold up in practice. Even so, there are some benefits that truly do help a solo dev. It's actually easier to de-bake yourself than use SD-derived assets. What I mean is that img2depth is severely lacking at the moment. The geometry it produces it unsuitable for compositing with other game assets. You'll notice compositing was avoided in the demo. As such it's easier to just block out geometry in Blender, and either project image as textures to it or repaint them by hand using diffusion output as a reference (or both if you're into that kind of thing). Even so, that's still incredibly useful. One of the most common things holding back a solo dev are art assets. I might not be able to create concept art, models, and textures ex novo, but I can pretty easily block out some geometry, repaint textures, and throw in some basic lighting. Looking forward, ideally img2depth+normal+diffuse+lighting would be a thing, but until that's something it's easier to take SD output as advice and suggestion than tangible assets and that's still worth a lot.
- smaddox 4y ago
  If you're creating the normal maps yourself, perhaps you would benefit from generating the image from that using https://github.com/lllyasviel/ControlNet#controlnet-with-normal-map https://github.com/lllyasviel/ControlNet#controlnet-with-nor...
  - cwkoss 4y ago
    I was thinking the same thing. Define the paths and objects in the space, then 'extrude' the graphics from that.
  - danielbln 4y ago
    CN is a total game changer for generative image models, it solves so many things that were problematic before (proper depth, pose, sensible text and much more). This along with LoRa[0] and other improvements from the SD community really turn this into a super capable toolchain. [0] https://github.com/cloneofsimo/lora https://github.com/cloneofsimo/lora
- herval 4y ago
  ControlNet is where it's at - the depth model can completely reskin an image [1] with a bit of prompting, and you can compose them (multiple controlnets feeding on each other) [1] https://twitter.com/herval/status/1631113661207945217/photo/1 https://twitter.com/herval/status/1631113661207945217/photo/...
  - bbminner 4y ago
    There's also MaskSketch from Google that will be presented at the upcoming CVPR https://github.com/google-research/masksketch https://github.com/google-research/masksketch
paczki 4y ago
Prototyping and concept arting for game assets using stable diffusion has been a very enjoyable process for me. Seeing it being utilized more for isometric is pretty exciting with the sheer amount of iteration you immediately get. Using img2img with your own rough drafts and seeing what the AI can come up with has knocked hours/days off of my process, couldn't be happier with these immensely powerful tools. Random aside, but AI is very good at making tile-able textures too, it's been great. The video in the blog looks very solid in motion, will be interesting to see as you develop better shader techniques on the pieces.
asynchronous 4y ago
I’ve seen similar techniques applied with 2D sprites to great impact, especially for someone terrible at drawing sprite art. On another note once SD can output blender files with somewhat correctly modeled 3D files solo game design could get much more accessible.
- Aerroon 4y ago
  I think animation is a bigger problem. If you have 2D artwork then creating reasonable 3D models off of that is something that you can reasonably learn to do. But animating that model is much harder (or learning to).
jejeyyy77 4y ago
gg game artists
- haolez 4y ago
  Well, these models were trained on data produced by artists. How far can they go into the future on their own?
  - shmerl 4y ago
    So does it mean benefiting on the work of the artists without giving them credit?
    - krapp 4y ago
      Yes. Not only without giving them credit, but for the most part making them unemployable or reducing the value of their expertise to subsistence level.
      - astrange 4y ago
        This has never happened and will never happen. Jevon's paradox means they'll get more work.
        krapp 4y ago
        The world is full of people who have been rendered unemployed or underemployed due to globalization and automation. Not only has it happened, it's happening now, and it will happen to knowledge workers and creatives. The increased demand for the work AI can produce will create jobs filled by AI, so Jevon's paradox doesn't guarantee anything. If it did, cities like Detroit wouldn't be ruins now, as the boom of outsourcing and automation in manufacturing would simply have created more jobs as demand increased. But it turns out the market isn't an infinitely frictionless superfluid and the incentives for businesses change - when they can create value without human labor at all, they will seek to do so in all possible cases.
smrtinsert 4y ago
I think this series might need to be updated with the advent of controlnet. Everything from pose through normal maps can be lifted from existing assets and applied as needed. Can't imagine how much more productive the author will be incorporating these major improvements.
paxys 4y ago
This example perfectly encapsulates the state of generative AI art. You can click a few buttons and instantly get artwork that is indistinguishable from stuff made by professional artists for top-quality games. Amazing! The downside is, your game is indistinguishable from every other game. Close your eyes and pick out anything in the isometric sci-fi/cyberpunk genre from Steam and the result will look exactly like this one. Heck you can probably import it directly from Unity's asset store without even bothering with Stable Diffusion. There's the broader question of – what is AI adding that will appeal to people wanting to buy your game, and the answer to that is still probably nothing.
- fooker 4y ago
  You are missing an important point. It significantly lowers the barrier to make a game, if you have a good idea.
  - paxys 4y ago
    Like I said, you can get an entire game world (maps, artwork, backgrounds, character sprites, weapons, vehicles, animation, sound effects, music) by clicking a few buttons in the Unity asset library. The barrier to entry for making a generic cookie-cutter game that a thousand people have made before is already nonexistent.
    - liuliu 4y ago
      Agreed. The power of SD is actually in its diverse styles. You should be able to aim at any specific style in latent space and get it. This is in contrast with Midjourney, which heavily favors sci-fi modern style.
      - aenvoker 4y ago
        With either tech, this thread is a very cynical take. If you give generic prompts, you get generic results. Want something personal, get personal and you will get personal results out of the AI. The barrier to creation has been lowered. But, when you can create anything, you still have to dig to figure out what you actually want.
aschearer 4y ago
Based on the skeptical comments I was expecting to be disappointed but this is really incredible! Great write-up, too. Thanks for sharing. This approach may not scale or deliver "good enough" results or whatever right now, but it blows my mind that one person can do everything outlined in this and the follow up post on characters. Incredible efficiency gains and this technology came out less than a year ago!
hypertele-Xii 4y ago
It's literally a greybox, with minor surface detail/texture. Not the most impressive demonstration. Any level designer could whip that up in a single afternoon...
- cwkoss 4y ago
  Most software engineers couldn't
- throwaway4aday 4y ago
  Sure, I'll just hire a level designer on my $0 side project budget.
epups 4y ago
I think it looks great as a proof-of-concept. The author also correctly identifies some of the challenges, of which I think the greatest will be consistency. I would explore picking out the best examples it generates, or a style from a game you want to mimic, and train a Dreambooth model on those.
transitivebs 4y ago
I love coming across a great blog like this an realizing it was built using two of my open source repos: - https://github.com/transitive-bullshit/nextjs-notion-starter-kit https://github.com/transitive-bullshit/nextjs-notion-starter... - https://github.com/NotionX/react-notion-x https://github.com/NotionX/react-notion-x
mtrycz2 4y ago
> Creating Generating is correct.
- throwaway4aday 4y ago
  Try following the other steps listed in the article. If you don't already have those skills you'll be spending a lot of time learning and mastering the tools and techniques. Even the SD generation steps require a decent knowledge of SD and then a lot of grind while you generate dozens or hundreds of images and tweak parameters until you get something good.
752963e64 4y ago
[dead]

Thank you to everyone who still works on isometric and 2D games. I hated it when my favorite titles like Civ went to cheesy 3D interfaces with assets that are indistinguishable until you're zoomed in so close its useless as a map.

patriciaaa 4y ago

[dead]

kelseyfrog 4y ago

I've proof of concept-ed the same workflow and it just doesn't hold up in practice. Even so, there are some benefits that truly do help a solo dev. It's actually easier to de-bake yourself than use SD-derived assets. What I mean is that img2depth is severely lacking at the moment. The geometry it produces it unsuitable for compositing with other game assets. You'll notice compositing was avoided in the demo. As such it's easier to just block out geometry in Blender, and either project image as textures to it or repaint them by hand using diffusion output as a reference (or both if you're into that kind of thing). Even so, that's still incredibly useful. One of the most common things holding back a solo dev are art assets. I might not be able to create concept art, models, and textures ex novo, but I can pretty easily block out some geometry, repaint textures, and throw in some basic lighting. Looking forward, ideally img2depth+normal+diffuse+lighting would be a thing, but until that's something it's easier to take SD output as advice and suggestion than tangible assets and that's still worth a lot.

smaddox 4y ago
If you're creating the normal maps yourself, perhaps you would benefit from generating the image from that using https://github.com/lllyasviel/ControlNet#controlnet-with-normal-map https://github.com/lllyasviel/ControlNet#controlnet-with-nor...
- cwkoss 4y ago
  I was thinking the same thing. Define the paths and objects in the space, then 'extrude' the graphics from that.
- danielbln 4y ago
  CN is a total game changer for generative image models, it solves so many things that were problematic before (proper depth, pose, sensible text and much more). This along with LoRa[0] and other improvements from the SD community really turn this into a super capable toolchain. [0] https://github.com/cloneofsimo/lora https://github.com/cloneofsimo/lora
herval 4y ago
ControlNet is where it's at - the depth model can completely reskin an image [1] with a bit of prompting, and you can compose them (multiple controlnets feeding on each other) [1] https://twitter.com/herval/status/1631113661207945217/photo/1 https://twitter.com/herval/status/1631113661207945217/photo/...
- bbminner 4y ago
  There's also MaskSketch from Google that will be presented at the upcoming CVPR https://github.com/google-research/masksketch https://github.com/google-research/masksketch

paczki 4y ago

Prototyping and concept arting for game assets using stable diffusion has been a very enjoyable process for me. Seeing it being utilized more for isometric is pretty exciting with the sheer amount of iteration you immediately get. Using img2img with your own rough drafts and seeing what the AI can come up with has knocked hours/days off of my process, couldn't be happier with these immensely powerful tools. Random aside, but AI is very good at making tile-able textures too, it's been great. The video in the blog looks very solid in motion, will be interesting to see as you develop better shader techniques on the pieces.

asynchronous 4y ago

I’ve seen similar techniques applied with 2D sprites to great impact, especially for someone terrible at drawing sprite art. On another note once SD can output blender files with somewhat correctly modeled 3D files solo game design could get much more accessible.

Aerroon 4y ago
I think animation is a bigger problem. If you have 2D artwork then creating reasonable 3D models off of that is something that you can reasonably learn to do. But animating that model is much harder (or learning to).

jejeyyy77 4y ago

gg game artists

haolez 4y ago
Well, these models were trained on data produced by artists. How far can they go into the future on their own?
- shmerl 4y ago
  So does it mean benefiting on the work of the artists without giving them credit?
  - krapp 4y ago
    Yes. Not only without giving them credit, but for the most part making them unemployable or reducing the value of their expertise to subsistence level.
    - astrange 4y ago
      This has never happened and will never happen. Jevon's paradox means they'll get more work.
      - krapp 4y ago
        The world is full of people who have been rendered unemployed or underemployed due to globalization and automation. Not only has it happened, it's happening now, and it will happen to knowledge workers and creatives. The increased demand for the work AI can produce will create jobs filled by AI, so Jevon's paradox doesn't guarantee anything. If it did, cities like Detroit wouldn't be ruins now, as the boom of outsourcing and automation in manufacturing would simply have created more jobs as demand increased. But it turns out the market isn't an infinitely frictionless superfluid and the incentives for businesses change - when they can create value without human labor at all, they will seek to do so in all possible cases.

smrtinsert 4y ago

I think this series might need to be updated with the advent of controlnet. Everything from pose through normal maps can be lifted from existing assets and applied as needed. Can't imagine how much more productive the author will be incorporating these major improvements.

paxys 4y ago

This example perfectly encapsulates the state of generative AI art. You can click a few buttons and instantly get artwork that is indistinguishable from stuff made by professional artists for top-quality games. Amazing! The downside is, your game is indistinguishable from every other game. Close your eyes and pick out anything in the isometric sci-fi/cyberpunk genre from Steam and the result will look exactly like this one. Heck you can probably import it directly from Unity's asset store without even bothering with Stable Diffusion. There's the broader question of – what is AI adding that will appeal to people wanting to buy your game, and the answer to that is still probably nothing.

fooker 4y ago
You are missing an important point. It significantly lowers the barrier to make a game, if you have a good idea.
- paxys 4y ago
  Like I said, you can get an entire game world (maps, artwork, backgrounds, character sprites, weapons, vehicles, animation, sound effects, music) by clicking a few buttons in the Unity asset library. The barrier to entry for making a generic cookie-cutter game that a thousand people have made before is already nonexistent.
  - liuliu 4y ago
    Agreed. The power of SD is actually in its diverse styles. You should be able to aim at any specific style in latent space and get it. This is in contrast with Midjourney, which heavily favors sci-fi modern style.
    - aenvoker 4y ago
      With either tech, this thread is a very cynical take. If you give generic prompts, you get generic results. Want something personal, get personal and you will get personal results out of the AI. The barrier to creation has been lowered. But, when you can create anything, you still have to dig to figure out what you actually want.

aschearer 4y ago

Based on the skeptical comments I was expecting to be disappointed but this is really incredible! Great write-up, too. Thanks for sharing. This approach may not scale or deliver "good enough" results or whatever right now, but it blows my mind that one person can do everything outlined in this and the follow up post on characters. Incredible efficiency gains and this technology came out less than a year ago!

hypertele-Xii 4y ago

It's literally a greybox, with minor surface detail/texture. Not the most impressive demonstration. Any level designer could whip that up in a single afternoon...

cwkoss 4y ago
Most software engineers couldn't
throwaway4aday 4y ago
Sure, I'll just hire a level designer on my $0 side project budget.

epups 4y ago

I think it looks great as a proof-of-concept. The author also correctly identifies some of the challenges, of which I think the greatest will be consistency. I would explore picking out the best examples it generates, or a style from a game you want to mimic, and train a Dreambooth model on those.

transitivebs 4y ago

I love coming across a great blog like this an realizing it was built using two of my open source repos: - https://github.com/transitive-bullshit/nextjs-notion-starter-kit https://github.com/transitive-bullshit/nextjs-notion-starter... - https://github.com/NotionX/react-notion-x https://github.com/NotionX/react-notion-x

mtrycz2 4y ago

> Creating Generating is correct.

throwaway4aday 4y ago
Try following the other steps listed in the article. If you don't already have those skills you'll be spending a lot of time learning and mastering the tools and techniques. Even the SD generation steps require a decent knowledge of SD and then a lot of grind while you generate dozens or hundreds of images and tweak parameters until you get something good.

752963e64 4y ago

[dead]