8 ms·

AI Seamless Texture Generator Built-In to Blender

336 points by myth_drannon 4y ago

cdiamand 4y ago
Very cool. There are some really interesting opportunities to integrate stable-diffusion into many creative apps right now. It's neat to see it all happening at once. Another interesting example of how stable-diffusion could be integrated into a workflow: https://www.reddit.com/r/StableDiffusion/comments/wys3w5/applying_img2img_to_video_3d_aladdin/ https://www.reddit.com/r/StableDiffusion/comments/wys3w5/app... So many applications...
shabbatt 4y ago
holy cow! this is insane. should be possible to create a mesh, get stable fusion to generate a UV texture map too! later we should be able to use prompts to generate 3d meshes with full uv texture map as photogammetry picks up pace. first they came for 2D, then they will come for 3D.
- JamesBarney 4y ago
  Yeah, I'm excited about what this means for indie games.
  - zactato 4y ago
    Aren't indie games already dangerously close to the commodity space? Steam is overwhelming these days. There are dozens of games I can build bridges in or new interesting strategy games. How is any one developer supposed to capture enough market share to make any money of their work? I am worried that tools like this will just lower the barrier even more. Maybe its a good thing because it will allow indie devs to spend less time/money on art.
    - myth_drannon 4y ago
      I don't think indie devs are in for the money. That ship sailed a decade ago (90's were the shareware era, 00's were the indie devs era).
      - deleted 4y ago
        [deleted]
      - postsantum 4y ago
        And 10's were the saas era
      - somenameforme 4y ago
        Yip. I think a large motivation for many games is not to make money but to make something that you personally want that isn't already out there that, where the money is just a nice perk. "UnReal World", for the most extreme example I know of, was released and has been in development for more than 3 decades. It's still receiving regular updates, with the dev kind of mixing game and life. It's a game about surviving in the Finnish wilds, by a dev who lives out in the middle of the Finnish wilds.
    - deleted 4y ago
      [deleted]
    - wongarsu 4y ago
      The barrier of entry has been on the floor ever since Steam discontinued Greenlight and started just allowing everyone on the platform. But at the same time they invested a lot in better content discovery: personalized recommendations, the discovery queue, curators you can follow, etc. If you're building the next rehash-of-popular-concept, this asset generator at best saves you a couple minutes shopping the Unity Asset Store, and selecting the right store-bought texture in blender. But it will raise the bar of what's possible with new, innovative settings, which I'm really looking forward to.
    - JellyBeanThief 4y ago
      > How is any one developer supposed to capture enough market share to make any money of their work? I am worried that tools like this will just lower the barrier even more. In an ideal society, everyone has time, energy, and resources to create art themselves just because it makes them happy, as opposed to having to turn a profit.
  - moogly 4y ago
    Maybe finally the pretending-my-programmer-art-is-a-super-opinionated-stylistic-choice-to-go-with-retro-pixel-art-and-not-just-because-it's-so-much-easier-not-to-hire-an-artist fad can be — if not laid to rest — perhaps toned down a bit.
    - badsectoracula 4y ago
      These tools wont replace artists or needing some sort of artistic sense - there are several indie games that had professional artists working on the assets but the developers behind them completely massacred their art. As an example check out Frayed Knights on Steam - i really like the game and think it is both very fun and a very competent freeform blobber RPG, but despite the author having help from artists (and he even worked in some beloved PS1 games himself so he wasn't an amateur at it), the visuals are downright ugly - the UI even looks worse than the default Torque theme! The fact that the game was shipped with what it looks like a rough placeholder made in MS paint for the inventory background, tells me that the only reason for that is that the developer (whom, do not get me wrong, i otherwise respect, just not when it comes to visuals) is blind when it comes to aesthetics (which is a shame because the actual game is both very humorous and has an actually deep character system - but due to the visuals it was largely ignored). This wont be solved by AI, at the end of the day someone will have to decide that something looks good and someone will have to integrate whatever output the AI creates with the game. What will actually happen is that people with some artistic skills will be able to do things faster than they were able before - it will improve several types of games (i.e. those whose art styles fit whatever the AI creates), but it wont let someone without artistic skills suddenly make high quality art assets.
- spyder 4y ago
  Yep: https://github.com/NasirKhalid24/CLIP-Mesh https://github.com/NasirKhalid24/CLIP-Mesh
  - shabbatt 4y ago
jonplackett 4y ago
Anyone know if there are non-blender-specific versions / prompt hacks to get seamless textures out of stable diffusion? Whenever I ask for something like ‘seamless tiling xxxxxx’ it kinda sorta gets the idea, but the resulting texture doesn’t quite tile right.
- duskwuff 4y ago
  If you're using a recent version of stable-diffusion, it's exposed as the --seamless option.
- capableweb 4y ago
  > Whenever I ask for something like ‘seamless tiling xxxxxx’ it kinda sorta gets the idea, but the resulting texture doesn’t quite tile right. Getting seamless tiling requires more than just have "seamless tiling" in the prompt. It also depends on if the fork you're using has that feature at all. https://github.com/lstein/stable-diffusion https://github.com/lstein/stable-diffusion has the feature, but you need to pass it outside the prompt. So if you use the `dream.py` prompt cli, you can pass it `"Hats on the ground" --seamless` and it should be perfectly tilable.
  - jonplackett 4y ago
    Great to know. Thank you!
BobbyJo 4y ago
I can't wait until these kinds of tools are usable live. I'd love open worlds with unique character interactions and scenery. I'm always incredibly disappointed when I've exhausted a game's content or when portions of content are obviously built on some simplistic pattern, either visual or interactive.
yesimahuman 4y ago
As a developer and past indie dev that creates awful art for any projects I take on, this is incredibly exciting. We're getting closer to the reality where an engineer can build a full game or experience completely on their own and maintain a high level of artistic quality.
- jlundberg 4y ago
  Another recommendation for any aspiring solo or small team game developer is the megascans library: https://quixel.com/megascans/ https://quixel.com/megascans/ Included in the Unreal Engine license after Epic purchased Quixel. :)
camjw 4y ago
I think it's so interesting and positive that Stable Diffusion has come out and absolutely destroyed DALL-E as a product. What are the best examples of DALL-E being integrated into a product? Are there any at all?
- skybrian 4y ago
  We haven't seen any integrations but that doesn't mean we have any idea how many users DALL-E has. Stuff showing up on Hacker News isn't a good proxy for this.
  - O__________O 4y ago
    Stable Diffusion easily has 100x more active users that Dall-e; this based on stats OpenAI released and private sources I been able to dig up. Rumored that Stability AI is in process of raising additional funding at over a billion dollar valuation. Unless OpenAI rapidly changes course, only matter of time before they are footnote in history of AI in my opinion, since Stability will likely rapidly go after every single current offering they have, including GPT3/CoPilot.
    - skybrian 4y ago
      Well, I guess it's fortunate that OpenAI the company is owned by a nonprofit that's devoted to research. As long as they get funding they can keep doing research.
      - O__________O 4y ago
        My understanding from Wikipedia is that in 2019, OpenAI transitioned from non-profit to "capped" for-profit. That said, also quickly found what appears to be 990s for a non-profit OpenAI here, but they end in 2020: https://projects.propublica.org/nonprofits/organizations/810861541 https://projects.propublica.org/nonprofits/organizations/810... Quick set of Google searches found nothing currently on their site — anyone have any additional information?
owenpalmer 4y ago
Love this idea!
DonHopkins 4y ago
Blender is the perfect platform for this kind of stuff, since it's all scripted in Python, which is the lingua franca of machine learning.
- robertlagrant 4y ago
  It makes sense that it is, as Python has if statements. ducks
- mkaic 4y ago
  I recently got into Blender addon development and have been really pleasantly surprised with how extensive its Python API actually is — it's not perfect yet, but it's certainly very very useful! My personal wishlist for the future still has things like direct memory access to volume trees, shipping with pyopenvdb, etc, but I'm pretty impressed with how extensible Blender is by default.
jcmontx 4y ago
It makes me so happy to see FOSS providing cutting edge tech for all of us :) This is absolute gold for indie game devs.
Animats 4y ago
Oh, it generates from a text prompt, not a sample texture. I thought this was just a tool to generate wrapped textures from non-wrapped ones. The licensing is a mess. The Blender plug-in is GPL 3, the stable diffusion code is MIT, and the weights for the model have a very restrictive custom license.[1] Whether the weights, which are program-generated, are copyrightable is a serious legal question. [1] https://github.com/lstein/stable-diffusion/blob/61f46cac31b53f3f043819358382102f1d1d0e78/LICENSE-ModelWeights.txt https://github.com/lstein/stable-diffusion/blob/61f46cac31b5...
- naillo 4y ago
  The weights are not particularly restrictive. You're totally allowed to use them to generate things you sell for instance.
  - teraflop 4y ago
    The license isn't particularly restrictive to users, but it does put restrictions on developers. For example, the way I interpret it, if you write software that incorporates Stable Diffusion then it's not legal to use any GPL-licensed code, or vice versa (thanks to the "use-based restrictions"). (Of course, you could argue that this isn't much more "restrictive" than the GPL itself is.)
- nullc 4y ago
  Pretty ironic to assert copyright on the weights while ignoring it on the training data the produces the weights. Are AI practitioners foolish enough to tempt that fate?
- kache_ 4y ago
  >very restrictive custom license restrictive in what sense? Doesn't seem restrictive to me
- wongarsu 4y ago
  Shouldn't the arguments for applying copyright to photographs apply nicely to applying copyright to ML weights too? Sure, the output is generated by a machine, but the output is also created by the creative choices of the machine's user. If anything, it would seem to me that photographs had a much better case to being uncopyrightable, with them being mechanistic reproductions of existing reality.
skykooler 4y ago
Is there a way to run things like this with an AMD graphics card? Every Stable Diffusion project I've seen seems to be CUDA focused.
- habibur 4y ago
  That's because Stable Diffusion is built with PyTorch. Which isn't optimized for anything but CUDA. Even the CPU is a second class citizen there. Let alone AMD or other graphics. Not saying PyTorch doesn't run on anything else. You can but those will lag and some will be hackish. Looks like Nvidia is on its way to be the next Intel.
  - mrguyorama 4y ago
    Part of this is simply that AMD does a TERRIBLE job of writing tooling and software for anything that isn't just "render these triangles for this videogame". Doing raw compute things with AMD GPUs just seems limited to those building supercomputers apparently. Their promised "cross GPU" solution in ROCm is only available on a tiny fraction of the GPUs they make, seemingly without architecture excuses for why it isn't available on 5000 series cards, it took them YEARS to provide a backend that blender could actually use, productively and without crashes and bugs, and their drivers are just in general very fragile. It's weird to me how much lip service AMD puts into making cross platform, developer friendly, free and open GPU compute standards, and then turn around and just not do that.
  - westurner 4y ago
    From the Arch wiki, which has a list of GPU runtimes (but not TPU or QPU runtimes) and arch package names: OpenCL, SYCL, ROCm, HIP,: https://wiki.archlinux.org/title/GPGPU https://wiki.archlinux.org/title/GPGPU : > GPGPU stands for General-purpose computing on graphics processing units. - "PyTorch OpenCL Support" https://github.com/pytorch/pytorch/issues/488 https://github.com/pytorch/pytorch/issues/488 - Blender re: removal of OpenCL support in 2021 : > The combination of the limited Cycles split kernel implementation, driver bugs, and stalled OpenCL standard has made maintenance too difficult. We can only make the kinds of bigger changes we are working on now by starting from a clean slate. We are working with AMD and Intel to get the new kernels working on their GPUs, possibly using different APIs (such as CYCL, HIP, Metal, …). - https://gitlab.com/illwieckz/i-love-compute https://gitlab.com/illwieckz/i-love-compute - https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA - https://github.com/RadeonOpenCompute/clang-ocl https://github.com/RadeonOpenCompute/clang-ocl AMD ROCm: https://en.wikipedia.org/wiki/ROCm https://en.wikipedia.org/wiki/ROCm AMD ROcm supports Pytorch, TensorFlow, MlOpen, rocBLAS on NVIDIA and AMD GPUs: https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni... RadeonOpenCompute/ROCm_Documentation: https://github.com/RadeonOpenCompute/ROCm_Documentation https://github.com/RadeonOpenCompute/ROCm_Documentation ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY : > hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced. ROCmSoftwarePlatform/gpufort: https://github.com/ROCmSoftwarePlatform/gpufort https://github.com/ROCmSoftwarePlatform/gpufort : > GPUFORT: S2S translation tool for CUDA Fortran and Fortran+X in the spirit of hipify ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP: > HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. [...] Key features include: > - HIP is very thin and has little or no performance impact over coding directly in CUDA mode. > - HIP allows coding in a single-source C++ programming language including features such as templates, C++11 lambdas, classes, namespaces, and more. > - HIP allows developers to use the "best" development environment and tools on each target platform. > - The [HIPIFY] tools automatically convert source from CUDA to HIP. > - * Developers can specialize for the platform (CUDA or AMD) to tune for performance or handle tricky cases.*
ByThyGrace 4y ago
Unfortunately none of the three textures shown as examples in the README are seamless pattern textures. That would have completely driven the point home. I really like the idea though.
- hifikuno 4y ago
  I feel like that shouldn't be too hard to add. One of the all-in-one solutions for Stable Diffusion[1] has this feature already. I don't know how it is implemented but it does kind of seem once one of them gets a new feature they all get that new feature before long. --- 1. https://nmkd.itch.io/t2i-gui https://nmkd.itch.io/t2i-gui
  - squeaky-clean 4y ago
    It already supports it (it uses the lstein fork, which most projects use. Probably your link does too). Just none of the sample images in the readme make use of the seamless feature. It would be nice to see what they look like without installing everything.
nullc 4y ago
Now-- someone figure out how to setup the boundary conditions so that it can fill in penrose or Jarkko Kari's aperiodic wang tilings to efficiently get aperiodic textures. If you fill in a set of these tiles with the right edge rules, then you can just formulaically fill a plane and get a non-repeating texture without generating unreasonable amounts of SD output.
Severian 4y ago
Ha, I knew this would come out sooner or later based on my own experiments with Stable Diffusion. It does very well with textures.
hyperific 4y ago
khangaroo 4y ago
I wonder if it would be viable to have a model that generates other components like normal maps based on the generated texture too.
anon012012 4y ago
I'm currently trying to put 1000x wallpaper seamless textures into UE5 Marketplace. I'm saddened to see this news.^^. Well, fuck money anyway right? Here's a tip, you can produce all you need if you follow this guide: https://rentry.org/voldy#-guide- https://rentry.org/voldy#-guide- Just check what this stuff can do: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki... This is the best page on the internet right now. The hottest stuff. Better than Bitcoin. You can get guidance and copy business ideas here: https://lexica.art/ https://lexica.art/ https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images?_sort=rowid https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/im... http://stable-diffusion-guide.s3-website-us-west-2.amazonaws.com/v2/people.html http://stable-diffusion-guide.s3-website-us-west-2.amazonaws... For textures: Once you have generated the color map (diffuse) from StableDiffusion, you can use CrazyBump to BATCH create the normal map and displacement map. I'm currently at my 200th file iteration. http://www.crazybump.com/ http://www.crazybump.com/ CrazyBump, all the cool kids are using it. Now this is where I'm at. Call me crazy. I'm forgetting stuff surely, but it's the best I can do. Go and change the world. PS: You can Batch Upscale the 512 to beautiful 2k+ with this link: https://github.com/xinntao/Real-ESRGAN https://github.com/xinntao/Real-ESRGAN
ramesh31 4y ago
I’m kind of overwhelmed by this stuff at the moment. On the one hand, it’s very clear by now that this new generation of AI is absolutely game changing. But for what? It feels a bit like discovering oil in 1850.
Waterluvian 4y ago
I’ve spent years false starting on personal game dev projects (all for fun and learning) because I have zero art skills. The ability to ask OpenAI for an entire set of game at work has changed everything for me. I’m positively jubilant about the ability to just ask for textures and such. Not necessarily ideal for some master vision of a professional game, but a game changer for people like me. Also probably great for proof of concept and such.
Daub 4y ago
Very exciting work. However, currently I would not yet make it part of my workflow as it does not output normal and roughness maps. However, I see that those are listed in 'Future Directions'.
gareth_untether 4y ago
This is exactly the kind of post that keeps me coming back to HN. I've been using AI for concept art, but hadn't even thought about it for texturing. And now we have a plugin for Blender to do exactly that, simply incredible!
hoppla 4y ago
I am curious to what AI generated content will bring to games. I imagine that almost any aspect of it could be generated
- echelon 4y ago
  The entire game itself at runtime. https://madebyoll.in/posts/game_emulation_via_dnn/demo/ https://madebyoll.in/posts/game_emulation_via_dnn/demo/
  - hoppla 4y ago
    That is amazing, thanks for sharing.
  - debdut 4y ago
    that's awesome but for real games, it would require heck of a GPU that's not available yet
    - echelon 4y ago
      I'm not convinced either way yet. We're just now starting to crack this problem open. Intuitively it seems like we could represent stable 3D positions and geometry that then passes through a separate shading stage. Use a model to replace only the engine and game logic itself. Faster physics and rules than a physics engine. Much less human engineering needed to build a final product. Add a final pass through an ML upscaler for photorealism or cartoon crispness. I'd gamble 50-50 that this is an existential risk to Epic Games. I'd also gamble 50-50 that we see this in five years. I can't think of any incumbent right now that is safe from disruption.

336 points by myth_drannon 4y ago

Very cool. There are some really interesting opportunities to integrate stable-diffusion into many creative apps right now. It's neat to see it all happening at once. Another interesting example of how stable-diffusion could be integrated into a workflow: https://www.reddit.com/r/StableDiffusion/comments/wys3w5/applying_img2img_to_video_3d_aladdin/ https://www.reddit.com/r/StableDiffusion/comments/wys3w5/app... So many applications...

shabbatt 4y ago

holy cow! this is insane. should be possible to create a mesh, get stable fusion to generate a UV texture map too! later we should be able to use prompts to generate 3d meshes with full uv texture map as photogammetry picks up pace. first they came for 2D, then they will come for 3D.

JamesBarney 4y ago
Yeah, I'm excited about what this means for indie games.
- zactato 4y ago
  Aren't indie games already dangerously close to the commodity space? Steam is overwhelming these days. There are dozens of games I can build bridges in or new interesting strategy games. How is any one developer supposed to capture enough market share to make any money of their work? I am worried that tools like this will just lower the barrier even more. Maybe its a good thing because it will allow indie devs to spend less time/money on art.
  - myth_drannon 4y ago
    I don't think indie devs are in for the money. That ship sailed a decade ago (90's were the shareware era, 00's were the indie devs era).
    - deleted 4y ago
      [deleted]
    - postsantum 4y ago
      And 10's were the saas era
    - somenameforme 4y ago
      Yip. I think a large motivation for many games is not to make money but to make something that you personally want that isn't already out there that, where the money is just a nice perk. "UnReal World", for the most extreme example I know of, was released and has been in development for more than 3 decades. It's still receiving regular updates, with the dev kind of mixing game and life. It's a game about surviving in the Finnish wilds, by a dev who lives out in the middle of the Finnish wilds.
  - deleted 4y ago
    [deleted]
  - wongarsu 4y ago
    The barrier of entry has been on the floor ever since Steam discontinued Greenlight and started just allowing everyone on the platform. But at the same time they invested a lot in better content discovery: personalized recommendations, the discovery queue, curators you can follow, etc. If you're building the next rehash-of-popular-concept, this asset generator at best saves you a couple minutes shopping the Unity Asset Store, and selecting the right store-bought texture in blender. But it will raise the bar of what's possible with new, innovative settings, which I'm really looking forward to.
  - JellyBeanThief 4y ago
    > How is any one developer supposed to capture enough market share to make any money of their work? I am worried that tools like this will just lower the barrier even more. In an ideal society, everyone has time, energy, and resources to create art themselves just because it makes them happy, as opposed to having to turn a profit.
- moogly 4y ago
  Maybe finally the pretending-my-programmer-art-is-a-super-opinionated-stylistic-choice-to-go-with-retro-pixel-art-and-not-just-because-it's-so-much-easier-not-to-hire-an-artist fad can be — if not laid to rest — perhaps toned down a bit.
  - badsectoracula 4y ago
    These tools wont replace artists or needing some sort of artistic sense - there are several indie games that had professional artists working on the assets but the developers behind them completely massacred their art. As an example check out Frayed Knights on Steam - i really like the game and think it is both very fun and a very competent freeform blobber RPG, but despite the author having help from artists (and he even worked in some beloved PS1 games himself so he wasn't an amateur at it), the visuals are downright ugly - the UI even looks worse than the default Torque theme! The fact that the game was shipped with what it looks like a rough placeholder made in MS paint for the inventory background, tells me that the only reason for that is that the developer (whom, do not get me wrong, i otherwise respect, just not when it comes to visuals) is blind when it comes to aesthetics (which is a shame because the actual game is both very humorous and has an actually deep character system - but due to the visuals it was largely ignored). This wont be solved by AI, at the end of the day someone will have to decide that something looks good and someone will have to integrate whatever output the AI creates with the game. What will actually happen is that people with some artistic skills will be able to do things faster than they were able before - it will improve several types of games (i.e. those whose art styles fit whatever the AI creates), but it wont let someone without artistic skills suddenly make high quality art assets.
spyder 4y ago
Yep: https://github.com/NasirKhalid24/CLIP-Mesh https://github.com/NasirKhalid24/CLIP-Mesh
- shabbatt 4y ago

jonplackett 4y ago

Anyone know if there are non-blender-specific versions / prompt hacks to get seamless textures out of stable diffusion? Whenever I ask for something like ‘seamless tiling xxxxxx’ it kinda sorta gets the idea, but the resulting texture doesn’t quite tile right.

duskwuff 4y ago
If you're using a recent version of stable-diffusion, it's exposed as the --seamless option.
capableweb 4y ago
> Whenever I ask for something like ‘seamless tiling xxxxxx’ it kinda sorta gets the idea, but the resulting texture doesn’t quite tile right. Getting seamless tiling requires more than just have "seamless tiling" in the prompt. It also depends on if the fork you're using has that feature at all. https://github.com/lstein/stable-diffusion https://github.com/lstein/stable-diffusion has the feature, but you need to pass it outside the prompt. So if you use the `dream.py` prompt cli, you can pass it `"Hats on the ground" --seamless` and it should be perfectly tilable.
- jonplackett 4y ago
  Great to know. Thank you!

BobbyJo 4y ago

I can't wait until these kinds of tools are usable live. I'd love open worlds with unique character interactions and scenery. I'm always incredibly disappointed when I've exhausted a game's content or when portions of content are obviously built on some simplistic pattern, either visual or interactive.

yesimahuman 4y ago

As a developer and past indie dev that creates awful art for any projects I take on, this is incredibly exciting. We're getting closer to the reality where an engineer can build a full game or experience completely on their own and maintain a high level of artistic quality.

jlundberg 4y ago
Another recommendation for any aspiring solo or small team game developer is the megascans library: https://quixel.com/megascans/ https://quixel.com/megascans/ Included in the Unreal Engine license after Epic purchased Quixel. :)

camjw 4y ago

I think it's so interesting and positive that Stable Diffusion has come out and absolutely destroyed DALL-E as a product. What are the best examples of DALL-E being integrated into a product? Are there any at all?

skybrian 4y ago
We haven't seen any integrations but that doesn't mean we have any idea how many users DALL-E has. Stuff showing up on Hacker News isn't a good proxy for this.
- O__________O 4y ago
  Stable Diffusion easily has 100x more active users that Dall-e; this based on stats OpenAI released and private sources I been able to dig up. Rumored that Stability AI is in process of raising additional funding at over a billion dollar valuation. Unless OpenAI rapidly changes course, only matter of time before they are footnote in history of AI in my opinion, since Stability will likely rapidly go after every single current offering they have, including GPT3/CoPilot.
  - skybrian 4y ago
    Well, I guess it's fortunate that OpenAI the company is owned by a nonprofit that's devoted to research. As long as they get funding they can keep doing research.
    - O__________O 4y ago
      My understanding from Wikipedia is that in 2019, OpenAI transitioned from non-profit to "capped" for-profit. That said, also quickly found what appears to be 990s for a non-profit OpenAI here, but they end in 2020: https://projects.propublica.org/nonprofits/organizations/810861541 https://projects.propublica.org/nonprofits/organizations/810... Quick set of Google searches found nothing currently on their site — anyone have any additional information?

owenpalmer 4y ago

Love this idea!

DonHopkins 4y ago

Blender is the perfect platform for this kind of stuff, since it's all scripted in Python, which is the lingua franca of machine learning.

robertlagrant 4y ago
It makes sense that it is, as Python has if statements. ducks
mkaic 4y ago
I recently got into Blender addon development and have been really pleasantly surprised with how extensive its Python API actually is — it's not perfect yet, but it's certainly very very useful! My personal wishlist for the future still has things like direct memory access to volume trees, shipping with pyopenvdb, etc, but I'm pretty impressed with how extensible Blender is by default.

jcmontx 4y ago

It makes me so happy to see FOSS providing cutting edge tech for all of us :) This is absolute gold for indie game devs.

Animats 4y ago

Oh, it generates from a text prompt, not a sample texture. I thought this was just a tool to generate wrapped textures from non-wrapped ones. The licensing is a mess. The Blender plug-in is GPL 3, the stable diffusion code is MIT, and the weights for the model have a very restrictive custom license.[1] Whether the weights, which are program-generated, are copyrightable is a serious legal question. [1] https://github.com/lstein/stable-diffusion/blob/61f46cac31b53f3f043819358382102f1d1d0e78/LICENSE-ModelWeights.txt https://github.com/lstein/stable-diffusion/blob/61f46cac31b5...

naillo 4y ago
The weights are not particularly restrictive. You're totally allowed to use them to generate things you sell for instance.
- teraflop 4y ago
  The license isn't particularly restrictive to users, but it does put restrictions on developers. For example, the way I interpret it, if you write software that incorporates Stable Diffusion then it's not legal to use any GPL-licensed code, or vice versa (thanks to the "use-based restrictions"). (Of course, you could argue that this isn't much more "restrictive" than the GPL itself is.)
nullc 4y ago
Pretty ironic to assert copyright on the weights while ignoring it on the training data the produces the weights. Are AI practitioners foolish enough to tempt that fate?
kache_ 4y ago
>very restrictive custom license restrictive in what sense? Doesn't seem restrictive to me
wongarsu 4y ago
Shouldn't the arguments for applying copyright to photographs apply nicely to applying copyright to ML weights too? Sure, the output is generated by a machine, but the output is also created by the creative choices of the machine's user. If anything, it would seem to me that photographs had a much better case to being uncopyrightable, with them being mechanistic reproductions of existing reality.

skykooler 4y ago

Is there a way to run things like this with an AMD graphics card? Every Stable Diffusion project I've seen seems to be CUDA focused.

habibur 4y ago
That's because Stable Diffusion is built with PyTorch. Which isn't optimized for anything but CUDA. Even the CPU is a second class citizen there. Let alone AMD or other graphics. Not saying PyTorch doesn't run on anything else. You can but those will lag and some will be hackish. Looks like Nvidia is on its way to be the next Intel.
- mrguyorama 4y ago
  Part of this is simply that AMD does a TERRIBLE job of writing tooling and software for anything that isn't just "render these triangles for this videogame". Doing raw compute things with AMD GPUs just seems limited to those building supercomputers apparently. Their promised "cross GPU" solution in ROCm is only available on a tiny fraction of the GPUs they make, seemingly without architecture excuses for why it isn't available on 5000 series cards, it took them YEARS to provide a backend that blender could actually use, productively and without crashes and bugs, and their drivers are just in general very fragile. It's weird to me how much lip service AMD puts into making cross platform, developer friendly, free and open GPU compute standards, and then turn around and just not do that.
- westurner 4y ago
  From the Arch wiki, which has a list of GPU runtimes (but not TPU or QPU runtimes) and arch package names: OpenCL, SYCL, ROCm, HIP,: https://wiki.archlinux.org/title/GPGPU https://wiki.archlinux.org/title/GPGPU : > GPGPU stands for General-purpose computing on graphics processing units. - "PyTorch OpenCL Support" https://github.com/pytorch/pytorch/issues/488 https://github.com/pytorch/pytorch/issues/488 - Blender re: removal of OpenCL support in 2021 : > The combination of the limited Cycles split kernel implementation, driver bugs, and stalled OpenCL standard has made maintenance too difficult. We can only make the kinds of bigger changes we are working on now by starting from a clean slate. We are working with AMD and Intel to get the new kernels working on their GPUs, possibly using different APIs (such as CYCL, HIP, Metal, …). - https://gitlab.com/illwieckz/i-love-compute https://gitlab.com/illwieckz/i-love-compute - https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA - https://github.com/RadeonOpenCompute/clang-ocl https://github.com/RadeonOpenCompute/clang-ocl AMD ROCm: https://en.wikipedia.org/wiki/ROCm https://en.wikipedia.org/wiki/ROCm AMD ROcm supports Pytorch, TensorFlow, MlOpen, rocBLAS on NVIDIA and AMD GPUs: https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni... RadeonOpenCompute/ROCm_Documentation: https://github.com/RadeonOpenCompute/ROCm_Documentation https://github.com/RadeonOpenCompute/ROCm_Documentation ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY : > hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced. ROCmSoftwarePlatform/gpufort: https://github.com/ROCmSoftwarePlatform/gpufort https://github.com/ROCmSoftwarePlatform/gpufort : > GPUFORT: S2S translation tool for CUDA Fortran and Fortran+X in the spirit of hipify ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP: > HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. [...] Key features include: > - HIP is very thin and has little or no performance impact over coding directly in CUDA mode. > - HIP allows coding in a single-source C++ programming language including features such as templates, C++11 lambdas, classes, namespaces, and more. > - HIP allows developers to use the "best" development environment and tools on each target platform. > - The [HIPIFY] tools automatically convert source from CUDA to HIP. > - * Developers can specialize for the platform (CUDA or AMD) to tune for performance or handle tricky cases.*

ByThyGrace 4y ago

Unfortunately none of the three textures shown as examples in the README are seamless pattern textures. That would have completely driven the point home. I really like the idea though.

hifikuno 4y ago
I feel like that shouldn't be too hard to add. One of the all-in-one solutions for Stable Diffusion[1] has this feature already. I don't know how it is implemented but it does kind of seem once one of them gets a new feature they all get that new feature before long. --- 1. https://nmkd.itch.io/t2i-gui https://nmkd.itch.io/t2i-gui
- squeaky-clean 4y ago
  It already supports it (it uses the lstein fork, which most projects use. Probably your link does too). Just none of the sample images in the readme make use of the seamless feature. It would be nice to see what they look like without installing everything.

nullc 4y ago

Now-- someone figure out how to setup the boundary conditions so that it can fill in penrose or Jarkko Kari's aperiodic wang tilings to efficiently get aperiodic textures. If you fill in a set of these tiles with the right edge rules, then you can just formulaically fill a plane and get a non-repeating texture without generating unreasonable amounts of SD output.

Severian 4y ago

Ha, I knew this would come out sooner or later based on my own experiments with Stable Diffusion. It does very well with textures.

hyperific 4y ago

khangaroo 4y ago

I wonder if it would be viable to have a model that generates other components like normal maps based on the generated texture too.

anon012012 4y ago

I'm currently trying to put 1000x wallpaper seamless textures into UE5 Marketplace. I'm saddened to see this news.^^. Well, fuck money anyway right? Here's a tip, you can produce all you need if you follow this guide: https://rentry.org/voldy#-guide- https://rentry.org/voldy#-guide- Just check what this stuff can do: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki... This is the best page on the internet right now. The hottest stuff. Better than Bitcoin. You can get guidance and copy business ideas here: https://lexica.art/ https://lexica.art/ https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images?_sort=rowid https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/im... http://stable-diffusion-guide.s3-website-us-west-2.amazonaws.com/v2/people.html http://stable-diffusion-guide.s3-website-us-west-2.amazonaws... For textures: Once you have generated the color map (diffuse) from StableDiffusion, you can use CrazyBump to BATCH create the normal map and displacement map. I'm currently at my 200th file iteration. http://www.crazybump.com/ http://www.crazybump.com/ CrazyBump, all the cool kids are using it. Now this is where I'm at. Call me crazy. I'm forgetting stuff surely, but it's the best I can do. Go and change the world. PS: You can Batch Upscale the 512 to beautiful 2k+ with this link: https://github.com/xinntao/Real-ESRGAN https://github.com/xinntao/Real-ESRGAN

ramesh31 4y ago

I’m kind of overwhelmed by this stuff at the moment. On the one hand, it’s very clear by now that this new generation of AI is absolutely game changing. But for what? It feels a bit like discovering oil in 1850.

Waterluvian 4y ago

I’ve spent years false starting on personal game dev projects (all for fun and learning) because I have zero art skills. The ability to ask OpenAI for an entire set of game at work has changed everything for me. I’m positively jubilant about the ability to just ask for textures and such. Not necessarily ideal for some master vision of a professional game, but a game changer for people like me. Also probably great for proof of concept and such.

Daub 4y ago

Very exciting work. However, currently I would not yet make it part of my workflow as it does not output normal and roughness maps. However, I see that those are listed in 'Future Directions'.

gareth_untether 4y ago

This is exactly the kind of post that keeps me coming back to HN. I've been using AI for concept art, but hadn't even thought about it for texturing. And now we have a plugin for Blender to do exactly that, simply incredible!

hoppla 4y ago

I am curious to what AI generated content will bring to games. I imagine that almost any aspect of it could be generated

echelon 4y ago
The entire game itself at runtime. https://madebyoll.in/posts/game_emulation_via_dnn/demo/ https://madebyoll.in/posts/game_emulation_via_dnn/demo/
- hoppla 4y ago
  That is amazing, thanks for sharing.
- debdut 4y ago
  that's awesome but for real games, it would require heck of a GPU that's not available yet
  - echelon 4y ago
    I'm not convinced either way yet. We're just now starting to crack this problem open. Intuitively it seems like we could represent stable 3D positions and geometry that then passes through a separate shading stage. Use a model to replace only the engine and game logic itself. Faster physics and rules than a physics engine. Much less human engineering needed to build a final product. Add a final pass through an ML upscaler for photorealism or cartoon crispness. I'd gamble 50-50 that this is an existential risk to Epic Games. I'd also gamble 50-50 that we see this in five years. I can't think of any incumbent right now that is safe from disruption.