19 ms·

Krita AI Diffusion

574 points by unstuck3958 3y ago

2Gkashmiri 3y ago
https://www.youtube.com/watch?v=Ly6USRwTHe0 https://www.youtube.com/watch?v=Ly6USRwTHe0 the video is mindblowing because on one hand, adobe photoshop announced this as "their own next big thing" and here we have an open source software replicating this same thing, so cool. edit: this also means photoshop doesnt have the "moat" they seem to have built around the generative ai thing and their software.
- mikeiz404 3y ago
  Another video from the page showing pose editing: https://www.youtube.com/watch?v=-QDPEcVmdLI https://www.youtube.com/watch?v=-QDPEcVmdLI
  - seanthemon 3y ago
    this is insane, I don't even know if adobe could replicate this easily with photoshop, too many missing features. Excited to see this going forward.
  - wodenokoto 3y ago
    That is absolutely amazing, but it is a shame it has to update the entire image on pose change, and you get a new background everytime.
    - IanCal 3y ago
      Elsewhere in the video they limit changes to a specific region - I wonder if that works currently with the pose changes.
    - anhner 3y ago
      i think one could generate the background first and then the characters separately on a different layer, so any editing only affects them and not the background Edit: just tried it and it mostly works. Generate background, then add pose, add new layer paint on top of pose, select area around character and click generate. The caveat is that it also generates a bit of background around the character, but it does not change it so dramatically
- unstuck3958 3y ago
  What Krita and the KDE project in general have achieved is nothing short of phenomenal, and I don't believe the power of libre software is recognized enough even in dev communities like Hacker News.
- dr_kiszonka 3y ago
  The beginning is impressive, but the owls made me actually want to try this. Super cool.
- qwertox 3y ago
  While watching the video I was also thinking "just like Adobe's stuff". Many of the Photoshop users will ask themselves why they should continue to pay them, if the evolution continues this way. Nice to see. Sure, Krita is not Photoshop, but for the tasks certain creators will be doing in the next decade, they won't have a need for Photoshop anymore. Interesting to see that the video is 2 months old.
  - unstuck3958 3y ago
    > they won't have a need for Photoshop anymore. That is already true for not just Photoshop, but for almost any kind of proprietary software. If you are willing to embrace the caveats and DIY nature of FOSS, for almost every task FOSS Software is good enough (and sometimes better than proprietary). I think one of the major reasons of popularity of proprietary software vs FOSS is marketing.
    - timeon 3y ago
      > FOSS Software is good enough 2d CAD drafting is still lacking a bit
- Traubenfuchs 3y ago
  Thanks for linking the video. The GitHub screenshots are completely useless, because there are no before/after comparisons.
  - vonjuice 3y ago
    At this point selecting good screenshots for git readme's should be a profession of its own, it's baffling how many projects' appeal could be really enhanced by simple informative screenshots.
  - shultays 3y ago
    It is not very obvious but the first image was a link to youtube video. The developer should put a play button on it, using his tool perhaps!
  - simbolit 3y ago
    If you look at the very right of the screenshots, there is a "history" of generations with unused alternatives. Me and my visual cortex managed to synthesize "before" images from this information. Very inconvenient? Yes. Completely useless? No.
- instagraham 3y ago
  Krita support for generative inpainting has been around since the beginning of the Stable Diffusion craze. It was one of the first AI projects I saved. It definitely predates Photoshop adding it. Off the top of my head, this plugin is from Nov 6, 2022, and I know there were others before this (or maybe it was just this shared in earlier form). https://github.com/sddebz/stable-diffusion-krita-plugin https://github.com/sddebz/stable-diffusion-krita-plugin Stable Diffusion heralded an explosion in generative AI that predated ChatGPT. Weird how OpenAI got all the credit when it was Stable Diffusion that first opened the gates.
  - dragonwriter 3y ago
    > or maybe it was just this shared in earlier form). https://github.com/sddebz/stable-diffusion-krita-plugin https://github.com/sddebz/stable-diffusion-krita-plugin Nah that is an early version of a plugin that uses A1111 as the backend instead of ComfyUI (it does have a newer and maintained replacement, but its not the one in OP, which uses a ComfyUI backend.)
  - zerd 3y ago
    > Weird how OpenAI got all the credit when it was Stable Diffusion that first opened the gates. Stable Diffusion came out much later than DALL-E by OpenAI, so I'd say they deserve some credit.
- jasonjayr 3y ago
  From that video you posted: https://youtu.be/Ly6USRwTHe0?t=127 https://youtu.be/Ly6USRwTHe0?t=127 <-- "now draw the rest of the owl"
irusensei 3y ago
> AMD GPU: supported via DirectML, Windows only Uh... I'm not happy with this trend. Thankfully there is an option for using a ComfyUI, a torch based project as a backend.
- Zetobal 3y ago
  That's not a trend that's basically the norm for generative ai and AMD is to blame for it not the devs.
  - dbrgn 3y ago
    Why is that the case? Tools like OpenCL do exist, but I assume CUDA is simply better suited for these tasks, is that true? (With the dominance of CUDA, choice of a GPU on Linux gets even harder. It used to be a clear "fuck you Nvidia" if you wanted to use Wayland, but Nvidia definitely has the lead when it's about video editing and machine learning.)
    - dotnet00 3y ago
      AMD's OpenCL implementation and tooling on Windows is terrible, even worse than NVIDIA's OpenCL tooling, and on Linux their ROCm stuff has been so unreliable in terms of its hardware support that it isn't worth the investment.
  - irusensei 3y ago
    It should not be. Torch and AMD has been a thing forever on Linux even before Windows. The underlying comfyUI supports it. In fact someone replied here it might have been a mistake.
- jstummbillig 3y ago
  Is it a trend, already? I mean I can get behind ambition, but jesus, everything is new, people are cooking. Let's give it a few month.
  - thot_experiment 3y ago
hafriedlander 3y ago
Somewhat tangential, but the Krita community and core team have been pretty explicitly anti-AI. https://krita-artists.org/t/change-in-policy-for-topics-related-to-generative-ai-tools-on-krita-artists/75230 https://krita-artists.org/t/change-in-policy-for-topics-rela... (I am part of a group that builds UI on top of open models, but we stopped working on our Krita version for that reason.)
- thot_experiment 3y ago
  The AI hate is suuuch a meme in the art community, it's very frustrating/alienating. (Though understandably, neoliberal capitalism is also extremely frustrating, so I see why artists mad, I just wish they'd be mad at the root cause.) ((the root cause is that an economic system fundamentally based on scarcity == value doesn't make sense when applied to things that are essentially infinite, and kludgeing in artificial scarcity to make things work is not a good take))
  - whywhywhywhy 3y ago
    It’s more just ridiculous because the same community is completely fine with photobashing and “paint overs” (aka tracing) and “fan art” (aka profiting from IP you don’t own).
    - raincole 3y ago
      Photobashing was hated to death when it was new. Now it's the norm (for environmental concept art at least). AI will go through the same process.
      - neoromantique 3y ago
        AI is the norm for concept art already, it's too useful for it not to be.
    - flxy 3y ago
      I don't know what sort of artists you know, but of all the artists I know, nobody is "completely fine" with tracing. It is okay but still frowned upon when people do it for "practice" without publishing the result, but anything that even looks remotely like it is traced gets called out and further investigated incredibly quickly. And as for fan art, a lot of companies explicitly allow art based on their IP, as long as it's used/published by the artists themselves and the commercial rights to the work aren't sold to some other company. In Japan, there is a whole industry based around derivative works, Doujin - self-published works, that works off of what is essentially a code of honor. Companies don't go against the artists, as long as said artists adhere to certain guidelines on what they're allowed to depict (eg. no NSFW content.) Many franchises have become a lot more popular due to fan art/derivative works alone (ie. Touhou Project, Fate Series.)
axytol 3y ago
They list under hardware requirements "a powerful graphics card with at least 6 GB VRAM is recommended. Otherwise generating images will take very long" Does anyone have any idea what would very long mean on a 4GB VRAM card?
- simbolit 3y ago
  user @bArray 35minutes ago: "Tested on a NVIDIA GeForce RTX 3050 under Ubuntu with 4GB VRAM. (...) lowered the canvas to 2Kx2K and it seems to just about be okay. My test prompt (...) produces a picture of rocks. (...) I get a nice scene (...) Both take about two minutes."
- deleted 3y ago
  [deleted]
- Zelizz 3y ago
  My very-rough feeling about it from playing around with Stable Diffusion is that it takes about 4x as long if it runs out of GPU memory and needs to shuttle data back and forth from system memory. There are a lot of variables though - on my 3070 with 8GB of RAM, I can get very impressive 512x512 images in about 10 seconds with somewhat low sample counts, or I can set it to a higher resolution and sample count with 2x upscaling and get a really sharp image in around 2 minutes.
tannhaeuser 3y ago
It says Mac OS support is untested, but wouldn't Mac OS be a great test bed, with many graphic pro users, and Apple Silicon running Stable Diffusion out of the box? DiffusionBee already does in-/outpainting and basically all the other things this integration is promising, you only have to copy/paste image data and resolution/context parameters I guess. But then this brings in the Python ML stack which seems like a no-go for an end-user product AFAICS, unless you wanted to generate endless support tickets.
- OsrsNeedsf2P 3y ago
  It was made by 2 people. Probably neither of them have Macbooks.
- deleted 3y ago
  [deleted]
dustypotato 3y ago
Too bad I don't have the Hardware to run it. Anyone had success with stable diffusion on Steam Deck ? The only thing that works for me is https://github.com/rupeshs/fastsdcpu https://github.com/rupeshs/fastsdcpu , but it takes 1m per 512x512 image and is LCM
- eterps 3y ago
  > Too bad I don't have the Hardware to run it. Cloud GPUs is supported if that is an option: https://github.com/Acly/krita-ai-diffusion/blob/main/doc/cloud-gpu.md https://github.com/Acly/krita-ai-diffusion/blob/main/doc/clo...
  - dustypotato 3y ago
    Ah I missed it. Thanks
bArray 3y ago
Trying it now and will update later (as a comment), takes a little while to download and install. One note about the installation on Ubuntu is that you need to install Krita first, run it, and then copy the plug-in to the desired folder - otherwise there is nowhere to copy it to.
- bArray 3y ago
  Tested on a NVIDIA GeForce RTX 3050 under Ubuntu with 4GB VRAM. Initially tried with a 4Kx4K canvas size, but it seems too much and fails. I lowered the canvas to 2Kx2K and it seems to just about be okay. My test prompt (to compare against other models): > (masterpiece, best quality), a giant made of rock, highly detailed, rock texture For Cinematic Photo XL this produces a picture of rocks. For Digital Artwork XL I get a nice scene with a complex rock structure. Both take about two minutes. It seems to work well and the integration into Krita seems quite nice. The settings are suitably simple, but would be nice if more was exposed in an advanced window or something.
unixhero 3y ago
This looks incredible. It runs locally???
esjeon 3y ago
I saw a person using this. The system had 4090, which can pull about 20-30 iter/sec. This roughly translates to 4 image/sec with 8 iter/image. This allows interactive AI drawing (thou a bit quirky). Once the desired image is reached, the user can re-run w/ 30-50 iterations to finalize the image. This is really cool.
- pedrovhb 3y ago
  Latent consistency models are a pretty radical game changer that came up recently. There are LoRAs [0] that you can just use alongside any SD or SDXL that just cut the number of inference steps you need to 2-8, rather than the usual ~25+. It's as close to magic as one could expect, and on ComfyUI my modest RX 5700XT spits out 512x512 images in probably around a second each, or a couple of seconds for a 4x batch. A more beefy GPU could certainly enable high res, very low latency interactive use. For even better latency perception, you could hook into the generation steps and have TAESD [1] decoding intermediate latents. [0] https://huggingface.co/collections/latent-consistency/latent-consistency-models-loras-654cdd24e111e16f0865fba6 https://huggingface.co/collections/latent-consistency/latent... [1] https://github.com/madebyollin/taesd https://github.com/madebyollin/taesd
Keyframe 3y ago
Does anyone know / tried if it works with multiple GPUs?
Fraterkes 3y ago
A theoretical nice thing about Krita and art in these past decades was that you could be an 18 year old with some ok drawing skills, a thinkpad, a secondhand wacom tablet and a version of krita, and the internet, this wonderful innovation, could enable you to make some money as an artist. If the future expectation is that artists all have 2000 euro graphics cards, I think that will really make art a lot less democratic.
- orbital-decay 3y ago
  That's not the expectation at all; a lot of work is being done to make it run on underpowered hardware. SD in particular runs on a 8-years-old potato, albeit slowly and with limitations, despite originally barely fitting into 10GB VRAM. >A theoretical nice thing about Krita and art in these past decades was that you could be an 18 year old with some ok drawing skills, a thinkpad, a secondhand wacom tablet and a version of krita You never needed a computer for that, just a pen/pencil and paper. For digital painting in particular though, that only became possible in the recent years. Free digital painting software sucked until recently, so 20 years ago every 18 years old just pirated commercial software. And drawing tablets only became cheap and good after Wacom battery-less patents expired (alternatively, with the advent of iPads with pens that a lot of parents bought for their kids, and cheap drawing software in the App Store). I'm not even starting on 3D, which always required beefy hardware. Tinkering with Maya/3DSMax/Lightwave in early 2000s required a really powerful gaming PC. These days you can at least rent a powerful GPU for peanuts to run the AI model.
  - Fraterkes 3y ago
    Sure, the part about everyone just pirating photoshop is absolutely tue (it comes out to be the same thing though, you can't pirate hardware). My point is the gap in potential quality and art output between photoshop on a powerful pc an a pirated copy of ps on a thinkpad is pretty small: you need a lot of ram to produce 4k art, but a thinkpad is fine for most comissions. The gap is obviously a lot larger with ai: you yourself mention that sd (just one of the models people are currently using) runs slowly and with limitations. If the expectation becomes that you deliver 100 4k permutations on a certain theme, the time it takes to achieve that from a human labor standpoint will be similar, but the time that takes to render wise will vary orders of magnitude based on your resources. Not to mention that a workflow with a realtime refresh rate is qualititavely different frome one that runs 0.1fps.
throwaway30230 3y ago
At this point it seems pointless to even bother to try given that AI will generate all possible artwork within a couple years. I mean. Say you get "good" at using this. What's the life expectancy at any kind of creative outlet you could have that would support you? I mean if we're talking this is fun as a toy, yeah ok. I could see that. But as a job? When everyone can paint no one is paid for it. I suppose that we could all go back to paying people who can physically lift things or wait on tables, but that's about it. I want to use this, but then I just think "Holy shit, what if I get good at this and then get my hopes up like I did with React? What am I going to do, sell artwork that anyone can make for next to nothing on the internet?" I believe I could probably come up with some cool paintings, but the question is "why"? Everyone else on the internet will generate all the possible content it's possible for me to come up with anyway, so why does it matter? And if that makes me care about "money" then yeah, I care about money. So what? All of that being said I'm now going to draw a latex glad ninja being molested by a demon. Also I'm broke and living in a homeless shelter. But I can get a supercomputer to make me draw sexy girls so I have that going for me.
- LoganDark 3y ago
  Never underestimate the number of kinks out there
  - lfkdev 3y ago
    I'm quite sure the will be models specific for all kind of kinks.
    - LoganDark 3y ago
      But not every permutation.
- lock-the-spock 3y ago
  Maybe a nice analogy are the old trades: knitting, weaving, ceramics, glass blowing, woodworking, ... Still down for pleasure and a niche audience.
- 3y ago

574 points by unstuck3958 3y ago

https://www.youtube.com/watch?v=Ly6USRwTHe0 https://www.youtube.com/watch?v=Ly6USRwTHe0 the video is mindblowing because on one hand, adobe photoshop announced this as "their own next big thing" and here we have an open source software replicating this same thing, so cool. edit: this also means photoshop doesnt have the "moat" they seem to have built around the generative ai thing and their software.

mikeiz404 3y ago
Another video from the page showing pose editing: https://www.youtube.com/watch?v=-QDPEcVmdLI https://www.youtube.com/watch?v=-QDPEcVmdLI
- seanthemon 3y ago
  this is insane, I don't even know if adobe could replicate this easily with photoshop, too many missing features. Excited to see this going forward.
- wodenokoto 3y ago
  That is absolutely amazing, but it is a shame it has to update the entire image on pose change, and you get a new background everytime.
  - IanCal 3y ago
    Elsewhere in the video they limit changes to a specific region - I wonder if that works currently with the pose changes.
  - anhner 3y ago
    i think one could generate the background first and then the characters separately on a different layer, so any editing only affects them and not the background Edit: just tried it and it mostly works. Generate background, then add pose, add new layer paint on top of pose, select area around character and click generate. The caveat is that it also generates a bit of background around the character, but it does not change it so dramatically
unstuck3958 3y ago
What Krita and the KDE project in general have achieved is nothing short of phenomenal, and I don't believe the power of libre software is recognized enough even in dev communities like Hacker News.
dr_kiszonka 3y ago
The beginning is impressive, but the owls made me actually want to try this. Super cool.
qwertox 3y ago
While watching the video I was also thinking "just like Adobe's stuff". Many of the Photoshop users will ask themselves why they should continue to pay them, if the evolution continues this way. Nice to see. Sure, Krita is not Photoshop, but for the tasks certain creators will be doing in the next decade, they won't have a need for Photoshop anymore. Interesting to see that the video is 2 months old.
- unstuck3958 3y ago
  > they won't have a need for Photoshop anymore. That is already true for not just Photoshop, but for almost any kind of proprietary software. If you are willing to embrace the caveats and DIY nature of FOSS, for almost every task FOSS Software is good enough (and sometimes better than proprietary). I think one of the major reasons of popularity of proprietary software vs FOSS is marketing.
  - timeon 3y ago
    > FOSS Software is good enough 2d CAD drafting is still lacking a bit
Traubenfuchs 3y ago
Thanks for linking the video. The GitHub screenshots are completely useless, because there are no before/after comparisons.
- vonjuice 3y ago
  At this point selecting good screenshots for git readme's should be a profession of its own, it's baffling how many projects' appeal could be really enhanced by simple informative screenshots.
- shultays 3y ago
  It is not very obvious but the first image was a link to youtube video. The developer should put a play button on it, using his tool perhaps!
- simbolit 3y ago
  If you look at the very right of the screenshots, there is a "history" of generations with unused alternatives. Me and my visual cortex managed to synthesize "before" images from this information. Very inconvenient? Yes. Completely useless? No.
instagraham 3y ago
Krita support for generative inpainting has been around since the beginning of the Stable Diffusion craze. It was one of the first AI projects I saved. It definitely predates Photoshop adding it. Off the top of my head, this plugin is from Nov 6, 2022, and I know there were others before this (or maybe it was just this shared in earlier form). https://github.com/sddebz/stable-diffusion-krita-plugin https://github.com/sddebz/stable-diffusion-krita-plugin Stable Diffusion heralded an explosion in generative AI that predated ChatGPT. Weird how OpenAI got all the credit when it was Stable Diffusion that first opened the gates.
- dragonwriter 3y ago
  > or maybe it was just this shared in earlier form). https://github.com/sddebz/stable-diffusion-krita-plugin https://github.com/sddebz/stable-diffusion-krita-plugin Nah that is an early version of a plugin that uses A1111 as the backend instead of ComfyUI (it does have a newer and maintained replacement, but its not the one in OP, which uses a ComfyUI backend.)
- zerd 3y ago
  > Weird how OpenAI got all the credit when it was Stable Diffusion that first opened the gates. Stable Diffusion came out much later than DALL-E by OpenAI, so I'd say they deserve some credit.
jasonjayr 3y ago
From that video you posted: https://youtu.be/Ly6USRwTHe0?t=127 https://youtu.be/Ly6USRwTHe0?t=127 <-- "now draw the rest of the owl"

irusensei 3y ago

> AMD GPU: supported via DirectML, Windows only Uh... I'm not happy with this trend. Thankfully there is an option for using a ComfyUI, a torch based project as a backend.

Zetobal 3y ago
That's not a trend that's basically the norm for generative ai and AMD is to blame for it not the devs.
- dbrgn 3y ago
  Why is that the case? Tools like OpenCL do exist, but I assume CUDA is simply better suited for these tasks, is that true? (With the dominance of CUDA, choice of a GPU on Linux gets even harder. It used to be a clear "fuck you Nvidia" if you wanted to use Wayland, but Nvidia definitely has the lead when it's about video editing and machine learning.)
  - dotnet00 3y ago
    AMD's OpenCL implementation and tooling on Windows is terrible, even worse than NVIDIA's OpenCL tooling, and on Linux their ROCm stuff has been so unreliable in terms of its hardware support that it isn't worth the investment.
- irusensei 3y ago
  It should not be. Torch and AMD has been a thing forever on Linux even before Windows. The underlying comfyUI supports it. In fact someone replied here it might have been a mistake.
jstummbillig 3y ago
Is it a trend, already? I mean I can get behind ambition, but jesus, everything is new, people are cooking. Let's give it a few month.
- thot_experiment 3y ago

hafriedlander 3y ago

Somewhat tangential, but the Krita community and core team have been pretty explicitly anti-AI. https://krita-artists.org/t/change-in-policy-for-topics-related-to-generative-ai-tools-on-krita-artists/75230 https://krita-artists.org/t/change-in-policy-for-topics-rela... (I am part of a group that builds UI on top of open models, but we stopped working on our Krita version for that reason.)

thot_experiment 3y ago
The AI hate is suuuch a meme in the art community, it's very frustrating/alienating. (Though understandably, neoliberal capitalism is also extremely frustrating, so I see why artists mad, I just wish they'd be mad at the root cause.) ((the root cause is that an economic system fundamentally based on scarcity == value doesn't make sense when applied to things that are essentially infinite, and kludgeing in artificial scarcity to make things work is not a good take))
- whywhywhywhy 3y ago
  It’s more just ridiculous because the same community is completely fine with photobashing and “paint overs” (aka tracing) and “fan art” (aka profiting from IP you don’t own).
  - raincole 3y ago
    Photobashing was hated to death when it was new. Now it's the norm (for environmental concept art at least). AI will go through the same process.
    - neoromantique 3y ago
      AI is the norm for concept art already, it's too useful for it not to be.
  - flxy 3y ago
    I don't know what sort of artists you know, but of all the artists I know, nobody is "completely fine" with tracing. It is okay but still frowned upon when people do it for "practice" without publishing the result, but anything that even looks remotely like it is traced gets called out and further investigated incredibly quickly. And as for fan art, a lot of companies explicitly allow art based on their IP, as long as it's used/published by the artists themselves and the commercial rights to the work aren't sold to some other company. In Japan, there is a whole industry based around derivative works, Doujin - self-published works, that works off of what is essentially a code of honor. Companies don't go against the artists, as long as said artists adhere to certain guidelines on what they're allowed to depict (eg. no NSFW content.) Many franchises have become a lot more popular due to fan art/derivative works alone (ie. Touhou Project, Fate Series.)

axytol 3y ago

They list under hardware requirements "a powerful graphics card with at least 6 GB VRAM is recommended. Otherwise generating images will take very long" Does anyone have any idea what would very long mean on a 4GB VRAM card?

simbolit 3y ago
user @bArray 35minutes ago: "Tested on a NVIDIA GeForce RTX 3050 under Ubuntu with 4GB VRAM. (...) lowered the canvas to 2Kx2K and it seems to just about be okay. My test prompt (...) produces a picture of rocks. (...) I get a nice scene (...) Both take about two minutes."
deleted 3y ago
[deleted]
Zelizz 3y ago
My very-rough feeling about it from playing around with Stable Diffusion is that it takes about 4x as long if it runs out of GPU memory and needs to shuttle data back and forth from system memory. There are a lot of variables though - on my 3070 with 8GB of RAM, I can get very impressive 512x512 images in about 10 seconds with somewhat low sample counts, or I can set it to a higher resolution and sample count with 2x upscaling and get a really sharp image in around 2 minutes.

tannhaeuser 3y ago

It says Mac OS support is untested, but wouldn't Mac OS be a great test bed, with many graphic pro users, and Apple Silicon running Stable Diffusion out of the box? DiffusionBee already does in-/outpainting and basically all the other things this integration is promising, you only have to copy/paste image data and resolution/context parameters I guess. But then this brings in the Python ML stack which seems like a no-go for an end-user product AFAICS, unless you wanted to generate endless support tickets.

OsrsNeedsf2P 3y ago
It was made by 2 people. Probably neither of them have Macbooks.
deleted 3y ago
[deleted]

dustypotato 3y ago

Too bad I don't have the Hardware to run it. Anyone had success with stable diffusion on Steam Deck ? The only thing that works for me is https://github.com/rupeshs/fastsdcpu https://github.com/rupeshs/fastsdcpu , but it takes 1m per 512x512 image and is LCM

eterps 3y ago
> Too bad I don't have the Hardware to run it. Cloud GPUs is supported if that is an option: https://github.com/Acly/krita-ai-diffusion/blob/main/doc/cloud-gpu.md https://github.com/Acly/krita-ai-diffusion/blob/main/doc/clo...
- dustypotato 3y ago
  Ah I missed it. Thanks

bArray 3y ago

Trying it now and will update later (as a comment), takes a little while to download and install. One note about the installation on Ubuntu is that you need to install Krita first, run it, and then copy the plug-in to the desired folder - otherwise there is nowhere to copy it to.

bArray 3y ago
Tested on a NVIDIA GeForce RTX 3050 under Ubuntu with 4GB VRAM. Initially tried with a 4Kx4K canvas size, but it seems too much and fails. I lowered the canvas to 2Kx2K and it seems to just about be okay. My test prompt (to compare against other models): > (masterpiece, best quality), a giant made of rock, highly detailed, rock texture For Cinematic Photo XL this produces a picture of rocks. For Digital Artwork XL I get a nice scene with a complex rock structure. Both take about two minutes. It seems to work well and the integration into Krita seems quite nice. The settings are suitably simple, but would be nice if more was exposed in an advanced window or something.

unixhero 3y ago

This looks incredible. It runs locally???

esjeon 3y ago

I saw a person using this. The system had 4090, which can pull about 20-30 iter/sec. This roughly translates to 4 image/sec with 8 iter/image. This allows interactive AI drawing (thou a bit quirky). Once the desired image is reached, the user can re-run w/ 30-50 iterations to finalize the image. This is really cool.

pedrovhb 3y ago
Latent consistency models are a pretty radical game changer that came up recently. There are LoRAs [0] that you can just use alongside any SD or SDXL that just cut the number of inference steps you need to 2-8, rather than the usual ~25+. It's as close to magic as one could expect, and on ComfyUI my modest RX 5700XT spits out 512x512 images in probably around a second each, or a couple of seconds for a 4x batch. A more beefy GPU could certainly enable high res, very low latency interactive use. For even better latency perception, you could hook into the generation steps and have TAESD [1] decoding intermediate latents. [0] https://huggingface.co/collections/latent-consistency/latent-consistency-models-loras-654cdd24e111e16f0865fba6 https://huggingface.co/collections/latent-consistency/latent... [1] https://github.com/madebyollin/taesd https://github.com/madebyollin/taesd

Keyframe 3y ago

Does anyone know / tried if it works with multiple GPUs?

Fraterkes 3y ago

A theoretical nice thing about Krita and art in these past decades was that you could be an 18 year old with some ok drawing skills, a thinkpad, a secondhand wacom tablet and a version of krita, and the internet, this wonderful innovation, could enable you to make some money as an artist. If the future expectation is that artists all have 2000 euro graphics cards, I think that will really make art a lot less democratic.

orbital-decay 3y ago
That's not the expectation at all; a lot of work is being done to make it run on underpowered hardware. SD in particular runs on a 8-years-old potato, albeit slowly and with limitations, despite originally barely fitting into 10GB VRAM. >A theoretical nice thing about Krita and art in these past decades was that you could be an 18 year old with some ok drawing skills, a thinkpad, a secondhand wacom tablet and a version of krita You never needed a computer for that, just a pen/pencil and paper. For digital painting in particular though, that only became possible in the recent years. Free digital painting software sucked until recently, so 20 years ago every 18 years old just pirated commercial software. And drawing tablets only became cheap and good after Wacom battery-less patents expired (alternatively, with the advent of iPads with pens that a lot of parents bought for their kids, and cheap drawing software in the App Store). I'm not even starting on 3D, which always required beefy hardware. Tinkering with Maya/3DSMax/Lightwave in early 2000s required a really powerful gaming PC. These days you can at least rent a powerful GPU for peanuts to run the AI model.
- Fraterkes 3y ago
  Sure, the part about everyone just pirating photoshop is absolutely tue (it comes out to be the same thing though, you can't pirate hardware). My point is the gap in potential quality and art output between photoshop on a powerful pc an a pirated copy of ps on a thinkpad is pretty small: you need a lot of ram to produce 4k art, but a thinkpad is fine for most comissions. The gap is obviously a lot larger with ai: you yourself mention that sd (just one of the models people are currently using) runs slowly and with limitations. If the expectation becomes that you deliver 100 4k permutations on a certain theme, the time it takes to achieve that from a human labor standpoint will be similar, but the time that takes to render wise will vary orders of magnitude based on your resources. Not to mention that a workflow with a realtime refresh rate is qualititavely different frome one that runs 0.1fps.

throwaway30230 3y ago

At this point it seems pointless to even bother to try given that AI will generate all possible artwork within a couple years. I mean. Say you get "good" at using this. What's the life expectancy at any kind of creative outlet you could have that would support you? I mean if we're talking this is fun as a toy, yeah ok. I could see that. But as a job? When everyone can paint no one is paid for it. I suppose that we could all go back to paying people who can physically lift things or wait on tables, but that's about it. I want to use this, but then I just think "Holy shit, what if I get good at this and then get my hopes up like I did with React? What am I going to do, sell artwork that anyone can make for next to nothing on the internet?" I believe I could probably come up with some cool paintings, but the question is "why"? Everyone else on the internet will generate all the possible content it's possible for me to come up with anyway, so why does it matter? And if that makes me care about "money" then yeah, I care about money. So what? All of that being said I'm now going to draw a latex glad ninja being molested by a demon. Also I'm broke and living in a homeless shelter. But I can get a supercomputer to make me draw sexy girls so I have that going for me.

LoganDark 3y ago
Never underestimate the number of kinks out there
- lfkdev 3y ago
  I'm quite sure the will be models specific for all kind of kinks.
  - LoganDark 3y ago
    But not every permutation.
lock-the-spock 3y ago
Maybe a nice analogy are the old trades: knitting, weaving, ceramics, glass blowing, woodworking, ... Still down for pleasure and a niche audience.
3y ago