4 ms·

I've done so much with a fine-tuned model of my dog. I previously made coloring pages for my daughter of our dog as an astronaut, wild west sheriff, etc. They

I've done so much with a fine-tuned model of my dog. I previously made coloring pages for my daughter of our dog as an astronaut, wild west sheriff, etc. They're the first pages she ever "colored," which was pretty special for us. Currently I'm working on making her into every type of Pokemon, just for fun.

mdp2021 4y ago
Using which tools, specifically?
- Auracle 4y ago
  Stable Diffusion, generically. StableTuner to fine tune the model - I can't recall the name of the model I trained on top of, but it was one of the top "broad" 1.5 based models on Civitai. Automatic1111 to do the actual generating. I used an anime line art LoRA (at a low weight) along with an offset noise LoRA for the coloring book pages as otherwise SD makes images be perfectly exposed. For something like that you obviously want a lot more white than black. EveryDream2 would be another good tuning solution. Unfortunately that end of things is far from easy. There are a lot of parameters to change and it's all a bit of a mess. I had an almost impossible time doing it with pictures of my niece, my wife is hit or miss, her sister worked really well for some reason, and our dog was also pretty easy.
Auracle 4y ago
I uploaded a couple of the Pokemon generations really quick as examples. I still need to go through and do quick fixes for double tails (the tails on Pokemon are not where they are on regular animals, apparently), watermarks, etc. and do a quick Img2Img on them. https://imgur.com/a/11OxoSA https://imgur.com/a/11OxoSA
- minimaxir 4y ago
  For generating Pokemon, I recommend using this model along with a textual inversion of your pet: https://huggingface.co/lambdalabs/sd-pokemon-diffusers https://huggingface.co/lambdalabs/sd-pokemon-diffusers
  - Filligree 4y ago
    Textual inversion? Not a LoRA / LyCORIS?
    - minimaxir 4y ago
      LoRA requires input images roughly in the same domain, and the Pokemon model is bespoke.
  - Auracle 4y ago
    Looking at the data set, it's a shame each Pokemon isn't at least named. Or, sometimes even worse, here's the caption for the Jolteon image: "a drawing of a yellow and white pikachu" Still, might be worth a try. Thanks.
- jakedahn 4y ago
  These are great!
  - Auracle 4y ago
    Thanks. They aren’t necessarily the best ones - I just uploaded some quickly. Like I said, they still need final touches too. I probably should have worked on the prompt a bit more before I went all in too. For anyone else doing it, the ability to do something like [vaporeon:cinderdog:.5] so it starts with a specific Pokémon and transitions into the dog later was great for some types. One of the fun things about this sort of thing are happy accidents. One of the fire types generated as two side by side - a puppy and an evolution.
go_discover 4y ago
Do you need an m1 macbook to do this? I have a 2015 macbook pro..
- Auracle 4y ago
  I’m on Windows, sorry. There are some colabs where you can do both the training and generation though.
- jeroenhd 4y ago
  Stable Diffusion can run on Intel CPUs through OpenVINO if you don't have a GPU or the funds to rent one online (Google Collab is often used). You still need a decent amount of RAM (running SD takes about 8GB, training seems to run at 6-8GB) so I'd consider 12 or 16GiB of RAM to be a requirement. There's a huge difference in performance (generating an image takes 10 minutes rather than 10 seconds and training a model would take forever) but with some Python knowledge and a lot of patience it can be done. Apple's Intel Macbooks are infamous for their insufficient cooling design for the CPUs they chose, which won't help maintaining a high clock speed for extended durations of time; you may want to find a way to help cool the laptop down to give the chip a chance to boost more, and to prevent prolonged high temperatures from wearing down the hardware quicker.
- ricochet11 4y ago
  theres lots of online services too if you dont have the hardware - e.g. https://www.astria.ai/ https://www.astria.ai/

mdp2021 4y ago

Using which tools, specifically?

Auracle 4y ago
Stable Diffusion, generically. StableTuner to fine tune the model - I can't recall the name of the model I trained on top of, but it was one of the top "broad" 1.5 based models on Civitai. Automatic1111 to do the actual generating. I used an anime line art LoRA (at a low weight) along with an offset noise LoRA for the coloring book pages as otherwise SD makes images be perfectly exposed. For something like that you obviously want a lot more white than black. EveryDream2 would be another good tuning solution. Unfortunately that end of things is far from easy. There are a lot of parameters to change and it's all a bit of a mess. I had an almost impossible time doing it with pictures of my niece, my wife is hit or miss, her sister worked really well for some reason, and our dog was also pretty easy.

Auracle 4y ago

I uploaded a couple of the Pokemon generations really quick as examples. I still need to go through and do quick fixes for double tails (the tails on Pokemon are not where they are on regular animals, apparently), watermarks, etc. and do a quick Img2Img on them. https://imgur.com/a/11OxoSA https://imgur.com/a/11OxoSA

minimaxir 4y ago
For generating Pokemon, I recommend using this model along with a textual inversion of your pet: https://huggingface.co/lambdalabs/sd-pokemon-diffusers https://huggingface.co/lambdalabs/sd-pokemon-diffusers
- Filligree 4y ago
  Textual inversion? Not a LoRA / LyCORIS?
  - minimaxir 4y ago
    LoRA requires input images roughly in the same domain, and the Pokemon model is bespoke.
- Auracle 4y ago
  Looking at the data set, it's a shame each Pokemon isn't at least named. Or, sometimes even worse, here's the caption for the Jolteon image: "a drawing of a yellow and white pikachu" Still, might be worth a try. Thanks.
jakedahn 4y ago
These are great!
- Auracle 4y ago
  Thanks. They aren’t necessarily the best ones - I just uploaded some quickly. Like I said, they still need final touches too. I probably should have worked on the prompt a bit more before I went all in too. For anyone else doing it, the ability to do something like [vaporeon:cinderdog:.5] so it starts with a specific Pokémon and transitions into the dog later was great for some types. One of the fun things about this sort of thing are happy accidents. One of the fire types generated as two side by side - a puppy and an evolution.

go_discover 4y ago

Do you need an m1 macbook to do this? I have a 2015 macbook pro..

Auracle 4y ago
I’m on Windows, sorry. There are some colabs where you can do both the training and generation though.
jeroenhd 4y ago
Stable Diffusion can run on Intel CPUs through OpenVINO if you don't have a GPU or the funds to rent one online (Google Collab is often used). You still need a decent amount of RAM (running SD takes about 8GB, training seems to run at 6-8GB) so I'd consider 12 or 16GiB of RAM to be a requirement. There's a huge difference in performance (generating an image takes 10 minutes rather than 10 seconds and training a model would take forever) but with some Python knowledge and a lot of patience it can be done. Apple's Intel Macbooks are infamous for their insufficient cooling design for the CPUs they chose, which won't help maintaining a high clock speed for extended durations of time; you may want to find a way to help cool the laptop down to give the chip a chance to boost more, and to prevent prolonged high temperatures from wearing down the hardware quicker.
ricochet11 4y ago
theres lots of online services too if you dont have the hardware - e.g. https://www.astria.ai/ https://www.astria.ai/