6 ms·

Pokemon GAN

77 points by aliabd 5y ago

chrisma0 5y ago
This makes me want to build a Twitter Bot "Cursed Pocket Monsters"...
MauroIksem 5y ago
How long before game freak uses something like this to create new Pokemon?
aliabd 5y ago
Just rebuilt the app with more examples, so the queue should stay a lot shorter now. This was built using Gradio[0] and is hosted on Spaces[1]. Check out the paper[2] and github repo[3]. [0]: https://github.com/gradio-app/gradio https://github.com/gradio-app/gradio [1]: https://huggingface.co/spaces https://huggingface.co/spaces [2]: http://www.cvlibs.net/publications/Sauer2021NEURIPS.pdf http://www.cvlibs.net/publications/Sauer2021NEURIPS.pdf [3]: https://github.com/autonomousvision/projected_gan https://github.com/autonomousvision/projected_gan
- ghostly_s 5y ago
  The examples aren't displaying for me.
  - aliabd 5y ago
    They are the numbers at the bottom left of the interface under the 'clear' button. Try clicking them and it will auto fill the input then you just click submit. Or what do you see?
    - ghostly_s 5y ago
      I see the numbers, but nothing appears when they're selected. If I generate a fresh one it shows up fine. iOS.
arcticbull 5y ago
These are wonderfully low-effort and I love each and every one of them.
minimaxir 5y ago
For context, this is not the same technique that was used to create the AI-generated Pokemon a couple months ago: https://www.reddit.com/r/pokemon/comments/rgmyxp/i_trained_an_ai_on_all_the_official_pokemon/ https://www.reddit.com/r/pokemon/comments/rgmyxp/i_trained_a... That used a finetuned ruDALL-E which is not based on a GAN architecture and is much, much slower than a GAN, albeit the generated Pokemon are more coherent. You can play with it (with no queue) on your own Colab Notebook here: https://colab.research.google.com/drive/1A3t2gQofQGeXo5z1BAr1zqYaqVg3czKd?usp=sharing https://colab.research.google.com/drive/1A3t2gQofQGeXo5z1BAr...
- flor1s 5y ago
  Browsing through their code, it looks like they are using a VQGAN for image generation and ESRGAN for super resolution.
- isoprophlex 5y ago
  Very nice! Unbelievable how clean looking and artifact-free these neural net generated images have become (in a short short time!) compared to eg. this deconvolved monstrosity https://www.christies.com/media-library/images/features/articles/2018/08/09/a-collaboration-between-two-artists-one-human-one-a-machine/edmond-de-belamy-framed-cropped.jpg?w=780 https://www.christies.com/media-library/images/features/arti...
_v7gu 5y ago
Some examples look like they have picked two pokemon and tried to merge them in a very weird way. I'll list what I see so others can compare: - 0 as Arceus + Shaymin S - 1 as Lickilicky + Espurr - 10 as Kirlia + Porygon 2, don't know where the colors are from - 20 as Shellos + Froslass - 30 as Roggenrolla/Oddish/Poliwag - 42 as Garchomp + Trevenant - I see the Kyurem in 60 but no idea where the tail is from - 102 as Dragonite + Hitmonlee
- Starlevel001 5y ago
  0 is closer to Sivally than Arceus. Some manual seeds: 4552 is a corrupted Talonflame 2518 is another Vivillion, and so is 3330, and so is 2216, and so is 3373, and so is 3151 (looks like all Vivillion forms were put in) 3129 is horrific, it's like Glaceon has been stretched into a Wurmple and put on top of a Dracovish head and an unknown body 1742 is a mutilated Slurpuff 2410 is a Tapu mixed with... Corsola-G? 3805 is obviously Magmortar with some dragon head 3546 is clearly Reshiram (with genie arms?), and 1878 is mega Tyranitar (both down to the colours) 2194 is Gible x Joltik 1576 is radioactive Alcremie 2000 is straight up Roselia
  - _v7gu 5y ago
    >0 is closer to Sivally than Arceus. Well, that makes sense with the lore
- minimaxir 5y ago
  The model appears to have overfit a bit (seed 535 is blatantly Vivillion: https://twitter.com/ak92501/status/1492560750103121920 https://twitter.com/ak92501/status/1492560750103121920 ) Which is interesting because GANs tend not to overfit.
  - flor1s 5y ago
    Traditional GANs typically have a different problem called mode collapse, in which the generator learns one (or a few) outputs that the discriminator likes. Conditional GANs are given a condition as input and have more chance of overfitting, as the loss function is a combination of the GAN output and a supervised loss based on the condition.
aliabd 5y ago
Just updated the link again - much faster now (inference down to 0.5s from 9s) Queue will move a lot quicker now.
catillac 5y ago
I am a big fan of hugging face, but whenever these examples come up, they always seem to be half baked and don’t quite work well, and are not compelling. I think it would behoove them to polish these a little more, in my opinion.
- Gigachad 5y ago
  The most interesting part I think is how I can't really describe why the results are bad. The colors, art style, silhouettes, etc are all exactly right. But they are just incoherent on a deeper level. Pokemon designs don't all match to living or realistic animals, but they largely do. When I look at these and imagine we made it real, I can't picture it actually being able to live while most pokemon do satisfy this test. I think this is why its failing. The test and criteria that make it more realistic are too hard for the GAN to work out.
  - gwern 5y ago
    That's my belief about why this (sort of) works where all the previous Pokemon-specific GANs fail so completely: it injects knowledge from ImageNet using a pretrained classifier. ImageNet has a decent number of animals, so some of the knowledge carries over. That makes them more coherent. But it's still going to fall short of the full distribution of animal images out there that SOTA stuff like ruDALL-E-*, GLIDE, ERNIE-ViLG, OFA, or Exaone train on.
- ianbutler 5y ago
  I believe this was made by a community member, huggingface allows people to publish to their model repository to foster more open sharing of work in the ML space. You should look at this as in progress research and not a polished thing, and we should be grateful the space is so open. Edit: Double checking it looks like they are affiliated with HF but it still is more of an open side project and anyone can contribute to the repository so quality will vary.
- minimaxir 5y ago
  These types of demos are typically intended for quick interactive open-source proof-of-concepts rather than an e2e app.
The_rationalist 5y ago

Pokemon GAN

77 points by aliabd 5y ago

chrisma0 5y ago

This makes me want to build a Twitter Bot "Cursed Pocket Monsters"...

MauroIksem 5y ago

How long before game freak uses something like this to create new Pokemon?

aliabd 5y ago

Just rebuilt the app with more examples, so the queue should stay a lot shorter now. This was built using Gradio[0] and is hosted on Spaces[1]. Check out the paper[2] and github repo[3]. [0]: https://github.com/gradio-app/gradio https://github.com/gradio-app/gradio [1]: https://huggingface.co/spaces https://huggingface.co/spaces [2]: http://www.cvlibs.net/publications/Sauer2021NEURIPS.pdf http://www.cvlibs.net/publications/Sauer2021NEURIPS.pdf [3]: https://github.com/autonomousvision/projected_gan https://github.com/autonomousvision/projected_gan

ghostly_s 5y ago
The examples aren't displaying for me.
- aliabd 5y ago
  They are the numbers at the bottom left of the interface under the 'clear' button. Try clicking them and it will auto fill the input then you just click submit. Or what do you see?
  - ghostly_s 5y ago
    I see the numbers, but nothing appears when they're selected. If I generate a fresh one it shows up fine. iOS.

arcticbull 5y ago

These are wonderfully low-effort and I love each and every one of them.

minimaxir 5y ago

For context, this is not the same technique that was used to create the AI-generated Pokemon a couple months ago: https://www.reddit.com/r/pokemon/comments/rgmyxp/i_trained_an_ai_on_all_the_official_pokemon/ https://www.reddit.com/r/pokemon/comments/rgmyxp/i_trained_a... That used a finetuned ruDALL-E which is not based on a GAN architecture and is much, much slower than a GAN, albeit the generated Pokemon are more coherent. You can play with it (with no queue) on your own Colab Notebook here: https://colab.research.google.com/drive/1A3t2gQofQGeXo5z1BAr1zqYaqVg3czKd?usp=sharing https://colab.research.google.com/drive/1A3t2gQofQGeXo5z1BAr...

flor1s 5y ago
Browsing through their code, it looks like they are using a VQGAN for image generation and ESRGAN for super resolution.
isoprophlex 5y ago
Very nice! Unbelievable how clean looking and artifact-free these neural net generated images have become (in a short short time!) compared to eg. this deconvolved monstrosity https://www.christies.com/media-library/images/features/articles/2018/08/09/a-collaboration-between-two-artists-one-human-one-a-machine/edmond-de-belamy-framed-cropped.jpg?w=780 https://www.christies.com/media-library/images/features/arti...

_v7gu 5y ago

Some examples look like they have picked two pokemon and tried to merge them in a very weird way. I'll list what I see so others can compare: - 0 as Arceus + Shaymin S - 1 as Lickilicky + Espurr - 10 as Kirlia + Porygon 2, don't know where the colors are from - 20 as Shellos + Froslass - 30 as Roggenrolla/Oddish/Poliwag - 42 as Garchomp + Trevenant - I see the Kyurem in 60 but no idea where the tail is from - 102 as Dragonite + Hitmonlee

Starlevel001 5y ago
0 is closer to Sivally than Arceus. Some manual seeds: 4552 is a corrupted Talonflame 2518 is another Vivillion, and so is 3330, and so is 2216, and so is 3373, and so is 3151 (looks like all Vivillion forms were put in) 3129 is horrific, it's like Glaceon has been stretched into a Wurmple and put on top of a Dracovish head and an unknown body 1742 is a mutilated Slurpuff 2410 is a Tapu mixed with... Corsola-G? 3805 is obviously Magmortar with some dragon head 3546 is clearly Reshiram (with genie arms?), and 1878 is mega Tyranitar (both down to the colours) 2194 is Gible x Joltik 1576 is radioactive Alcremie 2000 is straight up Roselia
- _v7gu 5y ago
  >0 is closer to Sivally than Arceus. Well, that makes sense with the lore
minimaxir 5y ago
The model appears to have overfit a bit (seed 535 is blatantly Vivillion: https://twitter.com/ak92501/status/1492560750103121920 https://twitter.com/ak92501/status/1492560750103121920 ) Which is interesting because GANs tend not to overfit.
- flor1s 5y ago
  Traditional GANs typically have a different problem called mode collapse, in which the generator learns one (or a few) outputs that the discriminator likes. Conditional GANs are given a condition as input and have more chance of overfitting, as the loss function is a combination of the GAN output and a supervised loss based on the condition.

aliabd 5y ago

Just updated the link again - much faster now (inference down to 0.5s from 9s) Queue will move a lot quicker now.

catillac 5y ago

I am a big fan of hugging face, but whenever these examples come up, they always seem to be half baked and don’t quite work well, and are not compelling. I think it would behoove them to polish these a little more, in my opinion.

Gigachad 5y ago
The most interesting part I think is how I can't really describe why the results are bad. The colors, art style, silhouettes, etc are all exactly right. But they are just incoherent on a deeper level. Pokemon designs don't all match to living or realistic animals, but they largely do. When I look at these and imagine we made it real, I can't picture it actually being able to live while most pokemon do satisfy this test. I think this is why its failing. The test and criteria that make it more realistic are too hard for the GAN to work out.
- gwern 5y ago
  That's my belief about why this (sort of) works where all the previous Pokemon-specific GANs fail so completely: it injects knowledge from ImageNet using a pretrained classifier. ImageNet has a decent number of animals, so some of the knowledge carries over. That makes them more coherent. But it's still going to fall short of the full distribution of animal images out there that SOTA stuff like ruDALL-E-*, GLIDE, ERNIE-ViLG, OFA, or Exaone train on.
ianbutler 5y ago
I believe this was made by a community member, huggingface allows people to publish to their model repository to foster more open sharing of work in the ML space. You should look at this as in progress research and not a polished thing, and we should be grateful the space is so open. Edit: Double checking it looks like they are affiliated with HF but it still is more of an open side project and anyone can contribute to the repository so quality will vary.
minimaxir 5y ago
These types of demos are typically intended for quick interactive open-source proof-of-concepts rather than an e2e app.

The_rationalist 5y ago