18 ms·

Welcome to Waifu Labs v2: How Do AIs Create?

455 points by Cixelyn 5y ago

easrng 5y ago
Cool, it's like an AI-generated picrew
liuru 5y ago
Hey HN, one of the team members here! I hope you all enjoy playing with the new and improved generator! We've been hard at work improving the model quality since the last time the site was posted[1] As both a professional fantasy illustrator & software engineer, I find the concept of AI creativity so fascinating. On one hand, I know that mathematically AI only can hallucinate images that fit within the distribution of things that it's seen. But from the artist perspective, the model's ability to blend two existing styles into something so distinctly new is so incredible (and not to mention also commercially useful!) Anyways, happy to answer any question, thoughts, or concerns! --- [1] https://news.ycombinator.com/item?id=20511459 https://news.ycombinator.com/item?id=20511459]
- hypertele-Xii 5y ago
  Why do stuff like this never come down from the web? I'd pay for a program I could download and use with my own image files.
  - liuru 5y ago
    While our model is not public, there are good resources online for playing with your own images! Like this one by fast.ai! https://docs.fast.ai/vision.gan.html https://docs.fast.ai/vision.gan.html
    - sillysaurusx 5y ago
      - spi 5y ago
        Except that then anybody could literally just download it and start a competing service saving 2 years of development and hundreds of thousands of $ in compute costs over that time?
        sillysaurusx 5y ago
        No, they couldn’t. And I guarantee you they spent nowhere near that sum to train the model, and that it wouldn’t take two years to clone it.
  - Afforess 5y ago
    Same reason the Coca-Cola recipe is not published nor made freely available by the Coca-Cola corporation.
  - Gigachad 5y ago
    They tend to require specific hardware like a NVIDIA GPU. As well as having an ever evolving large model file which they will want to frequently update. Some tools certainly have had offline versions but I guess not many people are interested in setting it all up and are happy with an instant web ui
  - zozbot234 5y ago
    You just need to code up your own model architecture and then train it on your data using some established ML framework. The first step is where well-chosen priors can make a real difference wrt. your end results.
- dimgl 5y ago
  You mention it took two weeks to get to the point that we see in the article. Does this mean two weeks of development, or two weeks to generate the images we're seeing? Or maybe did you train the model for two weeks? That point just wasn't exactly clear for me.
  - liuru 5y ago
    2 weeks to train the model! Development took on-and-off roughly 2 years to achieve the quality you see today.
    - dimgl 5y ago
      Cool! Might want to clarify that. This is crazy impressive!
      - liuru 5y ago
        Got it! Thank you! :D
- darkengine 5y ago
  Is the code or any of the models available to the public? I'd love to mess with this on a local GPU cluster.
  - liuru 5y ago
    Not at the moment! A similar project that I really admire is public, though! https://www.thiswaifudoesnotexist.net/ https://www.thiswaifudoesnotexist.net/
- 2bitencryption 5y ago
  Firstly, amazing work. My question is, how do you figure out how to parameterize "Same character, different pose" / "Same character, different eyes" / "Same character, different gender" / etc? My (super limited) understanding of GANs is that they slowly discover these features over time simply from observation in the data set, and not from any labels. So how could you make e.x. a slider for head position, style, pose, etc? How do you look at the resulting model and figure out "these are the inputs we have to fiddle with to make it use a certain pose"? You mention it a bit in this section, but I didn't fully understand: "By isolating the vectors that control certain features, we can create results like different pose, same character" And I assume the same step needs to be done every time the model is retrained or fine-tuned, because possibly the vectors have shifted within the model since they are not fixed by design?
  - liuru 5y ago
    Yes, your understanding is correct! You can think of it like coordinates on a many-dimensional vector grid. We craft the functions the functions that will illuminate sets of those points based on a combination of observation, what we know about our model architecture, and how our data is arranged. And yes, when the model is retrained, we have to discover them again!
    - chrisdsaldivar 5y ago
      Can you share any resources for reading on this particular topic?
  - flor1s 5y ago
    Not affiliated with this project, but there is a gazillion different variations of GANs. Most just change the adversarial loss to improve the learning rate / quality, but others focus on architectural changes, such as StarGAN, Pix2pix (conditional GAN), CycleGAN, MUNIT, etc. It's really a fascinating field.
- rackjack 5y ago
  I LOVE that "horror". Reminds me of some of the art I've seen on album/single covers. Any chance of letting people access that kind of intermediate step? (Though I know it's a niche as hell use case).
  - liuru 5y ago
    Ah yes, the fine line between charming anime character and lovecraftian horror There was such popular demand for these "horror" images that we made them part of the generation in V2! If you refresh enough on the webpage, you can find some horrors!
    - rackjack 5y ago
      For anyone looking to do this, here's some I made: https://i.imgur.com/1V1wPMC.jpg https://i.imgur.com/1V1wPMC.jpg I rolled around 40 times on the first stage and chose a horror. I skipped the second stage and didn't roll on the third stage, just chose one of the presented details. I skipped the fourth stage because I rolled maybe ~200 times and only saw around 1-2 comparable horrors. https://i.imgur.com/1vBeg1j.jpg https://i.imgur.com/1vBeg1j.jpg I rolled around 100 times on the first stage and saw about 3-4 horrors before choosing a horror. I rolled about 70 times on the second stage, didn't find anything interesting, just chose a normal color palette. I rolled maybe 80 times on the third stage and chose a horror, though the results were pretty consistently horrors. I rolled around 60 times on the fourth stage and saw about 1-2 other horrors before choosing a horror. Also, here's what it looked like after the third stage and before finishing the fourth stage: https://i.imgur.com/ditm8nF.jpg https://i.imgur.com/ditm8nF.jpg It's possible the third and fourth stages can produce horrors from normal faces, I didn't check.
- kregasaurusrex 5y ago
  I purchased a waifu from your vending machine (loved the blog post!) at Gen Con in 2019, but can't see the saved model in my account. Is there a way for me to get a v2 generation?
  - liuru 5y ago
    Welcome back! We're currently working on the data migration from V1! As long as you are using the same email as you did in 2019, you'll be able to see the image again! As for a V2 generation, sorry, because the models are different, you'll have to discover a similar image again, if you want a V2 version!
    - kregasaurusrex 5y ago
      Ah that's alright then, thanks for the quick response. I was so happy to see your guys' booth there and own an obscure piece of internet history! https://storage.cloud.google.com/waifus-images/6b94c2ea-51be-49b5-ab16-5ba3ca0ef133.png https://storage.cloud.google.com/waifus-images/6b94c2ea-51be...
    - natch 5y ago
      Can't you use projection with the original image as input? Not for an exact copy, of course, but for a similar V2 rendition?
- kouteiheika 5y ago
  What are the terms of use for the images generated through your website? I'm guessing any commercial use is forbidden? It would be nice if you could formally spell it out on the website.
  - JetAlone 5y ago
    I don't think there's any powerful enough way to stop people from generating one and then tracing over it to create their own linework, and customize things like the colouring and shading. The more broadly AI is able to create, the more niche and obfuscated directions human co-creators could take its products in.
    - marcan_42 5y ago
      Not to mention the copyrightability of AI output isn't legally well tested, and chances are it'll fall in the direction of being copyrighted by the user (i.e. the person clicking in the UI to make a character, which is where creative input happens), not by the AI creator (who has no creative input into the process; they merely made a tool, like any other piece of software - same reason documents printed from Microsoft Word aren't copyrighted by Microsoft). I'm not entirely sure how much legal weight a ToS on the website would have on what the users do with the output. As I understand it, you could e.g. forbid explicitly using the service/generator for commercial purposes (e.g. during game development), but if someone generates a cool character playing around with no particular commercial objective and then decides post facto to build a media megafranchise out of that character, absent any copyright claim over the image, I don't think there's anything stopping them. They wouldn't even need to trace over it, though if they want new artwork in different poses, they couldn't keep using the AI for that with explicit commercial intent; they'd have to get humans to re-draw it. Alternatively, a pessimistic view of the interaction between copyright and AI would be that the model is a derivative work of all the training input, and its output also is, and then good luck building a non copyright infringing AI. IANAL and all that, but it would definitely be legally risky to assume that as the provider of an AI generator you have any control over what users do with the output.
      - 5y ago
- searchableguy 5y ago
  Do you plan to provide an API to generate waifu? I think I could use this for a project.
  - liuru 5y ago
    In the future, perhaps! This is a popular request, so we are thinking about ways we can do this.
- unobatbayar 5y ago
  The quality and style is mindblowing! What data did you train?
  - Bombthecat 5y ago
    The more interesting question: What about the sources for the AI to train? How are those artists paid? Do we need to pay them? Or if it used by an AI as train data, we just say: Its like a human learning?
    - Gigachad 5y ago
      There is nothing that suggests it should be any different than human learning legally. As long as the output is significantly unique, it shouldn’t have any copyright issues.
      - nonbirithm 5y ago
        Does it matter if the artist themselves rejects the idea though? People were discussing having a "no machine learning clause" back when Copilot was being heavily scrutinized. I wouldn't be surprised if some artists allow republishing but not machine learning use. Plenty of artists already have a clause that prohibits any kind of republishing, and Danbooru is known to rehost some of those artists' content anyway until the artist notices and requests that it be taken down, if ever (for a time, they even allowed paid rewards from Patreon and other subscription services to be republished). The original dataset from Danbooru probably contained some percentage of content that would not have been there if the original artist had noticed in time.
        nybble41 5y ago
        > Does it matter if the artist themselves rejects the idea though? Why would it, if there are no copyright issues? No one is obligated to accept a license unless they require permission to do something which copyright would restrict. Of course redistributing the original image as part of a public dataset may be problematic, but simply using it to train an AI model—essentially the equivalent of studying it while teaching yourself to draw—is arguably not among the things covered by copyright, so you don't need a license for that and any clauses in such a license would be irrelevant. This is also basically educational in nature, even if it's a machine rather than a human being "educated", and educational use is often exempt from copyright in some degree or another to begin with. If the dataset is restricted to non-commercial research and educational use in the right jurisdictions then even redistribution may not be an issue.
- wodenokoto 5y ago
  Naïvely I thought Waifu generator was just “some guy having a laugh” fine-tuning a model off of hugging face, but reading through the comments here, it is obviously a much, much bigger enterprise. Can you talk a little about team size, work process, funding and revenue stream? I think the effort required for such an undertaking is vastly underestimated by readers.
  - hansel_der 5y ago
    > Naïvely I thought Waifu generator was just “some guy having a laugh” same here. what's naive about it? not to badmouth the undertaking, but wtf is this doing on HN?
    - wodenokoto 5y ago
      Apparently it takes 6 people making a business to run a waifu generator. That pretty far from one person doing it as a joke.
      - hansel_der 5y ago
        well, apparently... i sincerely applaud the creativity of establishing such a business
  - Cixelyn 5y ago
    Right now it's a small team of 6 people, and we have a bit of funding + compute credits to train models. There's a bit of revenue from some past projects and AI-consulting, but we're mostly betting big on our new AI-powerd mobile title Arrowmancer[1]. > I think the effort required for such an undertaking is vastly underestimated by readers. Haha for sure. Hosting a real-time ML model for people to do sub 1-second inferences at HN-load scale is definitely nontrivial. [1] https://arrowmancer.com https://arrowmancer.com
    - KronisLV 5y ago
      That seems like a nice project that you're working on, definitely more high effort than some other attempts at generated art (procedurally or otherwise). I can't help but to feel that this would be a better fit for the fad of NFTs as well, as opposed to ugly monkeys or other asset flips that were pretty obvious cash grabs. Either way, good luck!
- ridaj 5y ago
  Fascinating... Thanks for sharing A couple questions: 1) I didn't really understand how you went about identifying what vectors of the latent space stand for various things, like pose or color. Did you train one of the AIs to that effect, or did you manually inspect a bunch of vectors, twiddling through them one by one, did to the outcome? 2) If one were to train an AI to the same level using commodity cloud services, what's the order of magnitude cost that you would pay for the training? More like $100, $1,000, $10,000 or $100,000?
  - liuru 5y ago
    1) It was mostly manual, though AIs were useful in certain filtering tasks. 2) Depends on the quality you are seeking. If you only want one run of a similar, off-the-shelf model, around the 1000s is enough. But at the number of iterations you have to run to build your own and improve results, you probably need about 100k. To tackle this problem, we built our own supercomputer off of parts we bought off of ebay, though I can't say I recommend that route, because it now lives in our living room.
    - wnkrshm 5y ago
      Very curious about the computer, what are the internals?
      - Cixelyn 5y ago
        Will probably blog probably in a week or two. But the tldr; is a lot of scavenged EPYCs and nvidia GPUs all in a large sound-proof rack.
      - Filligree 5y ago
        Not the OP, but my guess would be threadrippers (or similar w/lots of PCIe lanes), each with a great number of GPUs. That's usually what you'd do for training AI in a home lab. Server processors gets you more bang for the buck... iff you're planning to run the hardware flat out for literally years. You save on power, but the up-front cost takes up most of that, so for a system that's mostly idle you wouldn't use them. On the other hand, any CPU with fewer PCIe lanes than a TR won't be able to run multiple GPUs optimally, and TRs are relatively cheap enough to make the reduction in PSUs/chassises worth it. Not to mention that there are some approaches to training you can only use if you have multiple GPUs on the same motherboard, aka. sharding a single model across GPUs without communications overhead killing any benefit of that.
- thyrox 5y ago
  Roughly speaking how much money did you invest into making this? Just curious if this is something an indie hacker can hope to do one day OR do you need some deep pockets to make a site like this?
- Cthulhu_ 5y ago
  With the game you're building, are the character portraits generated once and that's it, or do you plan on making them dynamic or frequently updated? I've seen a number of mobile games that just get flooded with characters; this tool looks like it could be used to automate that process. It could be combined with AI-generated character profiles as well, creating an 'infinite' character roster in video games.
- GoblinSlayer 5y ago
  Would you try to create a new style? Train the discriminator on the score tag of danbooru dataset, then use it to rate the generator's style, this way it should be able to create a new style.
- Terry_Roll 5y ago
  I wonder what an AI trained to spot deepfake Waifu's will detect. In humans, things like the pupil can be the give away. https://www.newscientist.com/article/2289815-ai-can-detect-a-deepfake-face-because-its-pupils-have-jagged-edges/ https://www.newscientist.com/article/2289815-ai-can-detect-a...
  - yccs27 5y ago
    This is a super interesting question, given that the generator model is trained to fool the discriminator, which is also an AI.
  - oneoff786 5y ago
    Highlight the pixels with high sharp values. Should be doable.
- CuriousCosmic 5y ago
  Something I was wondering but couldn't find on the site: What is the license for the generated works through the project?
- YeGoblynQueenne 5y ago
  Hello and thank you for answering questions. The following is a quote from your article: >> It is interesting to note that from this process, the AI is not merely learning to copy the works it has seen, but forming high-level (shapes) and low-level (texture) features for constructing original pictures in its own mental representation. Can you explain what you mean by "mental" representation? Does your system have a mind? Also, why are you calling it "an AI"? Is it because you think it is an artificial intelligence, say like the robots in science fiction movies? Is it capable of anything else than generating images?
  - xg15 5y ago
    Not OP, but I wonder if the process would be in some way comparable to rigging a 3D model. There is well, you usually have some high-level input parameters, which influence joints on a predefined skeleton, which in turn determines the position of individual vertices in the 3D body. Finally, the 3D shape is used to render the actual pixels. On each step, high-level parameters are combined with predefined weights to produce a more low-level output. Seems, a similar transformation is going on here, except that the weights and the structure are somehow learned on its own.
- simonebrunozzi 5y ago
  So neat! Where are you based? Boston, I assume? Is there an email to reach out to you or someone in the team? ($HNusername @ gmail)
  - Cixelyn 5y ago
    San Francisco! Just sent over a ping!
- tedmcory77 5y ago
  Who would someone speak with about licensing things made using waifu? My email contact is in my profile...
kadokaelan 5y ago
The explainer video about gans is top-notch! Excited for Arrowmancer!
- Cixelyn 5y ago
  (link for the lazy: https://youtu.be/Pab8pG5WbXQ https://youtu.be/Pab8pG5WbXQ) Thanks so much! It's done by our fantastic animator[1]! GANs are quite interesting and we didn't see many approachable explainer videos targeted at lay people, so we decided to make one ourselves! [1] https://twitter.com/bumblingbeebo https://twitter.com/bumblingbeebo
  - Bombthecat 5y ago
    Is there a gameplay video? Or at least screenshots of arrowverse?
    - Cixelyn 5y ago
      Here's our current game trailer! https://www.youtube.com/watch?v=8WvRgb6kh4s https://www.youtube.com/watch?v=8WvRgb6kh4s
- slimsag 5y ago
  For the author: there's a small typo "Discrimniator" instead of "Discriminator" in the video at 1:11 One thing I was confused by: the video says the discriminator "AI" is trained to detect true vs. generated results, with the hope the generator becomes good enough to fool the discriminator. But why is the discriminator useful, then? Couldn't you just tell generator "AI" whether the result it produced was true or not? I think the answer is.. you don't want just a perfect recreation of the training data you gave to the generator, instead you want the generator to produce variations of that training data, so there's a "how would you know if it's 'a true result' / good enough?" problem. So the discriminator is useful because it's not a direct comparison, but rather a "this looks approximately good enough" comparison of the true vs. generated result. This all makes me wonder: what sort of data set needs to be fed to the discriminator to train it? Is it some sort of "true image" and "true image w/bad alterations (e.g. lines, scratches, etc.) to it" data set?
ve55 5y ago
Links to related projects in anime art generation for those interested: Waifu Labs v2, referenced in this post (generate amazing custom anime face images): https://waifulabs.com https://waifulabs.com (write-up is the above link: https://waifulabs.com/blog/ai-creativity https://waifulabs.com/blog/ai-creativity) This Anime Does Not Exist (AI-generated anime-style artwork): https://thisanimedoesnotexist.ai https://thisanimedoesnotexist.ai (write-up https://www.gwern.net/Faces#extended-stylegan2-danbooru2019-aydao https://www.gwern.net/Faces#extended-stylegan2-danbooru2019-... and https://nearcyan.com/this-anime-does-not-exist https://nearcyan.com/this-anime-does-not-exist) This Waifu Does Not Exist (AI-generated anime-style faces): https://thiswaifudoesnotexist.net https://thiswaifudoesnotexist.net (write-up: https://www.gwern.net/Faces#twdne https://www.gwern.net/Faces#twdne) There's also a lot of literate on e.g. automatic manga coloration, auto-translation, image superresolution, anime frame interpolation, and much more. Worth checking out some places like https://old.reddit.com/r/AnimeResearch/ https://old.reddit.com/r/AnimeResearch/ if you're interested!
- the8472 5y ago
  Note that TADNE, being based on newer models, generates full images rather than only faces.
- numpad0 5y ago
  This one too https://twitter.com/t_takasaka/status/1479784432697749513 https://twitter.com/t_takasaka/status/1479784432697749513
spywaregorilla 5y ago
Is there any way to approximate embeddings for a novel image? novel meaning user provided, not generated by the model or in the training set.
- liuru 5y ago
  Not at the moment in our tool, though this is an area of great curiosity and research for us!
  - spywaregorilla 5y ago
    Does the discriminator model translate the images into an embedding space of it's own? Could such a space be used to generate images themselves?
mensetmanusman 5y ago
I can’t wait for this technology to come to video. Imagine a future where people can compile written scripts into Hollywood quality movies.
- thewarrior 5y ago
  It's already begun https://twitter.com/somnai_dreams/status/1477411531037937664?s=21 https://twitter.com/somnai_dreams/status/1477411531037937664...
  - FeepingCreature 5y ago
    I wonder what will happen when somebody combines a GAN with a feature recognizing network like the Tesla cars use, so it can use its own extrapolated map of the surroundings to stabilize its output as the camera moves around.
- deleted 5y ago
  [deleted]
- JetAlone 5y ago
  I think it will have its limits, but the possibilities for editing together, supplementing and modifying products from smaller AI modules should stretch out what you can do on a small budget.
ps91 5y ago
Amazing now I can create my own anime girl!
- soolinder 5y ago
KaoruAoiShiho 5y ago
I don't want to go all SJW on you guys, amazing work, but can you try to make sure there's an inclusive array of starting faces please? Talking about things like skin tones, thanks!
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- liuru 5y ago
  Indeed, we spent 2 years working on this!!! It's an extremely hard research problem, because darker skin tones account for only about 0.3% of all anime art produced in the world. We have employed an absolutely exhaustive array of art and data science tricks to give the model the ability to draw darker skin tones, though they are underrepresented. The results that you see today are the culmination of many months of careful tuning! It's not definitely perfect, but from a data science perspective, this situation can't be rectified until the art world makes a shift. Personally, I hope that more art representing dark skin tones will be created in the world!
  - KaoruAoiShiho 5y ago
    Thanks, happy to hear that you guys are on top of it.
  - pfisherman 5y ago
    I can’t understand why the GP comment is flagged. If you can look past all the “culture war” stuff, this is pointing out some of the limits of algorithmic creativity. It does not do well generating instances with features that are not well represented in the training dataset. Compare this to human creativity. I suspect that fulfilling GPs request would be almost trivial for a human professional artist. To be clear this is an amazing achievement, a creative use of the technology, and a positive contribution to the world. Pointing out limitations (i.e. areas with potential for future innovation) does not diminish it.
deleted 5y ago
[deleted]
Gigachad 5y ago
This is extremely impressive. It’s the first GAN I have seen which lets you tweak the result in a meaningful way rather than being just random. I think the speed that GANs have come in to the world has really shaken people up and it’s hard to process what this all means and what it will result in. Especially the ones which generate based on real people. But the feeling this gives me, is what happens for the future of art. Sure, this example is no where even close to replacing real artists, but it’s already generating images better than I can draw after a year of practice. It does give me a feeling of “what is the point”. Which might be an irrational feeling, but I’m sure others feel the same.
- zacmps 5y ago
  We have been here before, and we certainly will again. It was not so long ago computers bet humans at chess, yet people still play.
  - Gigachad 5y ago
    Yeah I totally get this is the same thing that has happened so many times before. But at least for chess, it was never a practical thing and was a competition. Like how the existence of aimbots does not ruin a game as long as you are playing against other humans. For art it feels a bit different since it’s not competitive and more a practicality thing. Perhaps art will shift from placing individual strokes on an image and move to making creative directions for AI to resolve in to an image or enable more people to create labor intensive works like animation.
  - _trampeltier 5y ago
    Or with the internet , Linux and open source. Suddenly you could downloads 1000s of apps, most much better, what I ever could do byself.
  - DavidPiper 5y ago
    I've been thinking about this a bit lately, incomplete thoughts ahead... Yes, people still play, but they no longer create. With the exception of Adversarial attacks on particular algorithms, no human is creating new Chess theory, discovering new openings, for example. As a game, challenge, competition, social activity, chess is alive and well. As a creative endeavour, or vehicle for discovery, Chess is solved. It is no longer an art of its own. We're part way through this transition now with Go as well. New opening theory, new joseki, new strategies are being played by robots, and at the highest professional levels we are playing catch-up to understand.
deft 5y ago
I made a waifu, then I pressed save. It wanted me to signup so I pressed back so I could just rightclick saveas but... I lost my waifu forever now :(
- jimmygrapes 5y ago
  This is like the condensed story of humanity
- GoblinSlayer 5y ago
  Waifu is kept in memory, normies.
- tsukikage 5y ago
  "Keep precious things inside you, or you will lose them"
- twic 5y ago
  It's okay, Step 43636 will come to console you in your dreams.
- keewee7 5y ago
  This is why we need local self-hosted AI. Keep the waifus safe.
- jcun4128 5y ago
  Could have been an NFT
- satronaut 5y ago
  comment of the year, and it's not even friday
- ASalazarMX 5y ago
  I mourn for your lost perfect waifu, but if it helps, your comment likely saved other waifus because we clicked [Download] instead of [Save].
echelon 5y ago
This is mind-blowingly good. You keep pushing the state of the art further to the point of broad applicability. It won't be long until everyone can be an artist without putting in the ten thousand hours of drudgery of training their muscles, hand-eye coordination, structure of shape and perspective, etc. I can't wait! Do you have a team page? How many of you are there? Do you work with gwern and nearcyan? Are you going to raise for this? (You should totally scale this!) Great work, and keep it up!
- Cixelyn 5y ago
  No team page yet! Friends with near, but haven't had a chance to meet gwern yet (maybe one day??) Arrowmancer is really our first attempt at scaling it up; hoping to do even cooler generative AI-related in that production.
  - echelon 5y ago
    All the best! Super excited!
dirtyid 5y ago
I hope these generators expands into non waifu / pretty boy anime depictions. There's a lot anime gaijin faces out there to explore.
- aniforprez 5y ago
  I would like to see how it generates late-80s/early-90s style features. The current pool of anime art styles are very generic (aside stylistic outliers) and I'd love to see Cowboy Bebop/Akira/Bubblegum Crisis type character designs
  - echelon 5y ago
    Miyazaki's style of character design and fantasy would be amazing.
    - neuronic 5y ago
      I'd be curious if such a GAN could actually beat Oda at generating new One Piece characters. 20 years into it he just doesn't seem to stop at creating hilarious characters.
    - the_af 5y ago
      Agreed. In many ways Miyazaki's style is "nonstandard" for anime, possibly because he was partly inspired by European artists (think Moebius, whose influences can be seen in Nausicaa for example).
- liuru 5y ago
  This one has some rough-looking ones! They're a bit rarer than most.
bmitc 5y ago
This isn't really creativity is it? I like to call stuff like this statistical copying, and indeed, the linked Wikipedia article on GANs says: > Given a training set, this technique learns to generate new data with the same statistics as the training set. There isn't a creative process here nor any creative introspection going on. While the technical results are impressive, this article does not address creativity even superficially, and just slaps the label on. There isn't any AI either. It's machine learning, i.e., statistical models and algorithms.
- liuru 5y ago
  read the bottom! The part about creativity is on the bottom :D
- zaik 5y ago
  > There isn't any AI either. It's machine learning, i.e., statistical models and algorithms. I always thaught those were synonyms.
  - xcambar 5y ago
    Both the A and the I in AI are debatable terms. The sole definition of intelligence is under heavy load of reconsideration the last decades with the emergence of a better knowledge of animal cognition, for example.
    - kortex 5y ago
      Why is the A[rtificial] under debate? It's not natural (biological) intelligence, unless one goes the route "humans are a part of nature therefore the things we make like plastic and skyscrapers and hamburgers are natural".
      - xcambar 5y ago
        It needs to be wondered whether drawing a face or winning a game of go may or may not actually be real intelligence, only very narrow. We like to think that AI is only statistical models, which may be true (is true?), but we have no idea that what happens in brains is anything else than a highly optimised, deconstructed, such model. Current datasets need 10ks pics of a cat while a kid only need to see a cat twice? Good for the kid, but there's no hint there's a fundamental difference of process.
aimor 5y ago
What methods are there to estimate how many unique characters a model can generate? The answer is not infinitely many, but determining when two images are of different 'characters' is fuzzy.
- gwern 5y ago
  It's hard to say, but I think a useful measure would be to look at mode-dropping compared to the training data. Whatever the 'number of unique characters is', it clearly ought to be at least as large as the characters you see in the original training data, right? For TADNE, Arfafax ran Danbooru2019 and a few million TADNE samples through CLIP to get the image embeddings, and clustered them; when the two sets of clusters were graphed using tsne, you could see that the TADNE StyleGAN2-ext did a lot of mode-dropping in that many smaller outlying clusters of characters/franchises/topics simply did not appear in TADNE samples. The TADNE looked like a big galaxy, while Danbooru2019 looked more like it was surrounded by archipelagos. TADNE was extensively trained on them and was a very large model, but the GAN dynamics & StyleGAN architecture mean it didn't do a good job absorbing rarer/more idiosyncratic Danbooru2019 image-clusters. I expect newer generative models which avoid GAN losses and which use more flexible (but expensive!) architectures, like DALL-E, would perform much better in terms of mode-dropping, so you'd see a lot more unique characters/images out of them. (I'm very excited about them. As good as TADNE or Waifu Labs v2 may be, I think they are still far behind what could be done with just existing data/arch/compute.)
tasha0663 5y ago
> Step 43636: During this phase, the training gets unstable at times, so we have snapshots of occasional horrors like this. Ah, make that three things the public shouldn't see being made: sausage, legislation, and waifus.
jakey_bakey 5y ago
I wish I had the stones to call my company Waifu Labs
- SavantIdiot 5y ago
  I didn't know what you meant so I googled it. The technology is cool, but the content is ... problematic. From urban dictionary: "Waifu" is used to refer to a fictional girl or woman (usually in Anime, Manga, or video-games) that you have sexual attraction to, and you would even marry. Huh.
  - kadokaelan 5y ago
    youd be surprised at the waifu market size
  - harpersealtako 5y ago
    A big part of it you're missing is that it's a joke. Anime fans definitely know they're weird, and are very passionate about the things they like (and are conscious of that), and thus a lot of humor in the community is self-deprecating and ironic (for example, calling your favorite fictional character your "waifu" or "husbando"). The fact that outside observers might think it's "problematic" is kind of the whole point.
akomtu 5y ago
This tool has many applications, and those that will make you rich isn't about anime.
- peoplefromibiza 5y ago
  step 1: generate random waifu step 2: NFT all the things step 3: profit step 4: GOTO step 1 step 5: automate steps 1 to 4
- sandos 5y ago
  Oh, I see what you are saying.... More things for, like, adults?
- scollet 5y ago
  It needs to make money?
suction 5y ago
- temptemptemp111 5y ago
- Ygg2 5y ago
  Painting with a brush stroke set to galaxy levels. I'm not saying there isn't creepy anime, or even creepy motifs in excellent anime. But that stuff sells over there.
  - nautilius 5y ago
    I'm not assessing whether it is creepy or not, but how can 'it sells' have any place in that argument?
    - Ygg2 5y ago
      Because if you are a mangaka/anime artist, you'll be pressured to add otaku stuff to boost sales.
      - nautilius 5y ago
        And then -- pedophile content is ok for you? Wow.
        Ygg2 5y ago
        You say pedophile, they say legal age of consent. I think it's different set of mores. Although it's complicated the age of consent depends on prefecture, age and circumstances and it's around 13 year old. And anime throw wrenches by having people's apparent age being different than their actual age. Take for example ReZero, where female heroine is 118 year old elf that looks like an average 17-18 year old, but has the same knowledge as a 12 year old human. This of course squicks out main protagonist who's about 17. Even though by applying our social mores, he'd be in the right. Different mores, different opinions. I honestly don't care too much about them. They are most often just stupid thing for Otakus to obsess about. And you are very fixated on pedo content, when the Otakus content is much, much broader see (and just as degenerate) - loli yuris, incest, BDSM, any combinations of previous, etc. And these are just tips of the shitberg.
w_t_payne 5y ago
I found the most interesting part was the evocative comment about the 'vast and parched' nature of the latent space. I wonder if the OP's intuition regarding the sparseness of the latent space, and the relatively small area occupied by the 'useful' manifold? embedded within it provide us any clues as to what symbol grounding might look like for some neuro-symbolic infrastructure that sits atop that latent space. I.e. how should we be trying to represent concepts like 'male' and 'female' within that space? Is it important to have these concepts represented as a low dimensional manifold? Is it important that this manifold be easily described by some simple geometric form like a convex polytope? Is it important that nuances and variations on the concept be separable within the bounds of the concept-specific manifold? What other properties might be important?
Kuinox 5y ago
It's funny how people complained about github copilot 'stealing' people code, but nobody here complained about this AI 'stealing' artworks. Don't get me wrong, I have nothing against this, but I think we should start discusing morality of AI generated content, even if it doesn't train on existing artworks/code.
- deleted 5y ago
  [deleted]
7373737373 5y ago
Have you considered applying similar models to VR avatar creation? That's a market in itself
Tade0 5y ago
Amazing work and progress. The previous version appears almost toyish in comparison. FYI uBlock Origin complains about the registration link, because it on "Peter Lowe’s Ad and tracking server list".
- pgl 5y ago
  That's because it bounces the registration link through a tracker. If you're OK with being tracked, you can permanently allow that domain.
jcun4128 5y ago
I would be interested to find one I like and use it for a desktop companion project, not an original idea but I am not an artist ha.
syntheweave 5y ago
The image quality is good, but now I realize I'm experiencing "uncanny waifu". Authentic character designs bear two things in common: 1. Simplifications of reality(the actual artist training method would be traditional studies off life and photo reference followed by gradual reduction and symbolization to a style) 2. Symbolic meaning. Things like the style of eyes, clothing, etc are all meant to signal personality. This is stuff that current AI techniques don't really touch upon in any direct sense. Since the ML method is built on interpolating off final results, it's going to lack in these qualities and produce something that is consistently an "average impression". Akin to asking the algorithm to generate mythical heroes by mashing up the various stories: you get a hero that is somehow the average of Icarus, Heracles and Achilles, which would be less of a character than the originals.
- someone7x 5y ago
  Could it work backwards? Eg, to take a hero like Heracles and determine he's say 40% shared with Gilgamesh. Then we might see that even the originals aren't very unique. Just a thought, I don't really know anything about ML.
rambojohnson 5y ago
project name is kinda incel cringe :(
dmix 5y ago
So putting some Anime digital illustrators out of work? ....or I could see many simply use them and pretend they did it themselves? Obvioisly there will be plenty of illustrators doing custom work that these can't (yet) replicate. Also good for those countless anime avatar'd Twitter users.
ausbah 5y ago
any good info on manipulating the "control vectors" in the latent space?

455 points by Cixelyn 5y ago

Cool, it's like an AI-generated picrew

Hey HN, one of the team members here! I hope you all enjoy playing with the new and improved generator! We've been hard at work improving the model quality since the last time the site was posted[1] As both a professional fantasy illustrator & software engineer, I find the concept of AI creativity so fascinating. On one hand, I know that mathematically AI only can hallucinate images that fit within the distribution of things that it's seen. But from the artist perspective, the model's ability to blend two existing styles into something so distinctly new is so incredible (and not to mention also commercially useful!) Anyways, happy to answer any question, thoughts, or concerns! --- [1] https://news.ycombinator.com/item?id=20511459 https://news.ycombinator.com/item?id=20511459]

hypertele-Xii 5y ago
Why do stuff like this never come down from the web? I'd pay for a program I could download and use with my own image files.
- liuru 5y ago
  While our model is not public, there are good resources online for playing with your own images! Like this one by fast.ai! https://docs.fast.ai/vision.gan.html https://docs.fast.ai/vision.gan.html
  - sillysaurusx 5y ago
    - spi 5y ago
      Except that then anybody could literally just download it and start a competing service saving 2 years of development and hundreds of thousands of $ in compute costs over that time?
      - sillysaurusx 5y ago
        No, they couldn’t. And I guarantee you they spent nowhere near that sum to train the model, and that it wouldn’t take two years to clone it.
- Afforess 5y ago
  Same reason the Coca-Cola recipe is not published nor made freely available by the Coca-Cola corporation.
- Gigachad 5y ago
  They tend to require specific hardware like a NVIDIA GPU. As well as having an ever evolving large model file which they will want to frequently update. Some tools certainly have had offline versions but I guess not many people are interested in setting it all up and are happy with an instant web ui
- zozbot234 5y ago
  You just need to code up your own model architecture and then train it on your data using some established ML framework. The first step is where well-chosen priors can make a real difference wrt. your end results.
dimgl 5y ago
You mention it took two weeks to get to the point that we see in the article. Does this mean two weeks of development, or two weeks to generate the images we're seeing? Or maybe did you train the model for two weeks? That point just wasn't exactly clear for me.
- liuru 5y ago
  2 weeks to train the model! Development took on-and-off roughly 2 years to achieve the quality you see today.
  - dimgl 5y ago
    Cool! Might want to clarify that. This is crazy impressive!
    - liuru 5y ago
      Got it! Thank you! :D
darkengine 5y ago
Is the code or any of the models available to the public? I'd love to mess with this on a local GPU cluster.
- liuru 5y ago
  Not at the moment! A similar project that I really admire is public, though! https://www.thiswaifudoesnotexist.net/ https://www.thiswaifudoesnotexist.net/
2bitencryption 5y ago
Firstly, amazing work. My question is, how do you figure out how to parameterize "Same character, different pose" / "Same character, different eyes" / "Same character, different gender" / etc? My (super limited) understanding of GANs is that they slowly discover these features over time simply from observation in the data set, and not from any labels. So how could you make e.x. a slider for head position, style, pose, etc? How do you look at the resulting model and figure out "these are the inputs we have to fiddle with to make it use a certain pose"? You mention it a bit in this section, but I didn't fully understand: "By isolating the vectors that control certain features, we can create results like different pose, same character" And I assume the same step needs to be done every time the model is retrained or fine-tuned, because possibly the vectors have shifted within the model since they are not fixed by design?
- liuru 5y ago
  Yes, your understanding is correct! You can think of it like coordinates on a many-dimensional vector grid. We craft the functions the functions that will illuminate sets of those points based on a combination of observation, what we know about our model architecture, and how our data is arranged. And yes, when the model is retrained, we have to discover them again!
  - chrisdsaldivar 5y ago
    Can you share any resources for reading on this particular topic?
- flor1s 5y ago
  Not affiliated with this project, but there is a gazillion different variations of GANs. Most just change the adversarial loss to improve the learning rate / quality, but others focus on architectural changes, such as StarGAN, Pix2pix (conditional GAN), CycleGAN, MUNIT, etc. It's really a fascinating field.
rackjack 5y ago
I LOVE that "horror". Reminds me of some of the art I've seen on album/single covers. Any chance of letting people access that kind of intermediate step? (Though I know it's a niche as hell use case).
- liuru 5y ago
  Ah yes, the fine line between charming anime character and lovecraftian horror There was such popular demand for these "horror" images that we made them part of the generation in V2! If you refresh enough on the webpage, you can find some horrors!
  - rackjack 5y ago
    For anyone looking to do this, here's some I made: https://i.imgur.com/1V1wPMC.jpg https://i.imgur.com/1V1wPMC.jpg I rolled around 40 times on the first stage and chose a horror. I skipped the second stage and didn't roll on the third stage, just chose one of the presented details. I skipped the fourth stage because I rolled maybe ~200 times and only saw around 1-2 comparable horrors. https://i.imgur.com/1vBeg1j.jpg https://i.imgur.com/1vBeg1j.jpg I rolled around 100 times on the first stage and saw about 3-4 horrors before choosing a horror. I rolled about 70 times on the second stage, didn't find anything interesting, just chose a normal color palette. I rolled maybe 80 times on the third stage and chose a horror, though the results were pretty consistently horrors. I rolled around 60 times on the fourth stage and saw about 1-2 other horrors before choosing a horror. Also, here's what it looked like after the third stage and before finishing the fourth stage: https://i.imgur.com/ditm8nF.jpg https://i.imgur.com/ditm8nF.jpg It's possible the third and fourth stages can produce horrors from normal faces, I didn't check.
kregasaurusrex 5y ago
I purchased a waifu from your vending machine (loved the blog post!) at Gen Con in 2019, but can't see the saved model in my account. Is there a way for me to get a v2 generation?
- liuru 5y ago
  Welcome back! We're currently working on the data migration from V1! As long as you are using the same email as you did in 2019, you'll be able to see the image again! As for a V2 generation, sorry, because the models are different, you'll have to discover a similar image again, if you want a V2 version!
  - kregasaurusrex 5y ago
    Ah that's alright then, thanks for the quick response. I was so happy to see your guys' booth there and own an obscure piece of internet history! https://storage.cloud.google.com/waifus-images/6b94c2ea-51be-49b5-ab16-5ba3ca0ef133.png https://storage.cloud.google.com/waifus-images/6b94c2ea-51be...
  - natch 5y ago
    Can't you use projection with the original image as input? Not for an exact copy, of course, but for a similar V2 rendition?
kouteiheika 5y ago
What are the terms of use for the images generated through your website? I'm guessing any commercial use is forbidden? It would be nice if you could formally spell it out on the website.
- JetAlone 5y ago
  I don't think there's any powerful enough way to stop people from generating one and then tracing over it to create their own linework, and customize things like the colouring and shading. The more broadly AI is able to create, the more niche and obfuscated directions human co-creators could take its products in.
  - marcan_42 5y ago
    Not to mention the copyrightability of AI output isn't legally well tested, and chances are it'll fall in the direction of being copyrighted by the user (i.e. the person clicking in the UI to make a character, which is where creative input happens), not by the AI creator (who has no creative input into the process; they merely made a tool, like any other piece of software - same reason documents printed from Microsoft Word aren't copyrighted by Microsoft). I'm not entirely sure how much legal weight a ToS on the website would have on what the users do with the output. As I understand it, you could e.g. forbid explicitly using the service/generator for commercial purposes (e.g. during game development), but if someone generates a cool character playing around with no particular commercial objective and then decides post facto to build a media megafranchise out of that character, absent any copyright claim over the image, I don't think there's anything stopping them. They wouldn't even need to trace over it, though if they want new artwork in different poses, they couldn't keep using the AI for that with explicit commercial intent; they'd have to get humans to re-draw it. Alternatively, a pessimistic view of the interaction between copyright and AI would be that the model is a derivative work of all the training input, and its output also is, and then good luck building a non copyright infringing AI. IANAL and all that, but it would definitely be legally risky to assume that as the provider of an AI generator you have any control over what users do with the output.
    - 5y ago
searchableguy 5y ago
Do you plan to provide an API to generate waifu? I think I could use this for a project.
- liuru 5y ago
  In the future, perhaps! This is a popular request, so we are thinking about ways we can do this.
unobatbayar 5y ago
The quality and style is mindblowing! What data did you train?
- Bombthecat 5y ago
  The more interesting question: What about the sources for the AI to train? How are those artists paid? Do we need to pay them? Or if it used by an AI as train data, we just say: Its like a human learning?
  - Gigachad 5y ago
    There is nothing that suggests it should be any different than human learning legally. As long as the output is significantly unique, it shouldn’t have any copyright issues.
    - nonbirithm 5y ago
      Does it matter if the artist themselves rejects the idea though? People were discussing having a "no machine learning clause" back when Copilot was being heavily scrutinized. I wouldn't be surprised if some artists allow republishing but not machine learning use. Plenty of artists already have a clause that prohibits any kind of republishing, and Danbooru is known to rehost some of those artists' content anyway until the artist notices and requests that it be taken down, if ever (for a time, they even allowed paid rewards from Patreon and other subscription services to be republished). The original dataset from Danbooru probably contained some percentage of content that would not have been there if the original artist had noticed in time.
      - nybble41 5y ago
        > Does it matter if the artist themselves rejects the idea though? Why would it, if there are no copyright issues? No one is obligated to accept a license unless they require permission to do something which copyright would restrict. Of course redistributing the original image as part of a public dataset may be problematic, but simply using it to train an AI model—essentially the equivalent of studying it while teaching yourself to draw—is arguably not among the things covered by copyright, so you don't need a license for that and any clauses in such a license would be irrelevant. This is also basically educational in nature, even if it's a machine rather than a human being "educated", and educational use is often exempt from copyright in some degree or another to begin with. If the dataset is restricted to non-commercial research and educational use in the right jurisdictions then even redistribution may not be an issue.
wodenokoto 5y ago
Naïvely I thought Waifu generator was just “some guy having a laugh” fine-tuning a model off of hugging face, but reading through the comments here, it is obviously a much, much bigger enterprise. Can you talk a little about team size, work process, funding and revenue stream? I think the effort required for such an undertaking is vastly underestimated by readers.
- hansel_der 5y ago
  > Naïvely I thought Waifu generator was just “some guy having a laugh” same here. what's naive about it? not to badmouth the undertaking, but wtf is this doing on HN?
  - wodenokoto 5y ago
    Apparently it takes 6 people making a business to run a waifu generator. That pretty far from one person doing it as a joke.
    - hansel_der 5y ago
      well, apparently... i sincerely applaud the creativity of establishing such a business
- Cixelyn 5y ago
  Right now it's a small team of 6 people, and we have a bit of funding + compute credits to train models. There's a bit of revenue from some past projects and AI-consulting, but we're mostly betting big on our new AI-powerd mobile title Arrowmancer[1]. > I think the effort required for such an undertaking is vastly underestimated by readers. Haha for sure. Hosting a real-time ML model for people to do sub 1-second inferences at HN-load scale is definitely nontrivial. [1] https://arrowmancer.com https://arrowmancer.com
  - KronisLV 5y ago
    That seems like a nice project that you're working on, definitely more high effort than some other attempts at generated art (procedurally or otherwise). I can't help but to feel that this would be a better fit for the fad of NFTs as well, as opposed to ugly monkeys or other asset flips that were pretty obvious cash grabs. Either way, good luck!
ridaj 5y ago
Fascinating... Thanks for sharing A couple questions: 1) I didn't really understand how you went about identifying what vectors of the latent space stand for various things, like pose or color. Did you train one of the AIs to that effect, or did you manually inspect a bunch of vectors, twiddling through them one by one, did to the outcome? 2) If one were to train an AI to the same level using commodity cloud services, what's the order of magnitude cost that you would pay for the training? More like $100, $1,000, $10,000 or $100,000?
- liuru 5y ago
  1) It was mostly manual, though AIs were useful in certain filtering tasks. 2) Depends on the quality you are seeking. If you only want one run of a similar, off-the-shelf model, around the 1000s is enough. But at the number of iterations you have to run to build your own and improve results, you probably need about 100k. To tackle this problem, we built our own supercomputer off of parts we bought off of ebay, though I can't say I recommend that route, because it now lives in our living room.
  - wnkrshm 5y ago
    Very curious about the computer, what are the internals?
    - Cixelyn 5y ago
      Will probably blog probably in a week or two. But the tldr; is a lot of scavenged EPYCs and nvidia GPUs all in a large sound-proof rack.
    - Filligree 5y ago
      Not the OP, but my guess would be threadrippers (or similar w/lots of PCIe lanes), each with a great number of GPUs. That's usually what you'd do for training AI in a home lab. Server processors gets you more bang for the buck... iff you're planning to run the hardware flat out for literally years. You save on power, but the up-front cost takes up most of that, so for a system that's mostly idle you wouldn't use them. On the other hand, any CPU with fewer PCIe lanes than a TR won't be able to run multiple GPUs optimally, and TRs are relatively cheap enough to make the reduction in PSUs/chassises worth it. Not to mention that there are some approaches to training you can only use if you have multiple GPUs on the same motherboard, aka. sharding a single model across GPUs without communications overhead killing any benefit of that.
thyrox 5y ago
Roughly speaking how much money did you invest into making this? Just curious if this is something an indie hacker can hope to do one day OR do you need some deep pockets to make a site like this?
Cthulhu_ 5y ago
With the game you're building, are the character portraits generated once and that's it, or do you plan on making them dynamic or frequently updated? I've seen a number of mobile games that just get flooded with characters; this tool looks like it could be used to automate that process. It could be combined with AI-generated character profiles as well, creating an 'infinite' character roster in video games.
GoblinSlayer 5y ago
Would you try to create a new style? Train the discriminator on the score tag of danbooru dataset, then use it to rate the generator's style, this way it should be able to create a new style.
Terry_Roll 5y ago
I wonder what an AI trained to spot deepfake Waifu's will detect. In humans, things like the pupil can be the give away. https://www.newscientist.com/article/2289815-ai-can-detect-a-deepfake-face-because-its-pupils-have-jagged-edges/ https://www.newscientist.com/article/2289815-ai-can-detect-a...
- yccs27 5y ago
  This is a super interesting question, given that the generator model is trained to fool the discriminator, which is also an AI.
- oneoff786 5y ago
  Highlight the pixels with high sharp values. Should be doable.
CuriousCosmic 5y ago
Something I was wondering but couldn't find on the site: What is the license for the generated works through the project?
YeGoblynQueenne 5y ago
Hello and thank you for answering questions. The following is a quote from your article: >> It is interesting to note that from this process, the AI is not merely learning to copy the works it has seen, but forming high-level (shapes) and low-level (texture) features for constructing original pictures in its own mental representation. Can you explain what you mean by "mental" representation? Does your system have a mind? Also, why are you calling it "an AI"? Is it because you think it is an artificial intelligence, say like the robots in science fiction movies? Is it capable of anything else than generating images?
- xg15 5y ago
  Not OP, but I wonder if the process would be in some way comparable to rigging a 3D model. There is well, you usually have some high-level input parameters, which influence joints on a predefined skeleton, which in turn determines the position of individual vertices in the 3D body. Finally, the 3D shape is used to render the actual pixels. On each step, high-level parameters are combined with predefined weights to produce a more low-level output. Seems, a similar transformation is going on here, except that the weights and the structure are somehow learned on its own.
simonebrunozzi 5y ago
So neat! Where are you based? Boston, I assume? Is there an email to reach out to you or someone in the team? ($HNusername @ gmail)
- Cixelyn 5y ago
  San Francisco! Just sent over a ping!
tedmcory77 5y ago
Who would someone speak with about licensing things made using waifu? My email contact is in my profile...

kadokaelan 5y ago

The explainer video about gans is top-notch! Excited for Arrowmancer!

Cixelyn 5y ago
(link for the lazy: https://youtu.be/Pab8pG5WbXQ https://youtu.be/Pab8pG5WbXQ) Thanks so much! It's done by our fantastic animator[1]! GANs are quite interesting and we didn't see many approachable explainer videos targeted at lay people, so we decided to make one ourselves! [1] https://twitter.com/bumblingbeebo https://twitter.com/bumblingbeebo
- Bombthecat 5y ago
  Is there a gameplay video? Or at least screenshots of arrowverse?
  - Cixelyn 5y ago
    Here's our current game trailer! https://www.youtube.com/watch?v=8WvRgb6kh4s https://www.youtube.com/watch?v=8WvRgb6kh4s
slimsag 5y ago
For the author: there's a small typo "Discrimniator" instead of "Discriminator" in the video at 1:11 One thing I was confused by: the video says the discriminator "AI" is trained to detect true vs. generated results, with the hope the generator becomes good enough to fool the discriminator. But why is the discriminator useful, then? Couldn't you just tell generator "AI" whether the result it produced was true or not? I think the answer is.. you don't want just a perfect recreation of the training data you gave to the generator, instead you want the generator to produce variations of that training data, so there's a "how would you know if it's 'a true result' / good enough?" problem. So the discriminator is useful because it's not a direct comparison, but rather a "this looks approximately good enough" comparison of the true vs. generated result. This all makes me wonder: what sort of data set needs to be fed to the discriminator to train it? Is it some sort of "true image" and "true image w/bad alterations (e.g. lines, scratches, etc.) to it" data set?

ve55 5y ago

Links to related projects in anime art generation for those interested: Waifu Labs v2, referenced in this post (generate amazing custom anime face images): https://waifulabs.com https://waifulabs.com (write-up is the above link: https://waifulabs.com/blog/ai-creativity https://waifulabs.com/blog/ai-creativity) This Anime Does Not Exist (AI-generated anime-style artwork): https://thisanimedoesnotexist.ai https://thisanimedoesnotexist.ai (write-up https://www.gwern.net/Faces#extended-stylegan2-danbooru2019-aydao https://www.gwern.net/Faces#extended-stylegan2-danbooru2019-... and https://nearcyan.com/this-anime-does-not-exist https://nearcyan.com/this-anime-does-not-exist) This Waifu Does Not Exist (AI-generated anime-style faces): https://thiswaifudoesnotexist.net https://thiswaifudoesnotexist.net (write-up: https://www.gwern.net/Faces#twdne https://www.gwern.net/Faces#twdne) There's also a lot of literate on e.g. automatic manga coloration, auto-translation, image superresolution, anime frame interpolation, and much more. Worth checking out some places like https://old.reddit.com/r/AnimeResearch/ https://old.reddit.com/r/AnimeResearch/ if you're interested!

the8472 5y ago
Note that TADNE, being based on newer models, generates full images rather than only faces.
numpad0 5y ago
This one too https://twitter.com/t_takasaka/status/1479784432697749513 https://twitter.com/t_takasaka/status/1479784432697749513

spywaregorilla 5y ago

Is there any way to approximate embeddings for a novel image? novel meaning user provided, not generated by the model or in the training set.

liuru 5y ago
Not at the moment in our tool, though this is an area of great curiosity and research for us!
- spywaregorilla 5y ago
  Does the discriminator model translate the images into an embedding space of it's own? Could such a space be used to generate images themselves?

mensetmanusman 5y ago

I can’t wait for this technology to come to video. Imagine a future where people can compile written scripts into Hollywood quality movies.

thewarrior 5y ago
It's already begun https://twitter.com/somnai_dreams/status/1477411531037937664?s=21 https://twitter.com/somnai_dreams/status/1477411531037937664...
- FeepingCreature 5y ago
  I wonder what will happen when somebody combines a GAN with a feature recognizing network like the Tesla cars use, so it can use its own extrapolated map of the surroundings to stabilize its output as the camera moves around.
deleted 5y ago
[deleted]
JetAlone 5y ago
I think it will have its limits, but the possibilities for editing together, supplementing and modifying products from smaller AI modules should stretch out what you can do on a small budget.

ps91 5y ago

Amazing now I can create my own anime girl!

soolinder 5y ago

KaoruAoiShiho 5y ago

I don't want to go all SJW on you guys, amazing work, but can you try to make sure there's an inclusive array of starting faces please? Talking about things like skin tones, thanks!

deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
liuru 5y ago
Indeed, we spent 2 years working on this!!! It's an extremely hard research problem, because darker skin tones account for only about 0.3% of all anime art produced in the world. We have employed an absolutely exhaustive array of art and data science tricks to give the model the ability to draw darker skin tones, though they are underrepresented. The results that you see today are the culmination of many months of careful tuning! It's not definitely perfect, but from a data science perspective, this situation can't be rectified until the art world makes a shift. Personally, I hope that more art representing dark skin tones will be created in the world!
- KaoruAoiShiho 5y ago
  Thanks, happy to hear that you guys are on top of it.
- pfisherman 5y ago
  I can’t understand why the GP comment is flagged. If you can look past all the “culture war” stuff, this is pointing out some of the limits of algorithmic creativity. It does not do well generating instances with features that are not well represented in the training dataset. Compare this to human creativity. I suspect that fulfilling GPs request would be almost trivial for a human professional artist. To be clear this is an amazing achievement, a creative use of the technology, and a positive contribution to the world. Pointing out limitations (i.e. areas with potential for future innovation) does not diminish it.

deleted 5y ago

[deleted]

Gigachad 5y ago

This is extremely impressive. It’s the first GAN I have seen which lets you tweak the result in a meaningful way rather than being just random. I think the speed that GANs have come in to the world has really shaken people up and it’s hard to process what this all means and what it will result in. Especially the ones which generate based on real people. But the feeling this gives me, is what happens for the future of art. Sure, this example is no where even close to replacing real artists, but it’s already generating images better than I can draw after a year of practice. It does give me a feeling of “what is the point”. Which might be an irrational feeling, but I’m sure others feel the same.

zacmps 5y ago
We have been here before, and we certainly will again. It was not so long ago computers bet humans at chess, yet people still play.
- Gigachad 5y ago
  Yeah I totally get this is the same thing that has happened so many times before. But at least for chess, it was never a practical thing and was a competition. Like how the existence of aimbots does not ruin a game as long as you are playing against other humans. For art it feels a bit different since it’s not competitive and more a practicality thing. Perhaps art will shift from placing individual strokes on an image and move to making creative directions for AI to resolve in to an image or enable more people to create labor intensive works like animation.
- _trampeltier 5y ago
  Or with the internet , Linux and open source. Suddenly you could downloads 1000s of apps, most much better, what I ever could do byself.
- DavidPiper 5y ago
  I've been thinking about this a bit lately, incomplete thoughts ahead... Yes, people still play, but they no longer create. With the exception of Adversarial attacks on particular algorithms, no human is creating new Chess theory, discovering new openings, for example. As a game, challenge, competition, social activity, chess is alive and well. As a creative endeavour, or vehicle for discovery, Chess is solved. It is no longer an art of its own. We're part way through this transition now with Go as well. New opening theory, new joseki, new strategies are being played by robots, and at the highest professional levels we are playing catch-up to understand.

deft 5y ago

I made a waifu, then I pressed save. It wanted me to signup so I pressed back so I could just rightclick saveas but... I lost my waifu forever now :(

jimmygrapes 5y ago
This is like the condensed story of humanity
GoblinSlayer 5y ago
Waifu is kept in memory, normies.
tsukikage 5y ago
"Keep precious things inside you, or you will lose them"
twic 5y ago
It's okay, Step 43636 will come to console you in your dreams.
keewee7 5y ago
This is why we need local self-hosted AI. Keep the waifus safe.
jcun4128 5y ago
Could have been an NFT
satronaut 5y ago
comment of the year, and it's not even friday
ASalazarMX 5y ago
I mourn for your lost perfect waifu, but if it helps, your comment likely saved other waifus because we clicked [Download] instead of [Save].

echelon 5y ago

This is mind-blowingly good. You keep pushing the state of the art further to the point of broad applicability. It won't be long until everyone can be an artist without putting in the ten thousand hours of drudgery of training their muscles, hand-eye coordination, structure of shape and perspective, etc. I can't wait! Do you have a team page? How many of you are there? Do you work with gwern and nearcyan? Are you going to raise for this? (You should totally scale this!) Great work, and keep it up!

Cixelyn 5y ago
No team page yet! Friends with near, but haven't had a chance to meet gwern yet (maybe one day??) Arrowmancer is really our first attempt at scaling it up; hoping to do even cooler generative AI-related in that production.
- echelon 5y ago
  All the best! Super excited!

dirtyid 5y ago

I hope these generators expands into non waifu / pretty boy anime depictions. There's a lot anime gaijin faces out there to explore.

aniforprez 5y ago
I would like to see how it generates late-80s/early-90s style features. The current pool of anime art styles are very generic (aside stylistic outliers) and I'd love to see Cowboy Bebop/Akira/Bubblegum Crisis type character designs
- echelon 5y ago
  Miyazaki's style of character design and fantasy would be amazing.
  - neuronic 5y ago
    I'd be curious if such a GAN could actually beat Oda at generating new One Piece characters. 20 years into it he just doesn't seem to stop at creating hilarious characters.
  - the_af 5y ago
    Agreed. In many ways Miyazaki's style is "nonstandard" for anime, possibly because he was partly inspired by European artists (think Moebius, whose influences can be seen in Nausicaa for example).
liuru 5y ago
This one has some rough-looking ones! They're a bit rarer than most.

bmitc 5y ago

This isn't really creativity is it? I like to call stuff like this statistical copying, and indeed, the linked Wikipedia article on GANs says: > Given a training set, this technique learns to generate new data with the same statistics as the training set. There isn't a creative process here nor any creative introspection going on. While the technical results are impressive, this article does not address creativity even superficially, and just slaps the label on. There isn't any AI either. It's machine learning, i.e., statistical models and algorithms.

liuru 5y ago
read the bottom! The part about creativity is on the bottom :D
zaik 5y ago
> There isn't any AI either. It's machine learning, i.e., statistical models and algorithms. I always thaught those were synonyms.
- xcambar 5y ago
  Both the A and the I in AI are debatable terms. The sole definition of intelligence is under heavy load of reconsideration the last decades with the emergence of a better knowledge of animal cognition, for example.
  - kortex 5y ago
    Why is the A[rtificial] under debate? It's not natural (biological) intelligence, unless one goes the route "humans are a part of nature therefore the things we make like plastic and skyscrapers and hamburgers are natural".
    - xcambar 5y ago
      It needs to be wondered whether drawing a face or winning a game of go may or may not actually be real intelligence, only very narrow. We like to think that AI is only statistical models, which may be true (is true?), but we have no idea that what happens in brains is anything else than a highly optimised, deconstructed, such model. Current datasets need 10ks pics of a cat while a kid only need to see a cat twice? Good for the kid, but there's no hint there's a fundamental difference of process.

aimor 5y ago

What methods are there to estimate how many unique characters a model can generate? The answer is not infinitely many, but determining when two images are of different 'characters' is fuzzy.

gwern 5y ago
It's hard to say, but I think a useful measure would be to look at mode-dropping compared to the training data. Whatever the 'number of unique characters is', it clearly ought to be at least as large as the characters you see in the original training data, right? For TADNE, Arfafax ran Danbooru2019 and a few million TADNE samples through CLIP to get the image embeddings, and clustered them; when the two sets of clusters were graphed using tsne, you could see that the TADNE StyleGAN2-ext did a lot of mode-dropping in that many smaller outlying clusters of characters/franchises/topics simply did not appear in TADNE samples. The TADNE looked like a big galaxy, while Danbooru2019 looked more like it was surrounded by archipelagos. TADNE was extensively trained on them and was a very large model, but the GAN dynamics & StyleGAN architecture mean it didn't do a good job absorbing rarer/more idiosyncratic Danbooru2019 image-clusters. I expect newer generative models which avoid GAN losses and which use more flexible (but expensive!) architectures, like DALL-E, would perform much better in terms of mode-dropping, so you'd see a lot more unique characters/images out of them. (I'm very excited about them. As good as TADNE or Waifu Labs v2 may be, I think they are still far behind what could be done with just existing data/arch/compute.)

tasha0663 5y ago

> Step 43636: During this phase, the training gets unstable at times, so we have snapshots of occasional horrors like this. Ah, make that three things the public shouldn't see being made: sausage, legislation, and waifus.

jakey_bakey 5y ago

I wish I had the stones to call my company Waifu Labs

SavantIdiot 5y ago
I didn't know what you meant so I googled it. The technology is cool, but the content is ... problematic. From urban dictionary: "Waifu" is used to refer to a fictional girl or woman (usually in Anime, Manga, or video-games) that you have sexual attraction to, and you would even marry. Huh.
- kadokaelan 5y ago
  youd be surprised at the waifu market size
- harpersealtako 5y ago
  A big part of it you're missing is that it's a joke. Anime fans definitely know they're weird, and are very passionate about the things they like (and are conscious of that), and thus a lot of humor in the community is self-deprecating and ironic (for example, calling your favorite fictional character your "waifu" or "husbando"). The fact that outside observers might think it's "problematic" is kind of the whole point.

akomtu 5y ago

This tool has many applications, and those that will make you rich isn't about anime.

peoplefromibiza 5y ago
step 1: generate random waifu step 2: NFT all the things step 3: profit step 4: GOTO step 1 step 5: automate steps 1 to 4
sandos 5y ago
Oh, I see what you are saying.... More things for, like, adults?
scollet 5y ago
It needs to make money?

suction 5y ago

temptemptemp111 5y ago
Ygg2 5y ago
Painting with a brush stroke set to galaxy levels. I'm not saying there isn't creepy anime, or even creepy motifs in excellent anime. But that stuff sells over there.
- nautilius 5y ago
  I'm not assessing whether it is creepy or not, but how can 'it sells' have any place in that argument?
  - Ygg2 5y ago
    Because if you are a mangaka/anime artist, you'll be pressured to add otaku stuff to boost sales.
    - nautilius 5y ago
      And then -- pedophile content is ok for you? Wow.
      - Ygg2 5y ago
        You say pedophile, they say legal age of consent. I think it's different set of mores. Although it's complicated the age of consent depends on prefecture, age and circumstances and it's around 13 year old. And anime throw wrenches by having people's apparent age being different than their actual age. Take for example ReZero, where female heroine is 118 year old elf that looks like an average 17-18 year old, but has the same knowledge as a 12 year old human. This of course squicks out main protagonist who's about 17. Even though by applying our social mores, he'd be in the right. Different mores, different opinions. I honestly don't care too much about them. They are most often just stupid thing for Otakus to obsess about. And you are very fixated on pedo content, when the Otakus content is much, much broader see (and just as degenerate) - loli yuris, incest, BDSM, any combinations of previous, etc. And these are just tips of the shitberg.

w_t_payne 5y ago

I found the most interesting part was the evocative comment about the 'vast and parched' nature of the latent space. I wonder if the OP's intuition regarding the sparseness of the latent space, and the relatively small area occupied by the 'useful' manifold? embedded within it provide us any clues as to what symbol grounding might look like for some neuro-symbolic infrastructure that sits atop that latent space. I.e. how should we be trying to represent concepts like 'male' and 'female' within that space? Is it important to have these concepts represented as a low dimensional manifold? Is it important that this manifold be easily described by some simple geometric form like a convex polytope? Is it important that nuances and variations on the concept be separable within the bounds of the concept-specific manifold? What other properties might be important?

Kuinox 5y ago

It's funny how people complained about github copilot 'stealing' people code, but nobody here complained about this AI 'stealing' artworks. Don't get me wrong, I have nothing against this, but I think we should start discusing morality of AI generated content, even if it doesn't train on existing artworks/code.

deleted 5y ago
[deleted]

7373737373 5y ago

Have you considered applying similar models to VR avatar creation? That's a market in itself

Tade0 5y ago

Amazing work and progress. The previous version appears almost toyish in comparison. FYI uBlock Origin complains about the registration link, because it on "Peter Lowe’s Ad and tracking server list".

pgl 5y ago
That's because it bounces the registration link through a tracker. If you're OK with being tracked, you can permanently allow that domain.

jcun4128 5y ago

I would be interested to find one I like and use it for a desktop companion project, not an original idea but I am not an artist ha.

syntheweave 5y ago

The image quality is good, but now I realize I'm experiencing "uncanny waifu". Authentic character designs bear two things in common: 1. Simplifications of reality(the actual artist training method would be traditional studies off life and photo reference followed by gradual reduction and symbolization to a style) 2. Symbolic meaning. Things like the style of eyes, clothing, etc are all meant to signal personality. This is stuff that current AI techniques don't really touch upon in any direct sense. Since the ML method is built on interpolating off final results, it's going to lack in these qualities and produce something that is consistently an "average impression". Akin to asking the algorithm to generate mythical heroes by mashing up the various stories: you get a hero that is somehow the average of Icarus, Heracles and Achilles, which would be less of a character than the originals.

someone7x 5y ago
Could it work backwards? Eg, to take a hero like Heracles and determine he's say 40% shared with Gilgamesh. Then we might see that even the originals aren't very unique. Just a thought, I don't really know anything about ML.

rambojohnson 5y ago

project name is kinda incel cringe :(

dmix 5y ago

So putting some Anime digital illustrators out of work? ....or I could see many simply use them and pretend they did it themselves? Obvioisly there will be plenty of illustrators doing custom work that these can't (yet) replicate. Also good for those countless anime avatar'd Twitter users.

ausbah 5y ago

any good info on manipulating the "control vectors" in the latent space?