10 ms·
Artbreeder – Extend Your Imagination with GANs
- sharemywin 6y agoreally cool.
- joelS 6y agoHey! this is Joel, the maker of Artbreeder (originally called ganbreeder). The project was inspired by interactive evolutionary algorithms and novelty search (https://www.youtube.com/watch?v=dXQPL9GooyI&t=1s https://www.youtube.com/watch?v=dXQPL9GooyI&t=1s) The goal was to make a tool that reflected the collaborative and explorative aspects of creativity. Also to make GAN's (and high-dimensional spaces more generally) accessible to everyone in a fun way. Let me know if you have any questions :)
- j4nt4b 6y agoAwesome work! I love this concept, but I'm a writer who's interested in collaborative generative writing. Do you know of any efforts to create something similar for writing, such as poetry or micro-fiction? If not, I'm curious if you could point me in the right direction for putting something like that together.
- joelS 6y agoHey, thanks! I dont know about anything collaborative and for writing. https://aidungeon.io/ https://aidungeon.io/ Maybe collaborative to play. And then there are a lot of writers exploring fine-tuning text models like GPT2 for personal use. https://twitter.com/MagicRealismBot https://twitter.com/MagicRealismBot is one of many examples. https://towardsdatascience.com/how-to-fine-tune-gpt-2-so-you-can-generate-long-form-creative-writing-7a5ae1314a61 https://towardsdatascience.com/how-to-fine-tune-gpt-2-so-you... Maybe you should make it!
- j4nt4b 6y agoThank you!
- kleer001 6y agoIf you check out the archives of NaNoGenMo over at github you'll likely find what you're looking for: https://nanogenmo.github.io/ https://nanogenmo.github.io/ Check out the yearly resources.
- gwern 6y agoIf you'd like to write poetry or fanfiction, I provide GPT-2-1.5b models for both on https://www.gwern.net/GPT-2 https://www.gwern.net/GPT-2 - I don't provide a slick interface for writing (although Astralite has an upcoming web interface which is very snazzy), but they could be plugged into any such interface.
- MattRix 6y agoSomething like https://talktotransformer.com/ https://talktotransformer.com/ ?
- Jack000 6y agoa direct transfer of this concept to text isn't possible because the SotA for text generation is based on transformers, which do not work using latent space sampling. The directly analogous version of stylegan in the nlp world would be vq-vae, but currently it doesn't match transformers in generation quality. There are other ways to do generative writing though, mostly with prompts.
- archibaldJ 6y agoGreat work mate! Any plans of openning up an API for a subscription fee or something? :))
- joelS 6y agoThanks! No plans at the moment but it’s possible!
- sillysaurusx 6y agoHi Joel, Given that nVidia has a pretty clear license saying that their model and all derivates may only be used non-commercially (https://github.com/NVlabs/stylegan2/blob/cec605e0834de5404d5c7e5cead7053bdd0e4dde/LICENSE.txt#L60-L64 https://github.com/NVlabs/stylegan2/blob/cec605e0834de5404d5...) and Artbreeder is clearly commercial (https://artbreeder.com/pricing https://artbreeder.com/pricing) but is using StyleGAN (https://artbreeder.com/i?k=755521d9fe07456286a06f06 https://artbreeder.com/i?k=755521d9fe07456286a06f06), do you feel that Artbreeder is intentionally violating nVidia's license? For what it's worth, I fully support your project and I nearly launched something similar myself. But many people choose not to do projects such as yours due to licensing considerations, and I'm wondering in general how to reconcile the moral dilemma. On one hand, nVidia almost certainly doesn't care. On the other hand, they did say very clearly "thou shalt not use this model, or any derivative thereof, commercially." (It's tempting to believe the license only covers the code, not the model. But the same license appears in their download drive: https://drive.google.com/drive/folders/1QHc-yF5C3DChRwSdZKcx1w6K8JvSxQi7 https://drive.google.com/drive/folders/1QHc-yF5C3DChRwSdZKcx...)
- momokoko 6y agoCan we safely take your comment as legal advice in making business decisions about the use of the nVidia model? If so, you may want to consider the liability you are opening yourself up to. Since HN is an international site, it may be helpful to specify which geographic region you practice law out of as well. Thanks for the free legal advice! It definitely will save a few people the cost of paying for a legal interpretation of this situation. It is very generous of you.
- aspenmayer 6y agoHe’s not nvidia’s lawyer either so what does it matter either way? Please don’t do this here.
- gumby 6y agoFWIW I read sillysaurusx's comment as someone being friendly and pointing out a potential bug, not as snark or arrogance.
- programmarchy 6y agoHow did you make it so fast? Are the multiple combinations generated in parallel ahead of time once you breed images? Seems like the GPU power must be immense.
- sillysaurusx 6y agoThe 1024x1024 StyleGAN2 network can generate an image in around 200ms - 500ms. And Artbreeder may be powered by some beefy GPUs, so that might reduce the latency to something like 100ms. Throw in 100ms of server lag time, and it feels almost instantaneous. It's quite a technical achievement!
- joelS 6y agoNothing fancy, just running on AWS :) Specifically, https://aws.amazon.com/ec2/instance-types/g4/ https://aws.amazon.com/ec2/instance-types/g4/ - The uploading is actually the expensive part since it can take ~2 minutes per image.
- stevofolife 6y agoHow much are you paying to keep this up? Is it expensive to run GPU instances like this?
- battery_cowboy 6y agoI recall you had 'a font one' in the works, but I can't find anything about it now, is that still coming? The demo is really neat in the header image!
- joelS 6y agoI still have the model! I started working on the image -> vector conversion (which is more complicated than I expected) and then got distracted. I should finish that up, thanks for the reminder :)
- battery_cowboy 6y agoIt's such a cool idea, imagine a webfont that changes slightly each page load. I could make a document that slowly became more illegible after each new visitor, just for fun!
- aabbcc1241 6y agoI had a similar idea inspired by JWildfire but didn't got enough momentum to make it. You made it! And you've so much content on it already. Appreciate your effort. Spent half day having fun on your website :)
- joelS 6y agoThanks! Glad you liked it. JWildfire is super cool, I would love to add a fractal flame category to Artbreeder.
- netsec_burn 6y agoJoel, heads up the images that aren't supposed to be visible allow directory browsing. e.g. https://s3.amazonaws.com/artbreederpublic-shortlived/ https://s3.amazonaws.com/artbreederpublic-shortlived/ https://s3.amazonaws.com/artbreederpublic/ https://s3.amazonaws.com/artbreederpublic/ https://s3.amazonaws.com/ganbreederpublic/ https://s3.amazonaws.com/ganbreederpublic/ s3cli can navigate and download the images. Cool project!
- zapt02 6y agoThey forgot one more use case: nightmares! ;D
- panzer0 6y agoAs a game master, it's basically an infinite source of NPC portraits and some are even PC-worthy.
- citizenkeen 6y agoRight?! If only I could reliably get elves, orcs, dwarves, etc.
- tsukurimashou 6y agoyou simply need to find a relatively big database of those and train a GAN against it!
- jelling 6y agoHow many images do I need to mark as interesting before it generates something? I did about 20 on iOS but nothing. Really interested in project though.
- joelS 6y agoMarking them interesting doesn't update the model. If you click on any mage you can 'breed' it with others, or go to the create age to compose or upload something!
- rubyfan 6y agothis is really wild and sort of causes my brain some distress in the same way that scene from Alien Resurrection when Ripley is in the clone lab with all the failed human alien hybrids
- jlee124 6y agoWeird and wild and fantastic!
- deleted 6y ago[deleted]
- tartoran 6y agoThis is impressive, but at the same time can't help but notice art is kind of its way out when no skills nor time spent doing anything can be exchanged for just one click and generate art. I watched the intro video and the narrator was mentioning "no skills are needed", "anybody can do this", etc and I can't stop thinking how making art is at anybody's fingertips and yet means so much less. I looked through the generated images and wow, one and one are not bad looking, but after a few hundred images i said OK and closed the tab. I had a quantity overload. So where is the human if novels can be auto generated and have a plot and be good novels and where movies can be made in such a way too with auto generated music and dialogue and so on. You get my drift. It's cool but what is it for, art or entertainment? Is this just a phase that will go away?
- chii 6y ago> where is the human if novels can be auto generated... not needed. That's where. Better hope you have ownership of some shares in the companies that are generating these things! > I can't stop thinking how making art is at anybody's fingertips and yet means so much less. art has always meant something, but only to those who feel it. There's no intrinsic value, no utility to art cept by the eyes of the beholder.
- tartoran 6y agoIn a sense I agree, but think that some people used to make a living out of it and that is becoming less and less possible. In the long run fewer people are going to be artists and more will steer towards engineering. I wonder if that is a good thing or not.
- etaioinshrdlu 6y agoIt is a good question relevant to the big 3 creative mediums today, visual, music, and text... which are all seeing very interesting results with regards to neural net generation. Remember that we are still in the very early stages of the development of these techniques. The quality will end up phenomenal and you will be able to generate works perhaps on par of quality with a human, but in infinite more volume and lower cost. There may even be artistic techniques that are simply out of reach without these techniques. (Think of trying to make movies in the year 1000... it's not quite the same as today.) I think we will also see that art is not entirely about communication, but also to a big deal simply created in the eye of the beholder.
- anamexis 6y agoPretty bizarre: this site reliably causes graphics glitches for me. MacBook Pro 16" + Chrome. https://imgur.com/a/TJnRSGY https://imgur.com/a/TJnRSGY
- splintercell 6y agoRestart your machine and report if the problem still exists.
- renewiltord 6y agoThis is amazing! What a clever concept to use these for inspiration.
- ttul 6y agohttps://artbreeder.com/i?k=de694b3944a30732a426 https://artbreeder.com/i?k=de694b3944a30732a426 Jean Luc, is that you?
- chrismsimpson 6y agoArt should not be easy
- chrismsimpson 6y ago-4 Karma. I'm going for a PB
- tsukurimashou 6y agomy little brother is making music and was quite depressed when I showed him AIVA, I can understand the mixed feelings of 'real' artists
- syntaxing 6y agoThis was much funner than I expected and the results were really impressive. I read the site title and I was like ehh, its probably another style transfer-like generator. Nope, this is probably one of the best generators I played with!
- mkarliner 6y agoSorry, but this site has a terrible, incomprehensible, user interface. No help, no tooltips on mysterious icons, no UX flow.
- tgvaughan 6y agoI always thought it would be amazing to have something like this for composing music.
- dimmuborgir 6y agoMagenta Studio https://magenta.tensorflow.org/studio https://magenta.tensorflow.org/studio
- hoerzu 6y agoI had the same idea for dogs: https://need4breed.github.io https://need4breed.github.io
- stekern 6y agoThis is a super-cool project. GANs are extremely interesting to experiment with in a creative manner. One thing I often miss in these websites, however, is what I think is one of the most interesting aspects of GAN visualization - namely, a video feature for showing latent interpolations. I guess this is often skipped because inference is done server-side and there's (naturally) limits enforced to avoid excessive use of resources. In that vein, I've created a webapp [1] that runs entirely client-side using TensorFlow.js, and let's you generate videos of latent interpolation, images and tweak with parameters using a less-conventional generator trained on many different datasets. [1]: https://www.thispicturedoesnotexist.com https://www.thispicturedoesnotexist.com
- joelS 6y agoVery cool! Artbreeder does have an animation tool, just go to create -> category -> animate best, joel
- weeb 6y agoThis is really interesting and fun! One bit of UX feedback - I played about with children, cross breeding and editing genes and never figured out a mental model for what interactions caused a derivative image to be previewed vs being used to update the current seed image. Clicking on images often saved them, though I'm not sure what that means for me, the user. Do I need to save a child image in order to derive from it? Even if it's just an intermediate result I want to play with? When I breed images, I see the image on the left is updated, but it doesn't seem to update the seed image being used in the 'Children' tab, or the genes available for editting? I ended up saving and then re-opening images as a relatively safe workflow, but it would have been a lot clearer if there was more UI feedback showing me what's a preview and what's an input at any point. All those questions were rhetorical btw, just talking through my user experience :)
- fxtentacle 6y agoIt's a pity that you cannot upload your own image(s) as starting points. My theory is that because this is built be using pre-trained BigGAN models, they do not have access to the encoder to convert an image to the latent space, but instead only to the generator that converts the latent space to an image. Accordingly, they can only remix parameters in latent space, but not locate the latent space that a real-world photo would reside at.
- gspr 6y ago> It's a pity that you cannot upload your own image(s) as starting points. What do you mean? Click for example "create" -> "portrait" -> "upload".
- fxtentacle 6y agoApparently it works for portraits. I tried the general case, which doesn't offer an upload.
- fxtentacle 6y agoIt sure is quite slow at ~1 hour per uploaded portrait. That looks more like an exhaustive parameter search to me, than a trained encoder network.
- oklol123 6y agoThis kills the arts
- coldcode 6y agoNot really, it can only make derivative art.
- xtiansimon 6y agoThis is a cool project. Generating 'new' and creative images with computation is very interesting topic. I was particularly curious about the architectural and interior explorations from this user [1]. I signed up and explored what the free account permits [3,4]; however, I have to say I feel the same about this project as I did about "NLP Tool for Technological Research" (30 days ago) [2]. Unless you can upload an image you're familiar with, something you've worked on, I just don't understand what I'm seeing. Personally, I enjoy studying composition. I enjoy abstract works of Suprematists, Cubists, Futurists, Abstract Expressionists. All of which I think could be more or less reimagined with computational means. So for my WOW, I want to see what Artbreeder does to an image I'm already familiar with, or how it can imitate other visual styles. In the last case I really need to see the input, because if the results appear similar to the input there will be copyright issues. In one of my experiments [5] I used the mixed images feature. This provides a graph which gives the appearance of showing how the image was created. Then again, not seeing any code, I don't know for sure what else was used which may or may not be subject to copyright. I'm not saying the project should be open source, but maybe there is some detail that can be exposed, and used to recreate the effect with a different image. If you can do that, then QED the result is not cribbed from un-cited sources. It's just computational pixel-pushing (heck, is it?). I did some graphic design work early in my career and when we "appropriated" an image to be part of a photoshop composition, the rule of thumb was always you should not be able to reference the source from what you sampled. Some people spoke about 5-10% of an image; however, if you were doing something with any significant visibility and didn't pay for the source, the first rule was make it unidentifiable. I don't have any particular need for images right now. This is just my first impression based on my experience and what I can't get enough of--more abstract art, or something that takes an image I've studied, have intimate familiarity, and says, This will look much better upside down! haha [1]: https://artbreeder.com/lineage?k=03b855cd8c852c56b338 https://artbreeder.com/lineage?k=03b855cd8c852c56b338 [2]: https://news.ycombinator.com/item?id=22847429 https://news.ycombinator.com/item?id=22847429 [3]: https://artbreeder.com/lineage?k=ee55675001b83a64f71f https://artbreeder.com/lineage?k=ee55675001b83a64f71f [4]: https://artbreeder.com/lineage?k=14366daddf2f12087211 https://artbreeder.com/lineage?k=14366daddf2f12087211 [5]: https://artbreeder.com/lineage?k=a8ec4e77d27dc33f92f3 https://artbreeder.com/lineage?k=a8ec4e77d27dc33f92f3