27 ms·

4.2 Gigabytes, Or: How to Draw Anything

naillo 4y ago
Not really relevant bc it's kinda nitpicky but amazingly it's not even 4gb, it's more like 2gb if you use the float16 version (which has no quality degredations). Quite amazing that so many images fit in that small a package.
- latchkey 4y ago
  That's how I felt about the original ipod.
- achr2 4y ago
  The images do not. The output of the trained / weighted correlations do.
  - naillo 4y ago
    Exchanging compute for space in the space-compute tradeoff curve.
- mkaic 4y ago
  hmm -- I tried converting to float16 just using a naive model.half() call and saw some quality degradation in my images compared to just autocasting parts of the model to float16 while leaving others at float32. Curious if anyone else has had the same experience.
  - zone411 4y ago
    You may want to try bfloat16 instead.
  - naillo 4y ago
    Might be that there's some degredation but I think it's pretty close. Anyway I'm using their 'official' fp16 version which they might be doing some extra magic on idk. I.e. via StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16, ...)
    - butwhywhyoh 4y ago
      But you just said "no degradation".
      - naillo 4y ago
        Try it yourself ;) I haven't spotted any but I can't dispute the parents observations.
        mkaic 4y ago
        My best guess is that I just did something wrong lol. Autocast seems to convert most of the model to fp16 anyways and that works great, so I'll just keep using that!
    - fomine3 4y ago
      Can I download ckpt file for fp16?
dejongh 4y ago
Wow. Standing at the edge of the uncanny valley.
protonbob 4y ago
I know that technology has displaced workers in the past, but it seems to be doing so at a faster and faster pace. It makes me rethink my involvement in the creation of software and its ethical implications.
- jermaustin1 4y ago
  Having worked with a lot of artists, they are seeing stable diffusion and other image generators as a way to generate lots of ideas. A base they would then ,usually starting from scratch, build their end product off of. So even though to a casual glance, these images look amazing, if you look closely, you see all the flaws that come with them being generated. Weird artifacts, a lack of symmetry that humans usually add to their creations. These flaws would not exist (to such a degree) when a professional artist paints the same scene.
  - tartoran 4y ago
    > Having worked with a lot of artists, they are seeing stable diffusion and other image generators as a way to generate lots of ideas. But it also dillutes their own ideas. I know of painters painting AI generated stuff and that is likely a new genre but a lot of genuine artwork will lose interest on the market. It is what it is and don’t I love or hate it…
    - jermaustin1 4y ago
      You don't just paint a better version of the generated images, you look at a bunch of generated images, and get a better idea of what you want to paint from that. Say you know you want to do a portrait of a woman in armor, you can generate a dozen of those in around a minute on a 3090, look at the generated armors, the faces (usually all sorts of screwed up), and the composition. Its just a way to kickstart the creation process.
      - tartoran 4y ago
        Yes, I get that but if everyone’s doing it it becomes a race to the bottom sort of thing… I hope I am wrong and things turn out in a completely different direction that I can’t see now
t00 4y ago
Irrelevant note - it seems page is stuck at the top when using page down to scroll down.
- andy_xor_andrew 4y ago
  thanks for letting me know. I never even thought to test that. I'll give it a look.
  - durkie 4y ago
    spacebar too. just jumps you right back to the top.
  - laszlokorte 4y ago
    the reason is the onkeydown=closeModal() that clears the window.location.hash
  - red_trumpet 4y ago
    I had some problems on Android+Firefox, where the page just decided to scroll to the Top. Also, the Back button did not work.
- detritus 4y ago
  Yeah, that was driving me nuts - can't PgDn nor use arrow keys. Doing so at any point after having scrolled down just resets to top. Very frustrating!
pram 4y ago
Here is my perspective on these kinds of images. This kind of 'picture' usually comes from speed-painters who incorporate techniques like photobashing. As in integrating 3D models and RL photos into their composition, or just painting over a 3D picture entirely. It was already a genre that highly incorporated computer assisted methods. There is a lot of doom and glooming going around, but honestly the modern process of creating 'concept art' was already extremely commodified and efficient. These weren't exactly your idealized vision of some artisan craftsman laboring weeks over a picture, they churned this stuff out in a few hours (if that)
- ricardobeat 4y ago
  I think you're not grasping the magnitude of the change. Creating even an average quality speed-painting requires tremendous amount of expertise in drawing, painting techniques, composition, lighting, perspective. It takes years of training. These models let anyone achieve similar results in minutes. Without any prior learning. It is not even lowering the bar, it is literally dropping the bar to the ground. Besides, stable diffusion is able to generate not only painterly scenery, but also photographic images that are almost indistinguishable from actual pictures (certainly helped by the fact images have a heavy digital look in our era).
  - shreyshnaccount 4y ago
    it just got faster. what's the drama about?
    - oefnak 4y ago
      I think it will be interesting to see how this comment has aged in five years.
    - dougmwne 4y ago
      I am talentless and untrained. Now with a combo of prompts and img2img, I can create awesome results on any topic and in any style that I have the rights to use. That’s a 0 to 1 moment. It didn’t get faster for me, it went from impossible to possible.
fabiospampinato 4y ago
Pretty exiting stuff. It feels like there's a new cool model being trained every week, the probability that some of the upcoming ones won't have a huge impact on the world in the next few decades seems close to 0% to me.
samwillis 4y ago
I think it's fascinating, and at least to me completely unexpected, how good these image generation models are. What happens when we have a 42gb model, or a 4.2tb model? For comparison, the human brain is estimated to have a (equivalent to a computer) capacity of about 2.5 petabytes [0]. I think I read in the past that the human mind holds memories in a picture like way, I wander if that's why these image based models are so incredible when compared to the text models. Maybe we are in a new "Moore's Law" like period where the complexity and size of these models is going to double something like every 18 months. It's going to be fascinating what's possible in a few short years time, I fully expect to be continually surprised. I'm looking forward to seeing a video model trained on 10 second clips, someone somewhere is working on it. 0: https://www.scientificamerican.com/article/what-is-the-memory-capacity/ https://www.scientificamerican.com/article/what-is-the-memor...
- dougmwne 4y ago
  There is evidence that the performance of the models scale linearly with size, so moores law scaling is likely to get us some “free” improvement even if no one ever invents a better ML technique.
  - happycube 4y ago
    I forget where I saw it, but with a 200B parameter model generated text actually makes sense.
    - barrkel 4y ago
      Google Parti, kangaroo holding a sign. 20B not 200B. https://parti.research.google/ https://parti.research.google/
      - dougmwne 4y ago
        Wow, somehow I missed this one with all the whirlwind of image models recently. It’s very illuminating how the capability keeps scaling in their examples.
jedberg 4y ago
Back in the 90s I bought a book that came with some floppy disks. The book was about 500 pages of clip art, and the disks were the actual images. At the time, people said such things would put graphic artists out of business. What actually happened is that it put mediocre graphic artists out of business and highlighted the difference between one that was mediocre and one that was good. I feel like this will happen again here with digital artists. The mediocre ones will be indistinguishable from AI, but the good ones will still stand out.
- z3c0 4y ago
  While I'm overall of the mind that nobody can really know what to expect, I think this may be a little more accurate of a reality than the aspiring Cassandras are trumpeting about. I'm willing to bet that most of the people who are going to utilize these technologies are people who never could afford a graphic designer to begin with. I'm sure some of the market will be lost to this tech, but a blackbox that shoots out passable images just isn't going to cut it for certain areas of the art market. I also wouldn't be shocked if a big portion of the market for this technology ends up being the artists themselves. I personally know a painter whose creative process has been overhauled by DALL-E. Brainstorming the next project is easier than ever, and unlike the DALL-E images inspiring them, the resulting paintings actually have the "human-touch" necessary to illicit the deeper emotional response that a good painting should bring about. Adding depth to a model doesn't necessarily add depth to the output. But like I said, I don't think anyone really knows what's going to happen. We'll see, I guess.
  - ItsMonkk 4y ago
    When new technology is introduced people always talk about how existing workflows are cheaper. That's true, but the biggest impacts are always driven by things that were previously cost prohibitive that now become possible. YouTubers are a perfect example of that, it used to take entire television broadcast studios to do what they do, and now it can be done solo or with just a few people. And YouTubers are exactly the audience for this new tech - the smaller ones who want to put out branded merch can not currently do so at a level of quality. But you hire one artist to go out and make 5-100 pictures of your branding, tell them to take their time and make those few images to perfection, and that can now be molded to anything they want to create. Indie game devs must be thrilled. Aspiring indie directors looking to make green-screen backgrounds are thrilled. VRchat users looking for 3d models are thrilled.
andrelaszlo 4y ago
I know HN, we're supposed to make thoughtful comments etc, but... holy shit. :) Fits on a USB, or even a DVD.
- trevcanhuman 4y ago
  That's very thoughtful. Maybe I'll get downvoted but I'll say this: One could just store this on the long-storage tech or in a temp-controlled seed farm in someplace cold with a computer, and english instructions on how to use this. Maybe one day a human (or even another being for that matter!) in the late late future will discover this and wonder what the hell this was about and how it really worked.
  - wccrawford 4y ago
    Or maybe it'll provide them insight into how their own modern technology works that they didn't understand any more. ;)
  - jameshart 4y ago
    It’s a trope in sci-fi - the protagonist encounters a mysterious technological object from an ancient civilization, a trove containing ‘all the knowledge of an extinct race’. Usually it gets downloaded into someone’s mind, triggering some kind of cascade of baffling imagery. Feels kind of odd that this model data actually… works sort of like that?
    - trevcanhuman 4y ago
      ha! Hadn't thought about it that way. But only so much can be encoded through history. I'd love to see a sci-fi movie combined with the butterfly effect. A somewhat advanced civilization in the past and another (maybe present) civilization where people try to find out stuff about the other one, maybe they're successful at the beginning at depicting how they were but they start to think they know everything and the whole perspective of the civilization changes.
  - dogcomplex 4y ago
cube2222 4y ago
I went through this process yesterday, trying to create a city on a floating island in the sky, and it's so fun. Basically, drawing sketches, editing (rudimentaly) in image editing software, img2img, edit, img2img, and a few more rounds, and you can get to something really, really cool. This Photoshop plugin demo blew my mind yesterday: https://www.reddit.com/r/StableDiffusion/comments/wyduk1/show_rstablediffusion_integrating_sd_in_photoshop/ https://www.reddit.com/r/StableDiffusion/comments/wyduk1/sho...
- davidro80 4y ago
  That demo is amazing. I feel like I’m living in the future watching that. Great stuff.
- naillo 4y ago
  I found this one to be really rad too: https://www.reddit.com/r/StableDiffusion/comments/x0u9a8/amazing_rapid_feedback/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/StableDiffusion/comments/x0u9a8/ama...
- schnevets 4y ago
  That was an awesome video. I can't wait for someone to inevitably start a The Joy of pAInting twitch feed, complete with chill commentary and occasionally ruining the canvas with a daring addition to the picture.
- Yhippa 4y ago
  Can we see your work of art?
  - cube2222 4y ago
    Sure, why not. Here's the before[0], and here's the after[1]. And an example of a step in-between: The base was [2] which I changed into [3]. That was I think the last step before the final generation. It's not great by any means, but it's miles ahead of what I could hope to achieve myself. The biggest problem was to stop stable diffusion from turning my flying island into a pretty standard mountain. It kept trying to connect it to the ground. Especially in further iterations. [0]: https://imgur.com/a/wKraDWn https://imgur.com/a/wKraDWn [1]: https://imgur.com/a/Vn0RS9O https://imgur.com/a/Vn0RS9O [2]: https://imgur.com/a/Ti5gazx https://imgur.com/a/Ti5gazx [3]: https://imgur.com/a/CDJoKQ1 https://imgur.com/a/CDJoKQ1
tshaddox 4y ago
The article starts off with me expecting the twist to be that the image was generated with a single text prompt. That would have been neat, and in line with the other recent sensational coverage of how these new models are BLOWING PEOPLE’S MINDS. But it’s actually a walkthrough where the author goes through a much more tedious process that I could ever imagine going through to get the level of quality of that final result.
- SamBam 4y ago
  Like my sister comment, I had the exact opposite reaction. Dall-E is very cool, no doubt, but the idea that I could actually work alongside the AI to produce something that is in my head was much more eye-opening.
- FemmeAndroid 4y ago
  Huh. I had almost the exact opposite opinion. As someone who likes to make art, this example leads me to believe that this kind of process can allow me to quickly create images that are far closer to what I have in my head than most of the DallE style examples I’ve seen recently.
  - vidarh 4y ago
    My reaction too. I've only superficially played with some of the AI image generators, and while it's easy to get something which looks interesting or even good, getting something which matches a specific idea precisely seemed hard. This walkthrough shows a clear technique which seems endlessly adaptable to using it as a tool to get to a specific goal.
- barrkel 4y ago
  This article demonstrates control via iterative techniques, which is more flexible than trying to encode everything in a sentence. The input image acts as state carrying forward much more information than a sentence from one iteration to the next.
- boppo1 4y ago
  I'm very skilled at illustration and fine painting. Compared to what it takes to "manually" make such an image, his process is about as trivial as a single text prompt.
songzme 4y ago
if I want to replicate what the author did, what drawing tool / software should I get to do the initial drawing? I use ubuntu so preferably something compatible with that.
- Evidlo 4y ago
  The author mentions using GIMP. There is also Krita
  - fragmede 4y ago
    /u/imperator-maximus is working on a SD plugin for Krita: https://www.reddit.com/r/StableDiffusion/comments/x13om1/i_am_working_on_a_stable_diffusion_plugin_for/ https://www.reddit.com/r/StableDiffusion/comments/x13om1/i_a...
VBprogrammer 4y ago
I think this might be the first time I've genuinely seen something and though this quote applied: “Any sufficiently advanced technology is indistinguishable from magic”. Don't get me wrong. I'm sure there is a skill and what I'm seeing in demos is the happy path where it all happens to work well. But damn it's impressive.
mNovak 4y ago
All I want is an AI to give me beautiful color palettes. For some reason that is the absolute hardest part for me to get right.
gabereiser 4y ago
Oh I’ve been doing this since they published. I warned a friend (who is a concept artist) that his days are numbered. Great article about one of the little known features of stable-diffusion. The img2img.py awesomeness that turns your 4 year olds finger paints into Picasso or Monet. It’s just mind boggling!
franciscop 4y ago
This is the first time I see this accessibility issue, pressing the "Down" key (▼) doesn't work to navigate the page (scroll down) and instead goes all the way to the top.
- devindotcom 4y ago
  PgDn also does this... strange. Scroll wheel works fine.
- zzo38computer 4y ago
  I do not have that problem; it works OK for me.
- simonw 4y ago
  Yeah there's something really screwy going on here - I ended up having to disable JavaScript on the page in order to copy and paste from it without it leaping me up to the top when I tried.
  - franciscop 4y ago
    It seems to be because of this, any other key closes the modal and resets the hash, and resetting the hash scrolls you up: function closeModal() { let modal = document.getElementById("myModal"); modal.style.display = "none"; window.location.hash = ''; } // Left and right arrow keys scrub through the gallery. // Everything else closes the modal. document.onkeydown = function (e) { if (e.keyCode === 37) { // Left arrow goBackPrevImage(); } else if (e.keyCode === 39) { // Right arrow advanceNextImage(); } else { closeModal(); } };
- agileAlligator 4y ago
  Works for me
RVRX 4y ago
Why does my down arrow-key refuse to bring me downwards on this webpage before it shoots me back to the top ;(
Bakary 4y ago
Looks like visual artists will be rewarded in the future for creating new styles and templates that can feed prompts. Instead of developing a single style and producing from it over a lifetime, developing multiple types and spreading them like young seaturtles. Companies could be hired to develop a particular keyword over a period of weeks or months to allow for more specific prompts.
- bckr 4y ago
  > Companies could be hired to develop a particular keyword Love that idea. The prompt economy.
  - SirYandi 4y ago
    Happening already: https://promptbase.com/ https://promptbase.com/
    - rvz 4y ago
      The grift continues.
d23 4y ago
The cynicism of some people in this crowd never ceases to amaze me. This stuff is nothing short of mind blowing. If you are someone who is about to comment "meh," you probably need to take a step back from the keyboard.
- TillE 4y ago
  I'm firmly in the moderate camp of "this stuff is incredibly cool and promising, but we're nowhere near the overblown predictions of human artists becoming obsolete".
  - quitit 4y ago
    The only people who think this are people who have no experience in the field. The creative process, like many kinds of fields applicable to business is about problem-solving. AI text-to-image generation doesn't replace that function, however it does form an excellent tool, especially when it comes to rapid conceptualisation. This will allow more people to be creative problem solvers without needing to possess technical skills in image creation. Much in the same way that graphics apps allowed people to make image without needing to learn studio or art skills. Or DTP tools allowed more people to publish without the tedium and high set up costs. I will still be hiring illustrators and designers, and this may be one of their tools, and it would be their responsibility to be experts in it, but it doesn't replace them - it makes them better illustrators and better artists. The right way to think about this is not that it shrinks the field, rather it opens it and accelerates it - for that it's a very welcome addition. No creative is scared of this - they're looking forward to the next-generation approach, and it's clear that 2D images are not the end point. Soon we'll have 3D (already in progress), soon we'll have music, soon we'll have this for animation and programming. People fiddling with the technology have noticed some obvious short-comings, such as getting consistent results - for example it's currently not possible to develop a series of story boards where the character is obviously the same. Instead some level of reseeding the image with the desired character or outright recomposing the graphics later is needed. These aren't things that can't be fixed however, what we're seeing now is definitely an exciting new tool in its infancy.
  - stocknoob 4y ago
keepquestioning 4y ago
After this, I'm joining r/preppers
keepquestioning 4y ago
Feeling sorry for matte painters in Hollywood studios now.
jrochkind1 4y ago
The 4.2G is pretty hard to believe. I would have told you it's not possible if it weren't right there showing it is.
zppln 4y ago
I find myself increasingly frustrated by the low resolution of the images coming out of these systems. It all feels like a huge tease, blocking my brain from deciding whether I should be impressed or not. And at other times I find myself not really interested in looking closer at all and just keep on scrolling.
- deleted 4y ago
  [deleted]
- simonw 4y ago
  People have been solving this by using up-scaling AIs to increase the resolution of their images. That's actually built in to some of these systems - Google Imagen for example generates a 64x64 image and then up-scales it to 1024x1024: https://www.assemblyai.com/blog/how-imagen-actually-works/ https://www.assemblyai.com/blog/how-imagen-actually-works/
  - XCSme 4y ago
    Yeah, can be easily combined with super-resolution or by generating the image one section at a time.
spaceheater 4y ago
How much of it is it `painting` though? It looks like your typical startup, combine two ideas into one. (Image) Search engine + Photoshop. You enter some prompts, it searches it's database for matches, and meshes all found results into a single visually pleasing image. Can it draw something that is not a mesh or a variation of it's database? Can you?
- kevmo314 4y ago
  What is painting if not a mesh of ideas?
  - spaceheater 4y ago
    At bare minimum I can think of the 3D space the character is in, and rotate and shift it to any location and position I can dream of, even if I have never seen it before. When we will have 3D aware AI, that would be interesting.
- simonw 4y ago
  I don't that "searches it's database for matches" description is a great metaphor for how this really works. As Andy points out in this article, the model itself is a 4.2GB file. That's way, way too small to work as a "database" of examples it can stitch together. I think of it instead as an enormous mass of loosley assembled impressions of concepts - everything from a low-level primitive like a triangle to a Star Destroyer. You can use text prompts to combine those primitives - so you could get it to generate something just from dots and lines and shapes, or you could mix in extremely detailed concepts like the Seattle skyline - or anything in between.
writeslowly 4y ago
I feel like a lot of the discussion is around using these things to directly create new artwork, but these technologies (and GPT-3) feel like a method of digital divination to me. It's basically a more sophisticated form of casting an I Ching or reading tea leaves for artistic inspiration. I personally think that divination is underrated in modern society, so these developments are an interesting trend to me.
- green_on_black 4y ago
  You call it divination, I call it a random seed.
- watercooler_guy 4y ago
  I think that’s an interesting connection and way to look at this. I first encountered that sort of artistic inspiration on a Twitter account that randomly generated sheet music snippets. Sure the generation could be garbage, but that gives the artist a starting point to say “this is bad because X, it should be more like Y”. And of course with these AI models, the generation is more likely to be good. Having a divined/generated starting point for creative work can definitely be an invaluable tool.
butz 4y ago
As always, I will suggest adding loading="lazy" attributes to all images on page (or even website), as loading currently is struggling. Hoping HN won't hug your server to death, as topic is very popular.
citizenpaul 4y ago
AI art will continue to contribute to the decline in quality of all media. I could write 10,000 more words on why but I'll just stop here.
- namose 4y ago
  I think pop media will continue to decline while the long tail will become better and better (this has already happened to music, film, tv, etc). If you use google and billboard charts as a discovery engine you’re gonna have a bad time, but people who seek out quality will have more options than ever.
i_like_apis 4y ago
That's not all, they say they can get the model down to around 100MB.
losvedir 4y ago
This is very cool. I've been blown away from my dabbling with these text-to-image models, but I love the steps here to help it generate more what you're envisioning. I'd love to follow this process to generate several that I have in mind to put up on my walls, but I keep running into the resolution limitation. You need a pretty high resolution to get a crisp image at a poster size. Is there a trick or a setting to get the models to output images suitable for posters?
- xena 4y ago
  I usually upscale the images with something like real-esrgan-vulkan. I've been using that to build up a bank of images. I'm considering getting posters made of a few of them, the most notable one is one of Richard Stallman and Bill Gates playing chess that I'm calling "The Good Future We Never Got".
SpaceManNabs 4y ago
I remember when I tried to get my classmates to sign up for the AI ethics course at my grad program. Most declined saying it would be boring, not markettable, or wtv. At the end, a small slice of my cohort took the class. Kinda scary if it is the same at similar programs. I feel like a lot more people need to be thinking about the ethics of this. This particular blog post is quite harmless tbh, but things quickly get out of hand with explosive growth.
yamtaddle 4y ago
Vonnegut wrote some about the effect of recording technology & mass media on the value of individual artistic talent—in short, that it all but obliterated both the monetary and, perhaps more importantly, social value of all but (literally) world-class skill. Fewer sing-alongs around the piano at home. No more small-time performance troupes (say, vaudevillians) making enough money to get by. That uncle who's an amazing story-teller just can't compete with radio programs, so the social value of that skill plummets, and it's like that for every medium that puts the ordinary in competition, if you will, with not just the world's most talented people, but, as fields advance, with entire teams dedicated to making those already-top-notch folks seem better-than-human. The benefits of this are clear, but the problem is that artistic expression and being able to receive small-scale rewards and genuine encouragement—at least in one's family or social circle—for even minor talent seem to be very healthy and fulfilling things for people to do. Taking that away came at an ongoing cost that none of the beneficiaries of that change had to pay. A kind of social negative externality. Relatedly, consider the sections of Graeber's Bullshit Jobs where he treats of the sort of work people tend to find fulfilling or are otherwise proud to do, or are very interested in doing (sometimes to the point that supply of eager workers badly exceeds demand and pay is through the floor, as in e.g. most roles related to writing or publishing). What kind of work is it? Mainly plainly pro-social work (to take one of Graeber's examples, the disaster-relief side of what the US military does, which is by no coincidence often heavily featured in their recruiting advertising; or teaching, for another one) or: creative, artistic work. Graeber notes an apparent trend whereby these jobs tend to pay pretty poorly either due to the aforementioned over-supply of interested workers, or because there's some societal expectation that you ought to just be glad to have a job that's obviously-good and accept the sacrifice of poor pay, and that you must be bad at it or otherwise unsuited if you want to make actual money doing it (teaching's a major case of the latter—I've seen that "if you care about being paid well you must be a bad teacher" POV, and the related "if we raised teacher pay it'd result in worse teachers", advanced on this very site, more than once—it's super-common). People really, really want plainly-good and/or creative jobs, but those don't pay worth a damn unless you're at the tip-top, either of talent level, or of some organization. This seems like another blow to the creative category of desirable jobs, at least. My point is: I wonder and worry about the effect this latest wave of AI art (in a broad sense—music and writing, too) generation is going to have on already-endangered basic human needs to feel useful and wanted, and to act creatively and be appreciated for it by those they're close to. There's already a gulf between the among-the-best-in-the-world art we actually enjoy and, should our friends present their creations, how we "enjoy" those these days, with the latter being much closer to how a parent enjoys their child's art, and everyone involved knows it. Used to be, hobby-level artistic talent and effort was useful and valuable to others in one's life. Now, that stuff's just for yourself, and others indulge you, at best. Why, with this tech, you can't even get by doing very-custom art, such that the customization, rather than the already-devalued-to-almost-nothing skill itself, is what delivers the value. Now the customization is practically free, too, and most anyone can do it. Getting real last-nail-in-the-coffin vibes from all this. I'm sure it'll enable some cool things, but I can't help but think we're exchanging some novelty and a certain kind of improved productivity, for the loss of the last shreds of a fundamental part of our humanity. I wonder if we'd do this (among other things) if we could charge the various players a fair value for harm to social and psychological well-being that happens as a side-effect of their "disruption"—alas, that pool's a free-for-all to piss in all one likes, in the name of profit (see also: advertising)
deathanatos 4y ago
So this "Stable Diffusion model", it was training on a bunch of copyrighted data, and anything that comes of it must hope to launder the copyright sufficiently to not constitute a derivative work, somehow? > these models were trained on image-text pairs from a broad internet scrape … yep. I've the same issue with this as with Github Copilot. I will admit, it is technically impressive, and something I would love to use, as someone who cannot draw worth a darn. And it is that I cannot draw that I do not feel morally comfortable with this: I am using a — complicated, admittedly — tool to just derive art from the unwilling talents of others. (Admittedly, my skill in prompting & editing might matter, but that's true of "normal" derivative works, too!)
- mkaic 4y ago
  All art is derivative and there's no such thing as originality. Every human artist draws inspiration from their visual and emotional experiences, copyrighted or otherwise, how is this different? If I watch Star Wars and then make a space opera film that's aesthetically similar to Star Wars, that's not copyright laundering, it's inspiration! Same principle applies here.
  - deathanatos 4y ago
    Because the AI doesn't have "experience", it has training data that it's deriving the work from. People have shown fairly convincing examples of this in the more general sense: e.g., they've had well-known stock image (e.g., iStockPhoto) watermarks get produced in the output from the AI models (when not prompted). An artist with "experience" would not reproduce a watermark. Or in this article[1], where an AI was requests to mimic another artists style, and the output was (attempting to) reproduce the artist's signature. (IANAL.) If you make a film that directly incorporates aspects from Star Wars (what I believe to be the more accurate version of what these models do), then yes, I would expect that you will be handed a C&D. "Glowing space swords" aren't copyrighted, but if you include something indistinguishable from a lightsaber & call it a lightsaber? I bet Disney would have something to say about that. [1]: https://kotaku.com/ai-art-dall-e-midjourney-stable-diffusion-copyright-1849388060 https://kotaku.com/ai-art-dall-e-midjourney-stable-diffusion...
gitfan86 4y ago
I'm a 40 year old engineer. The big events in my life from a technology perspective so far have been: 1989: Coding for the first time (Apple IIe) 1994: Getting on the Internet 1999: Using Google when it started to get really good 2005: Buying GOOG Stock 2014: Buying Tesla Stock 2022: Building a local Pipeline with GANs, SD, Nvidia 1660 Things are about to go nuts. Tim Urban explains it well. https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol...
- helloworld97 4y ago
  https://youtu.be/4KOgmxCKhws https://youtu.be/4KOgmxCKhws The technology might be there but the capital structures...are not?
  - gitfan86 4y ago
    As far as She-Hulk goes yes. This isn't going to replace Netflix shows, it is going to create entire new industries, just like how when YouTube came out no one thought that some 12 year old kid named Jimmy was going to build a 9 figure business on top of it.
- OOPMan 4y ago
  Things are always about to go nuts and yet mostly they don't seem to...
  - gitfan86 4y ago
    The article is from 2015, so yes, you are right, it is easy to say that we have not had insane changes since 2015. And certainly there have not been insane changes between yesterday and today. But if things start changing just 1% more each day it becomes a big deal within a few years.
  - futureshock 4y ago
    I suppose that’s how you measure things? 250 years ago we were mostly farmers and our society had barely changed in thousands of years. Now look at us. 25 years ago most people lived in a pre-digital world. Now there are 6 billion smartphone users. 2.5 months ago AI art was a weird curiosity and now John Oliver is doing skits on it. If this isn’t crazy then what is crazy?
OOPMan 4y ago
Ah yes, just what the world needs, more soul-less art
wazoox 4y ago
OT, but the page can't be scrolled using the keyboard, it just resets to home position. Weird and quite irritating :)
deleted 4y ago
[deleted]
akomtu 4y ago
I'm late to the party, but what stops training this SD model on audio spectrograms? Then you'd tell it "some mozart-style violin for 5 seconds, add drums in background." The spectrogram is then translated to sound and suddenly you're a very decent music writer. With img2txt you could give it an audio file, call it "S" and tell "music in S style, but with flute".
- donkarma 4y ago
  openai did this but it doesn't sound great, I think it's because sound has less information so the brain is very picky
  - akomtu 4y ago
    mp3 density: 30sec per 1MB (some instrumental music with background). jpg density: 12M pixels per MB (trees and some landscaping). I'd argue music has a lot more information, if we can compare seconds with pixels. Imho, OpenAI didnt do a great job: a small dataset and a limited model.
sleightofmind 4y ago
Can someone give this completely ignorant old fellow a description of what the author is doing in this article? What is the initial drawing being done with? What is this stable diffusion process? (Amazing is one answer.) A very brief description of the software used. Color me astounded. I actually liked (was delighted by) the results in step-6. When I realized it was done programmatically somehow, I had to pick myself up and get back in my chair.
- GaggiX 4y ago
  The author is using a text-to-image neural network to generate an image, in this case the author is also conditioning the model on a sketch to guide the generation process.
  - sleightofmind 4y ago
    Okay. Now I'm starting to wonder if folks who worry about AI taking over are onto something. Well, not really... yet. But color me absolutely flabbergasted. Thanks for the explain.
    - dougmwne 4y ago
      This has been flabbergasting a lot of us. I have been playing with these text to image models for about a month and I still cannot believe it. Try for yourself a bit. Here’s my current favorite interface to Stable Diffusion. Give it a little sketch, then go wild with the prompt description. Try different style descriptions like oil painting or comic book. https://huggingface.co/spaces/huggingface/diffuse-the-rest https://huggingface.co/spaces/huggingface/diffuse-the-rest
      - sleightofmind 4y ago
        Thanks! I'm gonna try it out. First, though, I think I'm going to add an exorcist to my contacts list. This is honestly the closest thing to magic I've ever encountered.
breck 4y ago
I think all of Wikipedia is 40GB (the text parts anyway). So doesn't seem too outlandish.
fragmede 4y ago
I fed the mid-post prompt into MidJourney (text-only) and got https://imgur.com/a/Xg0Byt1 https://imgur.com/a/Xg0Byt1. Guiding the input with a ms-paint worthy picture really adds to the "I made this!" feeling of img2img.
XCSme 4y ago
4.2 gigabytes???? That's insane, especially considering how poorly optimized the models are (with better algorithms and ways to understand and categorize the data, this could be easily reduced 10x or even 100x).
nullc 4y ago
@#$@ this page breaks scrolling.
nyanpasu64 4y ago
It strikes me that 4.2 gigabytes is comparable to the size of the human genome, which takes between 725 megabytes to 3 gigabytes to store uncompressed depending on if you pack 4 base pairs in a byte (https://stackoverflow.com/q/8954571 https://stackoverflow.com/q/8954571).
zamfi 4y ago
Mind blown. I don't often feel like I'm looking straight into the future, but the way Andy describes this process 100% evokes this feeling now. In some domains this will almost certainly transform the way people create art, and it will feel like it's happening overnight. Digital artists will surely adapt and use these technologies to their advantage, but transitioning will take time. Personally, this also feels like merely a first glimpse into a world where humans use AI-based "cultural technologies" (to borrow Alison Gopnik's term) to "fill in the blanks" on ideas across many domains. What we are seeing now is likely just the first pass at what this technology can do.
cbsudux 4y ago
This is great, any way to use img2img on an online serivce? I don't have GPU access
- mpaepper 4y ago
  Check out this demo here where you can draw and then use your drawing and a prompt to generate new images: https://huggingface.co/spaces/huggingface/diffuse-the-rest https://huggingface.co/spaces/huggingface/diffuse-the-rest To understand how stable diffusion works, see: https://www.paepper.com/blog/posts/how-and-why-stable-diffusion-works-for-text-to-image-generation/ https://www.paepper.com/blog/posts/how-and-why-stable-diffus...

Not really relevant bc it's kinda nitpicky but amazingly it's not even 4gb, it's more like 2gb if you use the float16 version (which has no quality degredations). Quite amazing that so many images fit in that small a package.

latchkey 4y ago
That's how I felt about the original ipod.
achr2 4y ago
The images do not. The output of the trained / weighted correlations do.
- naillo 4y ago
  Exchanging compute for space in the space-compute tradeoff curve.
mkaic 4y ago
hmm -- I tried converting to float16 just using a naive model.half() call and saw some quality degradation in my images compared to just autocasting parts of the model to float16 while leaving others at float32. Curious if anyone else has had the same experience.
- zone411 4y ago
  You may want to try bfloat16 instead.
- naillo 4y ago
  Might be that there's some degredation but I think it's pretty close. Anyway I'm using their 'official' fp16 version which they might be doing some extra magic on idk. I.e. via StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16, ...)
  - butwhywhyoh 4y ago
    But you just said "no degradation".
    - naillo 4y ago
      Try it yourself ;) I haven't spotted any but I can't dispute the parents observations.
      - mkaic 4y ago
        My best guess is that I just did something wrong lol. Autocast seems to convert most of the model to fp16 anyways and that works great, so I'll just keep using that!
  - fomine3 4y ago
    Can I download ckpt file for fp16?

dejongh 4y ago

Wow. Standing at the edge of the uncanny valley.

protonbob 4y ago

I know that technology has displaced workers in the past, but it seems to be doing so at a faster and faster pace. It makes me rethink my involvement in the creation of software and its ethical implications.

jermaustin1 4y ago
Having worked with a lot of artists, they are seeing stable diffusion and other image generators as a way to generate lots of ideas. A base they would then ,usually starting from scratch, build their end product off of. So even though to a casual glance, these images look amazing, if you look closely, you see all the flaws that come with them being generated. Weird artifacts, a lack of symmetry that humans usually add to their creations. These flaws would not exist (to such a degree) when a professional artist paints the same scene.
- tartoran 4y ago
  > Having worked with a lot of artists, they are seeing stable diffusion and other image generators as a way to generate lots of ideas. But it also dillutes their own ideas. I know of painters painting AI generated stuff and that is likely a new genre but a lot of genuine artwork will lose interest on the market. It is what it is and don’t I love or hate it…
  - jermaustin1 4y ago
    You don't just paint a better version of the generated images, you look at a bunch of generated images, and get a better idea of what you want to paint from that. Say you know you want to do a portrait of a woman in armor, you can generate a dozen of those in around a minute on a 3090, look at the generated armors, the faces (usually all sorts of screwed up), and the composition. Its just a way to kickstart the creation process.
    - tartoran 4y ago
      Yes, I get that but if everyone’s doing it it becomes a race to the bottom sort of thing… I hope I am wrong and things turn out in a completely different direction that I can’t see now

t00 4y ago

Irrelevant note - it seems page is stuck at the top when using page down to scroll down.

andy_xor_andrew 4y ago
thanks for letting me know. I never even thought to test that. I'll give it a look.
- durkie 4y ago
  spacebar too. just jumps you right back to the top.
- laszlokorte 4y ago
  the reason is the onkeydown=closeModal() that clears the window.location.hash
- red_trumpet 4y ago
  I had some problems on Android+Firefox, where the page just decided to scroll to the Top. Also, the Back button did not work.
detritus 4y ago
Yeah, that was driving me nuts - can't PgDn nor use arrow keys. Doing so at any point after having scrolled down just resets to top. Very frustrating!

pram 4y ago

Here is my perspective on these kinds of images. This kind of 'picture' usually comes from speed-painters who incorporate techniques like photobashing. As in integrating 3D models and RL photos into their composition, or just painting over a 3D picture entirely. It was already a genre that highly incorporated computer assisted methods. There is a lot of doom and glooming going around, but honestly the modern process of creating 'concept art' was already extremely commodified and efficient. These weren't exactly your idealized vision of some artisan craftsman laboring weeks over a picture, they churned this stuff out in a few hours (if that)

ricardobeat 4y ago
I think you're not grasping the magnitude of the change. Creating even an average quality speed-painting requires tremendous amount of expertise in drawing, painting techniques, composition, lighting, perspective. It takes years of training. These models let anyone achieve similar results in minutes. Without any prior learning. It is not even lowering the bar, it is literally dropping the bar to the ground. Besides, stable diffusion is able to generate not only painterly scenery, but also photographic images that are almost indistinguishable from actual pictures (certainly helped by the fact images have a heavy digital look in our era).
- shreyshnaccount 4y ago
  it just got faster. what's the drama about?
  - oefnak 4y ago
    I think it will be interesting to see how this comment has aged in five years.
  - dougmwne 4y ago
    I am talentless and untrained. Now with a combo of prompts and img2img, I can create awesome results on any topic and in any style that I have the rights to use. That’s a 0 to 1 moment. It didn’t get faster for me, it went from impossible to possible.

fabiospampinato 4y ago

Pretty exiting stuff. It feels like there's a new cool model being trained every week, the probability that some of the upcoming ones won't have a huge impact on the world in the next few decades seems close to 0% to me.

samwillis 4y ago

I think it's fascinating, and at least to me completely unexpected, how good these image generation models are. What happens when we have a 42gb model, or a 4.2tb model? For comparison, the human brain is estimated to have a (equivalent to a computer) capacity of about 2.5 petabytes [0]. I think I read in the past that the human mind holds memories in a picture like way, I wander if that's why these image based models are so incredible when compared to the text models. Maybe we are in a new "Moore's Law" like period where the complexity and size of these models is going to double something like every 18 months. It's going to be fascinating what's possible in a few short years time, I fully expect to be continually surprised. I'm looking forward to seeing a video model trained on 10 second clips, someone somewhere is working on it. 0: https://www.scientificamerican.com/article/what-is-the-memory-capacity/ https://www.scientificamerican.com/article/what-is-the-memor...

dougmwne 4y ago
There is evidence that the performance of the models scale linearly with size, so moores law scaling is likely to get us some “free” improvement even if no one ever invents a better ML technique.
- happycube 4y ago
  I forget where I saw it, but with a 200B parameter model generated text actually makes sense.
  - barrkel 4y ago
    Google Parti, kangaroo holding a sign. 20B not 200B. https://parti.research.google/ https://parti.research.google/
    - dougmwne 4y ago
      Wow, somehow I missed this one with all the whirlwind of image models recently. It’s very illuminating how the capability keeps scaling in their examples.

jedberg 4y ago

Back in the 90s I bought a book that came with some floppy disks. The book was about 500 pages of clip art, and the disks were the actual images. At the time, people said such things would put graphic artists out of business. What actually happened is that it put mediocre graphic artists out of business and highlighted the difference between one that was mediocre and one that was good. I feel like this will happen again here with digital artists. The mediocre ones will be indistinguishable from AI, but the good ones will still stand out.

z3c0 4y ago
While I'm overall of the mind that nobody can really know what to expect, I think this may be a little more accurate of a reality than the aspiring Cassandras are trumpeting about. I'm willing to bet that most of the people who are going to utilize these technologies are people who never could afford a graphic designer to begin with. I'm sure some of the market will be lost to this tech, but a blackbox that shoots out passable images just isn't going to cut it for certain areas of the art market. I also wouldn't be shocked if a big portion of the market for this technology ends up being the artists themselves. I personally know a painter whose creative process has been overhauled by DALL-E. Brainstorming the next project is easier than ever, and unlike the DALL-E images inspiring them, the resulting paintings actually have the "human-touch" necessary to illicit the deeper emotional response that a good painting should bring about. Adding depth to a model doesn't necessarily add depth to the output. But like I said, I don't think anyone really knows what's going to happen. We'll see, I guess.
- ItsMonkk 4y ago
  When new technology is introduced people always talk about how existing workflows are cheaper. That's true, but the biggest impacts are always driven by things that were previously cost prohibitive that now become possible. YouTubers are a perfect example of that, it used to take entire television broadcast studios to do what they do, and now it can be done solo or with just a few people. And YouTubers are exactly the audience for this new tech - the smaller ones who want to put out branded merch can not currently do so at a level of quality. But you hire one artist to go out and make 5-100 pictures of your branding, tell them to take their time and make those few images to perfection, and that can now be molded to anything they want to create. Indie game devs must be thrilled. Aspiring indie directors looking to make green-screen backgrounds are thrilled. VRchat users looking for 3d models are thrilled.

andrelaszlo 4y ago

I know HN, we're supposed to make thoughtful comments etc, but... holy shit. :) Fits on a USB, or even a DVD.

trevcanhuman 4y ago
That's very thoughtful. Maybe I'll get downvoted but I'll say this: One could just store this on the long-storage tech or in a temp-controlled seed farm in someplace cold with a computer, and english instructions on how to use this. Maybe one day a human (or even another being for that matter!) in the late late future will discover this and wonder what the hell this was about and how it really worked.
- wccrawford 4y ago
  Or maybe it'll provide them insight into how their own modern technology works that they didn't understand any more. ;)
- jameshart 4y ago
  It’s a trope in sci-fi - the protagonist encounters a mysterious technological object from an ancient civilization, a trove containing ‘all the knowledge of an extinct race’. Usually it gets downloaded into someone’s mind, triggering some kind of cascade of baffling imagery. Feels kind of odd that this model data actually… works sort of like that?
  - trevcanhuman 4y ago
    ha! Hadn't thought about it that way. But only so much can be encoded through history. I'd love to see a sci-fi movie combined with the butterfly effect. A somewhat advanced civilization in the past and another (maybe present) civilization where people try to find out stuff about the other one, maybe they're successful at the beginning at depicting how they were but they start to think they know everything and the whole perspective of the civilization changes.
- dogcomplex 4y ago

cube2222 4y ago

I went through this process yesterday, trying to create a city on a floating island in the sky, and it's so fun. Basically, drawing sketches, editing (rudimentaly) in image editing software, img2img, edit, img2img, and a few more rounds, and you can get to something really, really cool. This Photoshop plugin demo blew my mind yesterday: https://www.reddit.com/r/StableDiffusion/comments/wyduk1/show_rstablediffusion_integrating_sd_in_photoshop/ https://www.reddit.com/r/StableDiffusion/comments/wyduk1/sho...

davidro80 4y ago
That demo is amazing. I feel like I’m living in the future watching that. Great stuff.
naillo 4y ago
I found this one to be really rad too: https://www.reddit.com/r/StableDiffusion/comments/x0u9a8/amazing_rapid_feedback/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/StableDiffusion/comments/x0u9a8/ama...
schnevets 4y ago
That was an awesome video. I can't wait for someone to inevitably start a The Joy of pAInting twitch feed, complete with chill commentary and occasionally ruining the canvas with a daring addition to the picture.
Yhippa 4y ago
Can we see your work of art?
- cube2222 4y ago
  Sure, why not. Here's the before[0], and here's the after[1]. And an example of a step in-between: The base was [2] which I changed into [3]. That was I think the last step before the final generation. It's not great by any means, but it's miles ahead of what I could hope to achieve myself. The biggest problem was to stop stable diffusion from turning my flying island into a pretty standard mountain. It kept trying to connect it to the ground. Especially in further iterations. [0]: https://imgur.com/a/wKraDWn https://imgur.com/a/wKraDWn [1]: https://imgur.com/a/Vn0RS9O https://imgur.com/a/Vn0RS9O [2]: https://imgur.com/a/Ti5gazx https://imgur.com/a/Ti5gazx [3]: https://imgur.com/a/CDJoKQ1 https://imgur.com/a/CDJoKQ1

tshaddox 4y ago

The article starts off with me expecting the twist to be that the image was generated with a single text prompt. That would have been neat, and in line with the other recent sensational coverage of how these new models are BLOWING PEOPLE’S MINDS. But it’s actually a walkthrough where the author goes through a much more tedious process that I could ever imagine going through to get the level of quality of that final result.

SamBam 4y ago
Like my sister comment, I had the exact opposite reaction. Dall-E is very cool, no doubt, but the idea that I could actually work alongside the AI to produce something that is in my head was much more eye-opening.
FemmeAndroid 4y ago
Huh. I had almost the exact opposite opinion. As someone who likes to make art, this example leads me to believe that this kind of process can allow me to quickly create images that are far closer to what I have in my head than most of the DallE style examples I’ve seen recently.
- vidarh 4y ago
  My reaction too. I've only superficially played with some of the AI image generators, and while it's easy to get something which looks interesting or even good, getting something which matches a specific idea precisely seemed hard. This walkthrough shows a clear technique which seems endlessly adaptable to using it as a tool to get to a specific goal.
barrkel 4y ago
This article demonstrates control via iterative techniques, which is more flexible than trying to encode everything in a sentence. The input image acts as state carrying forward much more information than a sentence from one iteration to the next.
boppo1 4y ago
I'm very skilled at illustration and fine painting. Compared to what it takes to "manually" make such an image, his process is about as trivial as a single text prompt.

songzme 4y ago

if I want to replicate what the author did, what drawing tool / software should I get to do the initial drawing? I use ubuntu so preferably something compatible with that.

Evidlo 4y ago
The author mentions using GIMP. There is also Krita
- fragmede 4y ago
  /u/imperator-maximus is working on a SD plugin for Krita: https://www.reddit.com/r/StableDiffusion/comments/x13om1/i_am_working_on_a_stable_diffusion_plugin_for/ https://www.reddit.com/r/StableDiffusion/comments/x13om1/i_a...

VBprogrammer 4y ago

I think this might be the first time I've genuinely seen something and though this quote applied: “Any sufficiently advanced technology is indistinguishable from magic”. Don't get me wrong. I'm sure there is a skill and what I'm seeing in demos is the happy path where it all happens to work well. But damn it's impressive.

mNovak 4y ago

All I want is an AI to give me beautiful color palettes. For some reason that is the absolute hardest part for me to get right.

gabereiser 4y ago

Oh I’ve been doing this since they published. I warned a friend (who is a concept artist) that his days are numbered. Great article about one of the little known features of stable-diffusion. The img2img.py awesomeness that turns your 4 year olds finger paints into Picasso or Monet. It’s just mind boggling!

franciscop 4y ago

This is the first time I see this accessibility issue, pressing the "Down" key (▼) doesn't work to navigate the page (scroll down) and instead goes all the way to the top.

devindotcom 4y ago
PgDn also does this... strange. Scroll wheel works fine.
zzo38computer 4y ago
I do not have that problem; it works OK for me.
simonw 4y ago
Yeah there's something really screwy going on here - I ended up having to disable JavaScript on the page in order to copy and paste from it without it leaping me up to the top when I tried.
- franciscop 4y ago
  It seems to be because of this, any other key closes the modal and resets the hash, and resetting the hash scrolls you up: function closeModal() { let modal = document.getElementById("myModal"); modal.style.display = "none"; window.location.hash = ''; } // Left and right arrow keys scrub through the gallery. // Everything else closes the modal. document.onkeydown = function (e) { if (e.keyCode === 37) { // Left arrow goBackPrevImage(); } else if (e.keyCode === 39) { // Right arrow advanceNextImage(); } else { closeModal(); } };
agileAlligator 4y ago
Works for me

RVRX 4y ago

Why does my down arrow-key refuse to bring me downwards on this webpage before it shoots me back to the top ;(

Bakary 4y ago

Looks like visual artists will be rewarded in the future for creating new styles and templates that can feed prompts. Instead of developing a single style and producing from it over a lifetime, developing multiple types and spreading them like young seaturtles. Companies could be hired to develop a particular keyword over a period of weeks or months to allow for more specific prompts.

bckr 4y ago
> Companies could be hired to develop a particular keyword Love that idea. The prompt economy.
- SirYandi 4y ago
  Happening already: https://promptbase.com/ https://promptbase.com/
  - rvz 4y ago
    The grift continues.

d23 4y ago

The cynicism of some people in this crowd never ceases to amaze me. This stuff is nothing short of mind blowing. If you are someone who is about to comment "meh," you probably need to take a step back from the keyboard.

TillE 4y ago
I'm firmly in the moderate camp of "this stuff is incredibly cool and promising, but we're nowhere near the overblown predictions of human artists becoming obsolete".
- quitit 4y ago
  The only people who think this are people who have no experience in the field. The creative process, like many kinds of fields applicable to business is about problem-solving. AI text-to-image generation doesn't replace that function, however it does form an excellent tool, especially when it comes to rapid conceptualisation. This will allow more people to be creative problem solvers without needing to possess technical skills in image creation. Much in the same way that graphics apps allowed people to make image without needing to learn studio or art skills. Or DTP tools allowed more people to publish without the tedium and high set up costs. I will still be hiring illustrators and designers, and this may be one of their tools, and it would be their responsibility to be experts in it, but it doesn't replace them - it makes them better illustrators and better artists. The right way to think about this is not that it shrinks the field, rather it opens it and accelerates it - for that it's a very welcome addition. No creative is scared of this - they're looking forward to the next-generation approach, and it's clear that 2D images are not the end point. Soon we'll have 3D (already in progress), soon we'll have music, soon we'll have this for animation and programming. People fiddling with the technology have noticed some obvious short-comings, such as getting consistent results - for example it's currently not possible to develop a series of story boards where the character is obviously the same. Instead some level of reseeding the image with the desired character or outright recomposing the graphics later is needed. These aren't things that can't be fixed however, what we're seeing now is definitely an exciting new tool in its infancy.
- stocknoob 4y ago

keepquestioning 4y ago

After this, I'm joining r/preppers

keepquestioning 4y ago

Feeling sorry for matte painters in Hollywood studios now.

jrochkind1 4y ago

The 4.2G is pretty hard to believe. I would have told you it's not possible if it weren't right there showing it is.

zppln 4y ago

I find myself increasingly frustrated by the low resolution of the images coming out of these systems. It all feels like a huge tease, blocking my brain from deciding whether I should be impressed or not. And at other times I find myself not really interested in looking closer at all and just keep on scrolling.

deleted 4y ago
[deleted]
simonw 4y ago
People have been solving this by using up-scaling AIs to increase the resolution of their images. That's actually built in to some of these systems - Google Imagen for example generates a 64x64 image and then up-scales it to 1024x1024: https://www.assemblyai.com/blog/how-imagen-actually-works/ https://www.assemblyai.com/blog/how-imagen-actually-works/
- XCSme 4y ago
  Yeah, can be easily combined with super-resolution or by generating the image one section at a time.

spaceheater 4y ago

How much of it is it `painting` though? It looks like your typical startup, combine two ideas into one. (Image) Search engine + Photoshop. You enter some prompts, it searches it's database for matches, and meshes all found results into a single visually pleasing image. Can it draw something that is not a mesh or a variation of it's database? Can you?

kevmo314 4y ago
What is painting if not a mesh of ideas?
- spaceheater 4y ago
  At bare minimum I can think of the 3D space the character is in, and rotate and shift it to any location and position I can dream of, even if I have never seen it before. When we will have 3D aware AI, that would be interesting.
simonw 4y ago
I don't that "searches it's database for matches" description is a great metaphor for how this really works. As Andy points out in this article, the model itself is a 4.2GB file. That's way, way too small to work as a "database" of examples it can stitch together. I think of it instead as an enormous mass of loosley assembled impressions of concepts - everything from a low-level primitive like a triangle to a Star Destroyer. You can use text prompts to combine those primitives - so you could get it to generate something just from dots and lines and shapes, or you could mix in extremely detailed concepts like the Seattle skyline - or anything in between.

writeslowly 4y ago

I feel like a lot of the discussion is around using these things to directly create new artwork, but these technologies (and GPT-3) feel like a method of digital divination to me. It's basically a more sophisticated form of casting an I Ching or reading tea leaves for artistic inspiration. I personally think that divination is underrated in modern society, so these developments are an interesting trend to me.

green_on_black 4y ago
You call it divination, I call it a random seed.
watercooler_guy 4y ago
I think that’s an interesting connection and way to look at this. I first encountered that sort of artistic inspiration on a Twitter account that randomly generated sheet music snippets. Sure the generation could be garbage, but that gives the artist a starting point to say “this is bad because X, it should be more like Y”. And of course with these AI models, the generation is more likely to be good. Having a divined/generated starting point for creative work can definitely be an invaluable tool.

butz 4y ago

As always, I will suggest adding loading="lazy" attributes to all images on page (or even website), as loading currently is struggling. Hoping HN won't hug your server to death, as topic is very popular.

citizenpaul 4y ago

AI art will continue to contribute to the decline in quality of all media. I could write 10,000 more words on why but I'll just stop here.

namose 4y ago
I think pop media will continue to decline while the long tail will become better and better (this has already happened to music, film, tv, etc). If you use google and billboard charts as a discovery engine you’re gonna have a bad time, but people who seek out quality will have more options than ever.

i_like_apis 4y ago

That's not all, they say they can get the model down to around 100MB.

losvedir 4y ago

This is very cool. I've been blown away from my dabbling with these text-to-image models, but I love the steps here to help it generate more what you're envisioning. I'd love to follow this process to generate several that I have in mind to put up on my walls, but I keep running into the resolution limitation. You need a pretty high resolution to get a crisp image at a poster size. Is there a trick or a setting to get the models to output images suitable for posters?

xena 4y ago
I usually upscale the images with something like real-esrgan-vulkan. I've been using that to build up a bank of images. I'm considering getting posters made of a few of them, the most notable one is one of Richard Stallman and Bill Gates playing chess that I'm calling "The Good Future We Never Got".

SpaceManNabs 4y ago

I remember when I tried to get my classmates to sign up for the AI ethics course at my grad program. Most declined saying it would be boring, not markettable, or wtv. At the end, a small slice of my cohort took the class. Kinda scary if it is the same at similar programs. I feel like a lot more people need to be thinking about the ethics of this. This particular blog post is quite harmless tbh, but things quickly get out of hand with explosive growth.

yamtaddle 4y ago

Vonnegut wrote some about the effect of recording technology & mass media on the value of individual artistic talent—in short, that it all but obliterated both the monetary and, perhaps more importantly, social value of all but (literally) world-class skill. Fewer sing-alongs around the piano at home. No more small-time performance troupes (say, vaudevillians) making enough money to get by. That uncle who's an amazing story-teller just can't compete with radio programs, so the social value of that skill plummets, and it's like that for every medium that puts the ordinary in competition, if you will, with not just the world's most talented people, but, as fields advance, with entire teams dedicated to making those already-top-notch folks seem better-than-human. The benefits of this are clear, but the problem is that artistic expression and being able to receive small-scale rewards and genuine encouragement—at least in one's family or social circle—for even minor talent seem to be very healthy and fulfilling things for people to do. Taking that away came at an ongoing cost that none of the beneficiaries of that change had to pay. A kind of social negative externality. Relatedly, consider the sections of Graeber's Bullshit Jobs where he treats of the sort of work people tend to find fulfilling or are otherwise proud to do, or are very interested in doing (sometimes to the point that supply of eager workers badly exceeds demand and pay is through the floor, as in e.g. most roles related to writing or publishing). What kind of work is it? Mainly plainly pro-social work (to take one of Graeber's examples, the disaster-relief side of what the US military does, which is by no coincidence often heavily featured in their recruiting advertising; or teaching, for another one) or: creative, artistic work. Graeber notes an apparent trend whereby these jobs tend to pay pretty poorly either due to the aforementioned over-supply of interested workers, or because there's some societal expectation that you ought to just be glad to have a job that's obviously-good and accept the sacrifice of poor pay, and that you must be bad at it or otherwise unsuited if you want to make actual money doing it (teaching's a major case of the latter—I've seen that "if you care about being paid well you must be a bad teacher" POV, and the related "if we raised teacher pay it'd result in worse teachers", advanced on this very site, more than once—it's super-common). People really, really want plainly-good and/or creative jobs, but those don't pay worth a damn unless you're at the tip-top, either of talent level, or of some organization. This seems like another blow to the creative category of desirable jobs, at least. My point is: I wonder and worry about the effect this latest wave of AI art (in a broad sense—music and writing, too) generation is going to have on already-endangered basic human needs to feel useful and wanted, and to act creatively and be appreciated for it by those they're close to. There's already a gulf between the among-the-best-in-the-world art we actually enjoy and, should our friends present their creations, how we "enjoy" those these days, with the latter being much closer to how a parent enjoys their child's art, and everyone involved knows it. Used to be, hobby-level artistic talent and effort was useful and valuable to others in one's life. Now, that stuff's just for yourself, and others indulge you, at best. Why, with this tech, you can't even get by doing very-custom art, such that the customization, rather than the already-devalued-to-almost-nothing skill itself, is what delivers the value. Now the customization is practically free, too, and most anyone can do it. Getting real last-nail-in-the-coffin vibes from all this. I'm sure it'll enable some cool things, but I can't help but think we're exchanging some novelty and a certain kind of improved productivity, for the loss of the last shreds of a fundamental part of our humanity. I wonder if we'd do this (among other things) if we could charge the various players a fair value for harm to social and psychological well-being that happens as a side-effect of their "disruption"—alas, that pool's a free-for-all to piss in all one likes, in the name of profit (see also: advertising)

deathanatos 4y ago

So this "Stable Diffusion model", it was training on a bunch of copyrighted data, and anything that comes of it must hope to launder the copyright sufficiently to not constitute a derivative work, somehow? > these models were trained on image-text pairs from a broad internet scrape … yep. I've the same issue with this as with Github Copilot. I will admit, it is technically impressive, and something I would love to use, as someone who cannot draw worth a darn. And it is that I cannot draw that I do not feel morally comfortable with this: I am using a — complicated, admittedly — tool to just derive art from the unwilling talents of others. (Admittedly, my skill in prompting & editing might matter, but that's true of "normal" derivative works, too!)

mkaic 4y ago
All art is derivative and there's no such thing as originality. Every human artist draws inspiration from their visual and emotional experiences, copyrighted or otherwise, how is this different? If I watch Star Wars and then make a space opera film that's aesthetically similar to Star Wars, that's not copyright laundering, it's inspiration! Same principle applies here.
- deathanatos 4y ago
  Because the AI doesn't have "experience", it has training data that it's deriving the work from. People have shown fairly convincing examples of this in the more general sense: e.g., they've had well-known stock image (e.g., iStockPhoto) watermarks get produced in the output from the AI models (when not prompted). An artist with "experience" would not reproduce a watermark. Or in this article[1], where an AI was requests to mimic another artists style, and the output was (attempting to) reproduce the artist's signature. (IANAL.) If you make a film that directly incorporates aspects from Star Wars (what I believe to be the more accurate version of what these models do), then yes, I would expect that you will be handed a C&D. "Glowing space swords" aren't copyrighted, but if you include something indistinguishable from a lightsaber & call it a lightsaber? I bet Disney would have something to say about that. [1]: https://kotaku.com/ai-art-dall-e-midjourney-stable-diffusion-copyright-1849388060 https://kotaku.com/ai-art-dall-e-midjourney-stable-diffusion...

gitfan86 4y ago

I'm a 40 year old engineer. The big events in my life from a technology perspective so far have been: 1989: Coding for the first time (Apple IIe) 1994: Getting on the Internet 1999: Using Google when it started to get really good 2005: Buying GOOG Stock 2014: Buying Tesla Stock 2022: Building a local Pipeline with GANs, SD, Nvidia 1660 Things are about to go nuts. Tim Urban explains it well. https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol...

helloworld97 4y ago
https://youtu.be/4KOgmxCKhws https://youtu.be/4KOgmxCKhws The technology might be there but the capital structures...are not?
- gitfan86 4y ago
  As far as She-Hulk goes yes. This isn't going to replace Netflix shows, it is going to create entire new industries, just like how when YouTube came out no one thought that some 12 year old kid named Jimmy was going to build a 9 figure business on top of it.
OOPMan 4y ago
Things are always about to go nuts and yet mostly they don't seem to...
- gitfan86 4y ago
  The article is from 2015, so yes, you are right, it is easy to say that we have not had insane changes since 2015. And certainly there have not been insane changes between yesterday and today. But if things start changing just 1% more each day it becomes a big deal within a few years.
- futureshock 4y ago
  I suppose that’s how you measure things? 250 years ago we were mostly farmers and our society had barely changed in thousands of years. Now look at us. 25 years ago most people lived in a pre-digital world. Now there are 6 billion smartphone users. 2.5 months ago AI art was a weird curiosity and now John Oliver is doing skits on it. If this isn’t crazy then what is crazy?

OOPMan 4y ago

Ah yes, just what the world needs, more soul-less art

wazoox 4y ago

OT, but the page can't be scrolled using the keyboard, it just resets to home position. Weird and quite irritating :)

deleted 4y ago

[deleted]

akomtu 4y ago

I'm late to the party, but what stops training this SD model on audio spectrograms? Then you'd tell it "some mozart-style violin for 5 seconds, add drums in background." The spectrogram is then translated to sound and suddenly you're a very decent music writer. With img2txt you could give it an audio file, call it "S" and tell "music in S style, but with flute".

donkarma 4y ago
openai did this but it doesn't sound great, I think it's because sound has less information so the brain is very picky
- akomtu 4y ago
  mp3 density: 30sec per 1MB (some instrumental music with background). jpg density: 12M pixels per MB (trees and some landscaping). I'd argue music has a lot more information, if we can compare seconds with pixels. Imho, OpenAI didnt do a great job: a small dataset and a limited model.

sleightofmind 4y ago

Can someone give this completely ignorant old fellow a description of what the author is doing in this article? What is the initial drawing being done with? What is this stable diffusion process? (Amazing is one answer.) A very brief description of the software used. Color me astounded. I actually liked (was delighted by) the results in step-6. When I realized it was done programmatically somehow, I had to pick myself up and get back in my chair.

GaggiX 4y ago
The author is using a text-to-image neural network to generate an image, in this case the author is also conditioning the model on a sketch to guide the generation process.
- sleightofmind 4y ago
  Okay. Now I'm starting to wonder if folks who worry about AI taking over are onto something. Well, not really... yet. But color me absolutely flabbergasted. Thanks for the explain.
  - dougmwne 4y ago
    This has been flabbergasting a lot of us. I have been playing with these text to image models for about a month and I still cannot believe it. Try for yourself a bit. Here’s my current favorite interface to Stable Diffusion. Give it a little sketch, then go wild with the prompt description. Try different style descriptions like oil painting or comic book. https://huggingface.co/spaces/huggingface/diffuse-the-rest https://huggingface.co/spaces/huggingface/diffuse-the-rest
    - sleightofmind 4y ago
      Thanks! I'm gonna try it out. First, though, I think I'm going to add an exorcist to my contacts list. This is honestly the closest thing to magic I've ever encountered.

breck 4y ago

I think all of Wikipedia is 40GB (the text parts anyway). So doesn't seem too outlandish.

fragmede 4y ago

I fed the mid-post prompt into MidJourney (text-only) and got https://imgur.com/a/Xg0Byt1 https://imgur.com/a/Xg0Byt1. Guiding the input with a ms-paint worthy picture really adds to the "I made this!" feeling of img2img.

XCSme 4y ago

4.2 gigabytes???? That's insane, especially considering how poorly optimized the models are (with better algorithms and ways to understand and categorize the data, this could be easily reduced 10x or even 100x).

nullc 4y ago

@#$@ this page breaks scrolling.

nyanpasu64 4y ago

It strikes me that 4.2 gigabytes is comparable to the size of the human genome, which takes between 725 megabytes to 3 gigabytes to store uncompressed depending on if you pack 4 base pairs in a byte (https://stackoverflow.com/q/8954571 https://stackoverflow.com/q/8954571).

zamfi 4y ago

Mind blown. I don't often feel like I'm looking straight into the future, but the way Andy describes this process 100% evokes this feeling now. In some domains this will almost certainly transform the way people create art, and it will feel like it's happening overnight. Digital artists will surely adapt and use these technologies to their advantage, but transitioning will take time. Personally, this also feels like merely a first glimpse into a world where humans use AI-based "cultural technologies" (to borrow Alison Gopnik's term) to "fill in the blanks" on ideas across many domains. What we are seeing now is likely just the first pass at what this technology can do.

cbsudux 4y ago

This is great, any way to use img2img on an online serivce? I don't have GPU access

mpaepper 4y ago
Check out this demo here where you can draw and then use your drawing and a prompt to generate new images: https://huggingface.co/spaces/huggingface/diffuse-the-rest https://huggingface.co/spaces/huggingface/diffuse-the-rest To understand how stable diffusion works, see: https://www.paepper.com/blog/posts/how-and-why-stable-diffusion-works-for-text-to-image-generation/ https://www.paepper.com/blog/posts/how-and-why-stable-diffus...