13 ms·
Stable Diffusion Textual Inversion
- kelseyfrog 4y agoAnd in a blink of an eye, the career potential of all aspiring "Sr. Prompt Engineer"s vanished into the whirlpool of automatable tasks. On a more serious note, this opens up the door to exploring fixed points of txt2img->img2txt and img2txt -> txt2img. It may open the door to more model interpretability.
- keepquestioning 4y agoELI5 - why has there been a cavalcade of Stable Diffusion spam on HN recently? What does it all mean?
- fxtentacle 4y agoYou can now fire artists/designers and replace them with AI. Obviously, that's cheaper.
- dougmwne 4y agoSomeone will surely come by soon and tell us, “well actually… artists and graphic designers are irreplaceable.” But for real, plenty of people are going to start rolling their own art and skipping the artist. Not Coca-Cola, but small to medium businesses doing a brochure or PowerPoint? Sure!
- visarga 4y agoI think there's going to be plenty of work in stacking multiple AI prompts or manual retouching to fix rough spots. It automates a task, not a job. Some people won't use it at all and other people will use it only for reference - in the end doing everything by hand, as usual, because they have more control and because AI art has a specific smell to it and people will associate it with cheap. But it's not just for art and design, it has uses in brainstorming, planning, and just to visualise your ideas and extend your imagination. It's a bicycle for the mind. People will eat it up, old copyrights and jobs be damned. It's a cyborg moment when we extend our minds with AI and it feels great. By the end of the decade we'll have mature models for all modalities. We'll extend our minds in many ways, and applications will be countless. There's going to be a lot of work created around it.
- PcChip 4y ago>AI art has a specific smell to it and people will associate it with cheap. It might now, but I feel like that will be trained out of it a few more papers down the line
- Larrikin 4y agoPeople rarely write assembly code nowadays because we mostly all use higher level abstractions that let us write more powerful code with fewer lines. There are plenty of small shops now where somebody knows a little Photoshop and can eek out a design that they otherwise wouldn't be able to using pen and paper. There are also professionals that use the Adobe suite to enhance their abilities they've cultivated for years. AI art will simply be a tool that enhances artists but might take away some low hanging fruit jobs similar to how web frameworks pushed people out of the job of webmaster and into more specific roles.
- adrianN 4y agoIf you automate part of a job, you need fewer people doing the job. We still have farmers, but we automated enough of the job to do the same work with a thousand times fewer farmers.
- blensor 4y agoI think the first business to crack will be stock image sites like shutterstock. Those were already used in a "let's find something that roughly fits what I want to communicate with this text" way. Today I created a quick get well soon card using an image from the new Midjourney beta and I have to say the result was exactly as good as if I had used Shutterstock but it took me much less time because the search prompt matched created something I wanted on the third try. Comparing that to sifting through pages and pages of vaguely relevant images it's a clear win and a lot cheaper
- djmips 4y agoFor sure it automates some work. For example, my sometime hobby of making silly photoshops looks like it will now be a whole lot easier... Visual memes can just be a sentence now. For more serious work I wonder... But it does give pause about what it means for other forms of work.
- Eji1700 4y agoIt'll be an interesting line to be sure. Right now the tech still requires some nuance to be able to slap it all together into what I think most people would want. While i expect the interface and the like to get a lot better, all good tutorials of this tech so far show many iterations over many different parts of an image to get something "cohesive". Blending those little mini iterations together is VASTLY easier than just making the whole thing, but not just plug and play for something professional. Still there will be a huge dent in how long it takes to make certain styles of work and that will lower demand considerably, and there's a large market of artists who thrive on casual commissions which this might replace.
- boredtofears 4y agoWeren’t the small to medium businesses already mostly using stock images anyway? If anyone was to be worried I’d think it would be Getty Images.
- kazinator 4y agoThere is plenty of reason for artists who are hired for one-of-a-kind work to be worried. Getty Images will just get with the program and stock up on bazillions of AI generated images, indexed by the prompts used to generate them. Someone looking for stock images doesn't want to deal with artists, photographers or feeding prompts after prompt into some AI software, while not quite getting the desired result. If Getty makes it easier for someone to find some existing AI-generated image than to generate one, they still have something. A lot of the AI images we see in online blogs and galeries have been curated; people tinkered for hours with the stuff, and cherry-picked the best results. There could be some business model in that, at least for a while.
- CMCDragonkai 4y agoGetty would just be a cache with the prompts acting as the initial search index and buyers are just typing keywords in to buy cached images.
- kazinator 4y agoThat is not exactly so, because the prompts are not reproducible input cases; you don't get the same image every time for a given prompt. An association between a prompt and, say, around ten images would be something resembling a cache.
- CuriouslyC 4y agoThe stable diffusion model just got officially released recently, and in the last week a lot of easy to install repositories have been forked off the main one, so it's very accessible for people to do this at home. Additionally, the model is very impressive and it's a lot of fun to use it.
- keepquestioning 4y agoHow does it compare to DALL-E
- CuriouslyC 4y agoWorse at image cohesion and prompt matching, but competitive in terms of final image quality in the better cases.
- astrange 4y agoIts image quality is often better, mainly because you can run it on your own machine and increase the quality/time controls. DALLE2 lacks settings and doesn't run enough diffusion steps to have fine detail.
- foobiekr 4y agoHaving had access to DallE for awhile, I find both Midjourney and stable diffusion to be quite a bit more powerful.
- soperj 4y agoFor weird inputs I find DALL-E to be better, but both have failed at really specific stuff that I've tried to create as art. ie: a factory with human lungs. With stable diffusion, it really creates some nice stuff for what is already available, like a pizza with specific toppings [0]. I've been using it to add pictures to any of the recipes that are added to my wiki site without a picture. I originally tried this with DALL-E with similar prompts and the results are less appetizing. [0] - https://www.reciped.io/recipes/mushroom-and-onion-pizza/ https://www.reciped.io/recipes/mushroom-and-onion-pizza/
- hbn 4y agoIt's an impressive new technology, and there's nothing else out there like it in terms of the model being publicly available and able to be run on consumer GPUs.
- dougmwne 4y agoFirst, it was recently released, so there’s novelty. Second, the code and model weights were also released so it is open and extensible, which this community loves. Thirdly, these high quality image generation models are mind blowing to most and it’s not hard to imagine how transformative it will be to the arts and design space. If if has any greater meaning, we might all be a little nervous that it’ll come for our jobs next, or some piece of them. First it came for the logo designers, but I was not a logo designer, and so on.
- foobiekr 4y agothe other issue is DallE and them Google/Imagen made a big deal about _not_ giving broad access to it and making it an approval-only beta (DallE) or simply you-can't-have-it (imagen). So people were hyped up.
- hooloovoo_zoo 4y ago
- dang 4y agoIf you have evidence, let alone clear evidence, please let us know at hn@ycombinator.com. Please don't post insinuations about it here—those perceptions are a dime a dozen and the overwhelming majority turn out not to be supported by data. This is in the site guidelines: "Please don't post insinuations about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix=true&page=0&query=by:dang%20astroturf https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
- hwers 4y agoWe could always do im2tex via just clip embedding the image. The idea that you could hide/sell prompts is silly. (Having human interpretable im2tex is cool tho.)
- tough 4y agoYep https://replicate.com/pixray/text2image https://replicate.com/pixray/text2image
- naillo 4y agoAre you literally a bot? This is the second time I've seen you link to replicate.com (this time the link isn't relevant to the parent).
- tough 4y agoHmm, no, are you dang? Are you the link police? Fuck off, look at my history if you want, I'm trying to be helpful. I'm a happy customer of replicate, yes, but I also ordered a GPU to run on local... You say it's not relevant but actually I'm giving exactly the functionality that parent is asking, giving back a prompt without having to pay for it, that's what text2image does. Also replicate links you to github, but in case you cannot run (not have a GPU at home) you can just run some free queries on it. What's to hate about that? I don't care what you think, You can also check my other post on this thread where I link to the official paper, and has 11 upvotes. Go bot yourself Oh, also, very funny to have a 2m old account giving me lectures about what I'm allowed to post or not, you can't even downvote...
- smusamashah 4y agoAs someone mentioned in another comment, selling prompts isn't a joke https://promptbase.com/ https://promptbase.com/
- kelseyfrog 4y agoIt has a lot more joke potential if prompts can simply be generated from images.
- recov 4y agoSomething about this just turns me off, not sure why.
- boppo1 4y agoDon't worry. People who can actually paint will mercilessly attack the insecurities of anyone who is too serious about prompts. The artists hardly made money before and the AI will just take more off the table. All they (we) will have left is spite.
- Kiro 4y agoA craft you can't earn money on is called a hobby. I have a lot of hobbies I don't earn a dime on. Why would art be any different?
- throwaway82388 4y agoI feel it too, the idea of rent seeking in the least difficult part of the process. But people calling themselves prompt engineers (without irony) is at least amusing, like a sanitation expert or a maintenance engineer.
- _1 4y agohmmm.... https://promptbase.com/prompt/hacker-news-viral-title-generator https://promptbase.com/prompt/hacker-news-viral-title-genera...
- 4y ago
- VMG 4y agoThe input image is just another kind of prompt. These models have hundreds of input parameters, not just prompts. There are many ways to configure the different models and various techniques and link up the processing stages. Getting the best results in a short amount of time requires highly specialized knowledge. The job description isn't "prompt engineer" but something close to that.
- frebord 4y agoHow many years until we can generate a feature length film from a script?
- bitwize 4y agoI want to see the Batman film where the Joker gives Batman a coupon for new parents but it is expired. That should really be a real film in theatres.
- anigbrowl 4y ago5 You could do storyboards from a shooting script* now, but generalizing to synthesizing character and camera movement as well as object physics is a ways off. * A version of the script used mainly by director and cinematographer with details of each different angle to be used covering the scene.
- bottlepalm 4y agoIt looks like this is trending towards making our dreams/thoughts reality in a way in that what we imagine can easily be turned into media - music, books, movies, etc.. Pair this up with VR 'the metaverse' and you literally do get the ability to turn thoughts into personalized explorable realities.. what happens after that? * Do we get lost in it? * Does today's 'professional' fiction become a lot less lucrative when we can create our own? * Is there a to leverage this technology the improve the human condition somehow?
- xdfgh1112 4y agoI can create and explore realities using my imagination alone, though. I personally don't think having it become actual 2d or 3d art will have a lasting impact. It might be fun for a while, but it will get old.
- globalvisualmem 4y agoThere is also recent work by Google called DreamBooth, though similar to Imagen/Parti Google refuses to release any model or code. https://dreambooth.github.io/ https://dreambooth.github.io/
- habitue 4y agoYeah, they allude to supporting more than one token for the identifier, which would be nice
- GaggiX 4y agoTextual inversion also supports more than one embedding for identifier, just change num_vectors_per_token in the yaml config. Example: https://www.reddit.com/r/StableDiffusion/comments/wzf1qk/sd_with_textual_inversion_bugatti_mistral/ https://www.reddit.com/r/StableDiffusion/comments/wzf1qk/sd_...
- lucidrains 4y agoso is dreambooth worth open-sourcing then given textual inversion?
- beyondarmonia 4y agoFrom the textual inversion guy's own comment on Twitter >The objective is similar, but it's: (1) A different approach - they also fine tune the model itself, and they get much much better identity preservation!
- deleted 4y ago[deleted]
- GaggiX 4y agodreambooth retains higher fidelity as the model is finetuned, but to be honest I think textual inversion is actually more applicable as you can just add some embeddings to inject new knowledge into the model and not an whole new model just for a single concept (if you want to share it with others). Also I have not seen dreambooth being applied to replicate styles.
- bottlepalm 4y agoWow, this is pretty cool. Instead of turning a picture back into text, turn it into a unique concept expressed as variable S* that can be used in later prompts. It's like how humans create new words for new ideas, use AI to process a visual scene and generate a unique 'word' for it that can be used in future prompts. What would a 'dictionary' of these variables enable? AI with it's own language with orders of magnitude more words. Will a language be created that interfaces between all these image generation systems? Feels like just he beginning here..
- tough 4y agoRight on, you could see a marketplace of those mini- pretrained weights to have stuff a styles much more available in a UI like setting without needing to add style manually... Very interesting.
- daenz 4y agoThis is a big deal! This adds a super power to communication, similar to how a photo is worth 1000 words. An inversion is worth 1000 diffusions!
- hbn 4y agoI saw talk the other day how these ML art models aren't really suited to doing something like illustrating a picture book because it can synthesize a character once but wouldn't be able to reliably recreate that character in other situations. Didn't take long for someone to resolve that issue!
- deleted 4y ago[deleted]
- zone411 4y agoIt's not quite at that level yet. The paper introducing it recommends using only 5 images as the fine-tuning set so the results are not yet very accurate.
- MacsHeadroom 4y agoThe model was released exactly one week ago. At this rate we'll be well past "there" in another week.
- Gigachad 4y agoThis doesn't feel too far off. With the img2img stuff you can give a picture of a character and the tool can spit out new images of that character or transpose them in to new art styles. It doesn't feel like this tech has hit any hard limits on things that are impossible or very far away yet. Every limitation seems to be getting broken at rapid pace.
- bravura 4y agoIs there a colab or easy to use demo of this?
- deleted 4y ago[deleted]
- tough 4y agoThis tutorial on reddit best close thing so far. https://www.reddit.com/r/StableDiffusion/comments/wvzr7s/tutorial_fine_tuning_stable_diffusion_using_only/ https://www.reddit.com/r/StableDiffusion/comments/wvzr7s/tut...
- macrolime 4y agoNote that generating "inappropriate" images on colab could result in your entire Google account getting banned. I wouldn't risk it.
- zone411 4y agoIt should be noted that the official repo now also supports Stable Diffusion: https://github.com/rinongal/textual_inversion https://github.com/rinongal/textual_inversion.
- ionwake 4y agoAnyone else starting to feel uncomfortable with the rate of progress?
- nmca 4y agoYes. https://80000hours.org/problem-profiles/artificial-intelligence/ https://80000hours.org/problem-profiles/artificial-intellige...
- smt88 4y agoI'm not worried about unemployment, although that is a problem. I'm more worried about bad actors being able to flood the web (even more than it already is) with realistic-enough content that makes it utterly unusable and unreliable. Imagine entire subreddits consisting of posts, comments, memes, and photos, and 100% of it is pro-[insert authoritarian regime] and it essentially only cost $1M to do it.
- flycaliguy 4y agoHonestly, I’m worried that shocking pornographic depictions of every women who’s ever posted her face online is coming. AI’s first big splash in our society is going to be a traumatic sexual assault of all women.
- thomashop 4y agoHow is drawing imaginary pictures sexual assault?
- userbinator 4y agoIn the same way that speech is now considered violence by an unfortunately large amount of the population, I guess.
- tgv 4y agoThat's not assault, but it can definitely be used for harassment and other forms of social damaging. I posted this an excerpt from an article (on anonymous Telegram groups) a few days ago: Filing charges is pointless, says Ezra. Since two years, she's being harassed on Telegram. It started when she was sixteen: photoshopped nudes with her snapchat account were circulated. They had taken selfies from her social media, and those of her family, and combined them with porn fragments. She doesn't know the perpetrator, but that person takes a lot of trouble to ruin her. "Nowadays, the boys have so many ways to make it look real." source: de Groene Amsterdammer,146/33, p. 21.
- cube2222 4y agoIt's impressive how all of this is quickly picking up steam thanks to the Stable Diffusion model being open source with pertained weights available. It's like every week there's another breakthrough or two. I think the main issue here is the computational cost, as - if I understand correctly - you basically have to do training for each concept you want to learn. Are pretrained embeddings available anywhere for common words?
- xur17 4y ago100% agreed. That's one thing that really bothered me about "openai".
- optimalsolver 4y agoDid OpenAI (Dall-E) and Google (Imagen) know Stable Diffusion was coming? I'm sure they were looking forward to many months of maintaining highly exclusive access and playing "too dangerous to release" games before SD completely upended the table.
- dannyw 4y agoToo dangerous to release, a codeword for selling access at 100% markup.
- jwitthuhn 4y agoBonus points: If dall-e rejects an output image because it thinks the image is inappropriate they won't show it. But they will still charge you for the prompt.
- O__________O 4y agoIs there even a public keyword list of words you’re not supposed to include?
- cjbgkagh 4y ago
- vinkelhake 4y agoRIP promptbase.com - they had a good run.
- kelseyfrog 4y agoThe won't even need to renew the domain name when it expires on 2023-02-28
- bongoman37 4y ago
- avocado2 4y agoweb demo for stable diffusion (txt2img): https://huggingface.co/spaces/stabilityai/stable-diffusion https://huggingface.co/spaces/stabilityai/stable-diffusion github with web ui: https://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion dev repo (more features, may have bugs): https://github.com/hlky/stable-diffusion-webui https://github.com/hlky/stable-diffusion-webui repo with docker: https://github.com/AbdBarho/stable-diffusion-webui-docker https://github.com/AbdBarho/stable-diffusion-webui-docker colab repo (new): https://github.com/altryne/sd-webui-colab https://github.com/altryne/sd-webui-colab can also run it in colab (includes img2img): https://colab.research.google.com/drive/1NfgqublyT_MWtR5Csmr https://colab.research.google.com/drive/1NfgqublyT_MWtR5Csmr demo made with gradio: https://github.com/gradio-app/gradio https://github.com/gradio-app/gradio
- frozencell 4y agoHey the colab link is dead :(
- nl 4y agoPeople seem to have missed the point of what this is. This is not "work out what the prompt is for an image" Instead it lets you give the model a "thing" (as an image) and then use that thing in your prompts. So for example they give it a picture of a statue and name it "S", and then can say "Elmo sitting in the same pose as S" and it correctly generates it.
- 01100011 4y agoHow far are we away from throwing a screenplay and some reference scenes at an AI and getting back a full movie? 10 years? Doesn't seem very far away now...
- tough 4y ago> https://github.com/rinongal/textual_inversion https://github.com/rinongal/textual_inversion Netflix UI of 2033: I want an episode of Senfield with X, Y, Z... Starting brand new (never aired) just generated by the AI episode now.
- ilaksh 4y agoIf you can ask for anything, why would it be Seinfeld?
- Jack000 4y agoAI: remake seasons 5-8 of game of thrones, based on GRRM's final two books which he finally finished.
- inawarminister 4y agoAI: remake seasons 5-8 of game of thrones, based on the critical consensus version of the final two books autocompleted by GRRM-style emulating generative network.
- tough 4y agoI wouldn't, tbh, It's just the example someone else gave used Seinfield, and I was trying to be relatable to an american audience, I never seen it. LOL
- adrianN 4y agoJust make sure not to ask for a Moriarty that can beat Data.
- jjcm 4y agoWhat I want to know here is can I seed it with a human face? If so, this really becomes a breakthrough for fast meme generation / other more nefarious uses. That will really be the thing that opens the floodgates on this.
- MacsHeadroom 4y agoYes, you can. For well-known people you can already upload a meme and ask SD to replace the face. I replaced a Biden meme with Shrek just yesterday using Stable Diffusion image2image in Colab.
- tough 4y agoThis is escalating quickly I'm experimenting with generative art with all this new models, and what's coming as output is wild and beautiful. You can basically make a full AI experience now as a one man show world building with the capabilities only prior to marvel or disney... This might be a better link> https://textual-inversion.github.io/ https://textual-inversion.github.io/ Also this reddit tutorial might be useful to you https://www.reddit.com/r/StableDiffusion/comments/wvzr7s/tutorial_fine_tuning_stable_diffusion_using_only/ https://www.reddit.com/r/StableDiffusion/comments/wvzr7s/tut...
- jcul 4y agoGenuinely mind blowing stuff. That link is better in my opinion. Showing multiple variables, styles Sx in the style of Sy; it's really amazing.
- debugnik 4y agoThe fact that they chose to recreate Qinni's artstyle, who passed away just a couple of years ago and had a notable online following that still remembers her, makes me slightly uncomfortable; feels too soon for me, I guess. Also, if models start to accept prompts like "in the style of Qinni", surely we're back to the copyright debate. They get away because everyone's art is mixed into a single model, but once distilling someone's artstyle is a feature…
- lbotos 4y agoArt style is not copyrightable as far as I understand. It could be trademarked though if specific enough.
- keepquestioning 4y agoI'm fearful of these algorithms, how can I ensure my economic status will not be affected? Any stock tickers to buy?
- Vecr 4y agoAmazon sells compute, and that's always needed for this sort of thing. They may develop new AI accelerator stacks (including racked hardware) in the future as well, so sure. I don't want to be one of those "the Internet is a fad" people, but I'm absolutely not telling you do do anything rash, go do your own research. Even if you bet at well as you can, the dot-com crash wiped people like you out, and that's assuming you get the general picture right.
- cbozeman 4y agoHow can you ensure your economic status will not be affected? How were farriers able to ensure their economic status wasn't affected by cars? How were switchboard operators able to ensure their economic status wasn't affected by automatic switching? How were travel agents able to ensure their economic status wasn't affected by Expedia, Priceline, and the dozen other sites out there? No one can know when or how their job is going to go extinct. You just need to pay attention to the changing winds of technology and adjust your ship's sail accordingly.
- rpigab 4y agoAnother example : https://en.wikipedia.org/wiki/Water_carrier https://en.wikipedia.org/wiki/Water_carrier
- namrog84 4y agoInvest in relevant technologies e.g. GPU compute cycle companies (e.g. nvidia/amd/intel/arm/etc..) or invest in cloud compute companies (Amazon/Microsoft/Google/etc..) Or learn relevant skills and try to have relevant marketable skills. (Always be learning)
- keepquestioning 4y ago
- davesque 4y agoIt's interesting that, in a certain phase of the learning process, words in natural language already work this way. Before a language learner has enough knowledge to understand word etymology, they have to just form an association between a word and a certain set of sensory experiences that the word is meant to represent. Seems like we have the same thing happening here. The input images are the sensory experiences. The dummy word "S*" is the linguistic symbol that is attached to them.
- andai 4y agoSee also: CLIP Interrogator https://colab.research.google.com/github/pharmapsychotic/clip-interrogator/blob/main/clip_interrogator.ipynb https://colab.research.google.com/github/pharmapsychotic/cli...
- LaughingCat 4y agoI wonder how this will impact market opportunity for Dall-E which already lost quite of wind thanks to their weird monetisation model (compared to a more affordable Midjourney, which does use Stable Diffusion).
- ccozan 4y agoI mean, what stops us a life-imitates-art system where we have speech2img like in the Westworld ( the narative creating scenes ) ? I guess, I hope someone reads this and will pick up this. Maybe coupled with a VR set?
- yewenjie 4y agoIs there an easily self-hostable all in one text to image thing?
- nickthegreek 4y agohttps://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion
- ManuelKiessling 4y agoTangential: I've set up a Discord Bot that turns your text prompt into images using Stable Diffusion. You can invite the bot to your server via https://discord.com/api/oauth2/authorize?client_id=1013373043625705513&permissions=117760&scope=bot%20applications.commands https://discord.com/api/oauth2/authorize?client_id=101337304... Talk to it using the /draw Slash Command. It's very much a quick weekend hack, so no guarantees whatsoever. Not sure how long I can afford the AWS g4dn instance, so get it while it's hot. Oh and get your prompt ideas from https://lexica.art https://lexica.art if you want good results. PS: Anyone knows where to host reliable NVIDIA-equipped VMs at a reasonable price?