8 ms·
Concept sliders: LoRA adaptors for precise control in diffusion models
- mpaepper 3y agoCan you apply the sliders also to existing images that were not generated with SD?
- dragonwriter 3y agoYou can only apply a LoRA (concept slider or otherwise) to the model (well. model “family”) it is based on, not to an image directly, its a model modifier than adjusts model behavior, not an image filter. You can use SD with image as an input in place of or in addition to a prompt, and you could apply one or more LoRA (concept slider or otherwise) to the model when doing so.
- verdverm 3y agoI wonder if this can also work for text generation?
- edkennedy 3y agoThere are so many overlaps between diffusion and LLM, I wonder this kind of thing all the time!
- UberFly 3y ago"Artists spend significant time crafting prompts and finding seeds to generate a desired image with text-to-image models." I wish there was a different description of those who generate art via AI than "artists". When ChatGPT writes me a story based on my prompts I'm still not an "author". Sorry, I know it's a tired argument already but it'll never stop bugging me.
- PeterStuer 3y agoFirst of all being labeled an 'artist' has little relation to the tools used to create 'art'. 'Art' by 'artists' does inlude the Sistene Chapel amf the Matthew Passion, but also trowing a bucket of paint at a canvas, wrapping a bridge, having ice melt and dragging a urinal into a museum. That does not make you an artist whenever you spill ketchup on your shirt, wrap a Chrismas gift or pour yourself a scotch on the rocks. Second, while you can indeed just type a prompt into Midjourney and some images will be generated, there are 'artists' using diffusion models to meticulously craft 'art' using hundreds of itrrations combining dozens of models and loras with inpainting. outpainting, recomposing and fusing, dtawing and more to come to a unique piece of 'art'. Third, didn't we have the same thing before? Are DJ's musicians? Some most definetly are. Creating new music by adapting, restructuring. remixing and extending existing tracks, while others use turntables as a real instrument. Others just put on some records at a wedding. It's about intention and context, and yes, creativity and skill, not about the tools used.
- taopai 3y agoGreat response. I would also add that art is also "created" when you consume it. When we consume it, we generate some interpretation, feelings or imagery. This is the last step. That's internal, but this processing of art is also art. That explain why some people find something as art and others as BS. As you say it's about intention and context, from the artist point of view and also from the consumer.
- iteygib 3y ago"Artist" in the context of image creation means you're the main supplier of the talent. In other words, you painted the picture yourself and spent time developing your talent. I can make a video recording of you making the picture from scratch with your own hand. AI "artist" isn't an artist. It's someone who uses an external product to automate the talent for them by creating an average image based on samples created by artists, of which were curated by individuals with particular tastes. It's why AI art mostly looks the same. It's also the reason a DJ a is called a disc jockey and not a musician. They are labeled differently to show the difference in the generation of the supplied content. You might as well say that someone who Googles "picture of a dog" is an artist as well since they learned the ins and outs of the search engine. A better term for AI Artist would be something that mentions the collage aspect of it.
- dragonwriter 3y ago> I wish there was a different description of those who generate art via AI than "artists" Typically, if you generate art, regardless of the tool or medium or form, you are an artist. There may also be a more specific term for artists working with/in a specific tool, medium, or form, but the general term still applies. Stable Diffusion is a tool used yb a person, not a person.
- deleted 3y ago[deleted]
- lbeltrame 3y agoWhile I won't ever call myself an artist, there are people who do not just click and get an image. My workflow involves getting the right prompt out, then manual touch up of the images (sometimes drastically so) using Krita and a Wacom tablet, or composing scenes with 3D mannequins from CLIP STUDIO PAINT and then feeding them to ControlNet (up to several CNs at once), generating and touching up again. I would argue there's a definite creative process here.
- iteygib 3y agoI completely agree with you. I think AI art is a good tool, but referring to the users as "artists" is confusing to actual practice. You aren't creating the content, but rather are dictating it like a patron or client. You are just telling an engine what you hope to see, in the same way someone might tell you what to paint while watching over your shoulder. It's also not a tired argument. You are just posting in a place that is very pro tech and not art-oriented. If you posted this in an art forum you'd get the opposite reaction.
- dataangel 3y agoArtist is still going to be the right word with these kinds of tools because you're still adjusting the result to get the desired look. Even if it were just collage it would still be art.
- iteygib 3y agoThey didn't make the content, they just adjusted it based on samples from others who made the art by hand.
- tguvot 3y agoso, if i retain painter/sculptor/etc to perform a job, and go with him for multiple iteration to get a desired look, will I be artist as well ?
- tomatotomato31 3y agoEither yes or perhaps curator?
- klik99 3y agoIt's semantics, but if you consider Jeff Koons to be an artist, then yes. AFAIK he has never (or at least when he was starting) used his own hands to fully create something, from the very beginning he hired people to do it for him. The "Desired Look" can be considered the "artistic" part of art, while the production can be considered the craft. Koons is the most extreme example I can think of, but there are plenty of others who got their start creating stuff themselves and then hired people to do the total of the work. Is a director an artist, when he doesn't write the script, touch the camera, edit the film, or act in the movie?
- artninja1988 3y agoSure. Why not?
- tguvot 3y agoand if i'll retain programmer to develop software, will after 50 sprints with my feedback i will be called programmer ?
- fzzzy 3y agoI wish there was a different description of those who generate art via Word Processor than "artists". If they aren't impressing upon a cuneiform tablet I am not interested.
- artninja1988 3y agoDo you consider photographers artists? I really don't see why we should be gatekeeping the term either way
- UberFly 3y agoMore composers than artists.
- dontupvoteme 3y agoMore like medicine - Science and an Art. Ironically the photography community already went through this sort of discussion as the Art of the darkroom became the 'art' of photoshop Though the scale is lessened due to the significant barrier of entry for high end photography (a few grand per body and per lens)
- minimaxir 3y agoAI art and its modern tooling ecosystem is at a point now that it unironically takes more effort to generate an intentionally innovative AI image than it is to draw it from scratch. Just look at ComfyUI: https://github.com/comfyanonymous/ComfyUI https://github.com/comfyanonymous/ComfyUI
- spookie 3y agoHonestly, that happened to me. Was making some UI elements for a game, and tried to generate some. After a while, I was so tired of getting results that weren't quite what I wanted that I just drew them. It certainly has its place, but the "its taking people's jobs" argument kinda fell apart for me after that moment
- MrScruff 3y agoIf you find ComfyUI offputting you really wouldn't like the nodegraph based applications we use in VFX.
- minimaxir 3y agoThat is a correct inference.
- whywhywhywhy 3y agoPeople honestly overcomplicate it, the variety you can get just out of base SDXL without even using loras or other models and comfy nodes is crazy once you get a feel of it but a lot of people make the mistake of not spending the time on it to figure out how to push it in the direction they want and instead think piling on a bunch of loras and an extra model is the key to getting something different.
- tracerbulletx 3y agoIt unironically absolutely does not. Source: every visual artist that's spent the last 20 years slowly improving their work.
- Der_Einzige 3y ago
- flessner 3y agoThis discussion might be heavy on semantics, but I don't think you are wrong for bringing this up. AI can be used as a tool to empower authors and artists to create more and/or better works, but it can also be used by a lazy student to finish their 2 page writing assignment in seconds. The only thing AI will clearly do over the next few years is raise the bar. Beginner art or copywriting is already outclassed and assignments will assume access to AI tools, similarly how many assume internet access nowadays. Nevertheless, I truly believe that there will always be space for human art... just look at chess, with engines far better than humans, what are the tournaments about? the players.
- deleted 3y ago[deleted]
- neural_thing 3y agoOoh, I've been waiting for this. Exciting! Can't wait to try it out in a tool.
- z3phyr 3y agoI always confuse the subject with the radio technology LoRaWAN
- deleted 3y ago[deleted]
- devsda 3y agoHad the same thought. It might already be known to those keeping track of AI tech but I wish the article mentioned what LoRA meant atleast once. For those in the similar boat, it apparently means Low Rank Adaptation (for Large Language Models)
- dragonwriter 3y ago> For those in the similar boat, it apparently means Low Rank Adaptation (for Large Language Models) Well, the abbreviation expands to Low Rank Adaptation, but the article is about diffusion models, not LLMs, and LoRA is an model-modifier approach that applies to many classes of models, not just Large Languag Models.
- barnabyjones 3y agoSo I suppose the main difference with using LoRAs normally is that less of the image is changed? Because it seems most of this could be done with inpainting and increasing LoRA strength, but I admit it would be difficult to keep facial features the same the way they seem to have done. What I notice are missing from these examples are actions/poses, so I wonder if it's good for that or we still need openpose/controlnet.
- yorwba 3y agoThese are LoRAs being used normally, and the level of modification is controlled by varying LoRA strength. The main difference is in the training procedure, where they try to only target specific attributes while leaving others unchanged. You could probably use this to create LoRAs for a specific pose, but if you want to try out many different poses and freely adjust them, the more flexible controlnet approach is likely to be more comfortable.
- dragonwriter 3y ago> So I suppose the main difference with using LoRAs normally is that less of the image is changed? The distinct thing about concept slider LoRA isn't how much of the image is changed (which will vary between LoRA within the type widely), but that the weight at which the LoRA is applied, rather than just setting how strongly the image tends toward representing a fixed concept, chooses which concept on a continuum the image tends towards. > What I notice are missing from these examples are actions/poses, so I wonder if it's good for that or we still need openpose/controlnet. You can do actions/poses via LoRA, but the control you get is qualitatively different than what you get with any of the controlnets, so its good to have both tools available. I haven't seen concepr slider LoRA specifically being used for poses, though conceptually doing something like seated<->standing or standing<->running as a concept slider would make some sense.
- deleted 3y ago[deleted]
- itpcc 3y agoWait, how IoT radio thing related to AI stuff?
- stavros 3y agoThat's LoRa, not LoRA.
- the-anarchist 3y agoFrankly, these acronyms are becoming absurd. I'm really hoping for somebody to build an insanely successful programming language called LOra, just for the sake of giving everyone nightmares.