7 ms·
Prompt Engineering Is Hard
- engineer_22 4y agoIt seems to me that this is very loose use of the term 'engineering'
- bottlepalm 4y agoIt’s funny even though you can create passable art easily with AI, the results are still infinite and finding that special prompt that matches your vision is not easy. People who create top AI art that get lots of ‘stars’ in the SD discord are easily spending days coming up with the right prompt and tweaking it. It’s hard and personally I’m not as good as them (there is skill involved) it’s a different kind of hard than what traditional art has been. Different tools, brush vs keyboard, but similar in that you need vision of what you want to create with those tools.
- astorsnk 4y agoIs it really skilled? I feel like it's just throwing shit at the wall to see what sticks. You have an idea, write a prompt, and then edit and re-run that prompt repeatedly until you accidentally get what you want.
- mahathu 4y agoI think I agree, but couldn't the same be said about Mozart or Bach writing a symphony? They have an idea, write a score, then iterate on it until they accidentally get something that sounds good.
- Bakary 4y agoThis does not take into account the years of practice and the natural talent that led to these composers being able to do this. In comparison, any schmuck can learn how to find prompts within a few hours, and can get pretty good at it within a few weeks.
- bottlepalm 4y agoLike I said, people trivialize and discount how hard it is to create good AI art. It’s a lot tougher since everyone now can create something passable, it comes down to vision. Is your vision great enough to create award winning AI art? Mine isn’t, yours probably isn’t either.
- Bakary 4y agoThe current type of AI art is less than a year old, so no person can have more than a year of experience, at most. Most artists spend years and years developing their craft, and it involves daily deliberate multi-sensory practice. By comparison, all a prompter has to do is play around with types of words and iterate over a few weeks. Yes, they might need some eye for detail or composition or some intuition with the sort of words that could work, but since the AI is a blackbox anyway it cannot logically be much more but throwing mud and finding what sticks.
- bottlepalm 4y agoThe real limit with AI art is imagination. The bottleneck of physical ability has been removed. The new bottleneck is imagination itself. The more you have, the better your AI art will be. You should try sitting at a prompt and making something compelling.
- Bakary 4y agoDon't get me wrong, I am in fact excited by this technology. The applications are clear in terms of generating new art, in particular in hybrid contexts where a person talented in one sphere only could finally make it work e.g. graphic novels I suppose what bothers me is the idea of a prompt user seeing themselves as an artist, not because I want to gatekeep people's minds but because that misunderstands the nature of the technology. A visitor to an AI art gallery (I do mean material that is 100% the direct result of prompt generation) could be able to generate maybe not the exact same art but something of equivalent worth upon arriving at the door. The contribution of the human being or to be more precise the specific individual is almost immaterial. Sure, they might have come up with a prompt with many words but you could come up with any number of equivalent artworks in the same breath. The artist is wholly interchangeable in a way that did not exist before. I'll admit that it does require cultural understanding of art and some understanding of how prompts work because you need to know the names of artists and styles to fit into the prompt. But beyond having a good vocabulary there's not that much more involved. By comparison, if we're talking about artwork that uses AI as a tool as opposed to being the entire thing, then it becomes much more interesting and is what I am excited about because there will still be meaningful human input in there.
- bottlepalm 4y agoIt’s a skill if you can get what you want quickly. It’s also a skill to create art better than everyone one else when it is so ‘easy’ to create the bottleneck is only your imagination.
- O__________O 4y agoMy experience is that anyone that says predictably generating art is easy using AI is either lying, never done it, or exclusively copying prompts others have posted. Beyond that, lots of people are unable to even explain how prompts are parsed or per token explain how it impacts what is generated. For example, do commas matter? “Prompt engineering” is real, still unclear to me though what the future for it looks like, since easy to see via comparable things like Google search operators that the general public has little interest in learning command parameters.
- danielvf 4y agoOne fun tip for anyone who wants to understand more how their prompts work - lock down to a single seed, no variations, then change one word at a time in your prompt. Then you can see what effects each word has. Or you can answer your question about the effects of commas. You can try moving a word or phrase forwards or backwards and see what effect it has. By working with a single snapshot of randomness, you can see the word effects more clearly.
- O__________O 4y agoAgree; to the expand on your points, even trying ultra simple prompt like [black circle on a white background] and then running the output through CLIP to get its text outputs. Then try to narrow the iterations between the two to be predictable and controlled by you. Obviously understanding the code and data the model is based on is likely best path to better understanding what is going on; for example, last time I checked, Stable Diffusion was limited to 77 tokens. Ultimately though, possible paths to understanding are like unlimited and just as creative as any other aspect of the topic.
- zorked 4y agoPeople are way overvaluing this thing right? AI "art", prompt "engineering". All I see is people with too much time throwing words at a parser without understanding what it does until something sticks. And the result is probably a copyright violation of some poor artist's work.
- O__________O 4y agoLittle strange to comment that users do not understand a technology, then comment that outputs are copyright violations; which to me says you neither understand copyright, nor how the technology works.
- astorsnk 4y agoLegal or illegal, there's a big ethical debate to be had. Swallowing up the work of others and using that to generate derivative/new work that _a lot_ of people (I've seen half a dozen stock photo AI sites this week alone) are then trying to use to replace the original creators doesn't feel right.
- O__________O 4y agoArt history is literally full of transformation of prior works, technology is clearly transformative (if you understand it), copyright law clearly allows for it, and indexing publicly available data is also legal — there’s no reasonable basis for debating the topic being illegal or unethical.
- tsbinz 4y agoAnd yet, every time GitHub Copilot comes up, people on HN are in arms about their code being used to train a model.
- O__________O 4y agoAlready responded to comparison to GitHub CoPilot in this thread here: https://news.ycombinator.com/item?id=33079278 https://news.ycombinator.com/item?id=33079278
- d--b 4y agoIt's only a matter of months before someone comes up with a half-text / half-image prompting method for these models. As in: you'll be able to draw a bunch of circles and arrows with labels in them like "astronaut" / "horse" => "sunset" and some environmental info like "in the style of van gogh" and bang you get a proper image.
- lifthrasiir 4y agoIn some sense img2img can already do this, I have seen enough artists trying to leverage img2img in their creative process (as opposed to txt2img, which really seems only useful for ideation).
- willy_k 4y agoimg2img and in/outpainting are already two forms that this idea exists in. Maybe not with textual labels in the image itself, but the concept of having a starting image that a model can modify for a desired change is definitely already a thing.
- jrussbowman 4y agoI've seen some people use () and other syntax on the stable diffusion subreddit. I've been trying to find a guide for the syntax but haven't had much luck with Google. Is there a resource for this?
- jaqalopes 4y agoThe idea of prompt “engineering” strikes me as nonsense. I’ve spent many hours on Midjourney, and it makes some astonishing artworks. But it essentially never makes what you expect. I see that as a feature rather than a bug. Instead of professional prompt engineers who somehow know the precise incantation to generate exactly the picture that’s in your head (they don’t), I think the actual future direction of this business is large numbers of people skimming through thousands of fairly similar images and varying the preferred ones over and over to inch towards the desired output. Of course, all that will be obviated when the AI itself advances by another big step, anyhow.
- pwinnski 4y agoMidJourney is more heavily tuned to produce its "house style" unless you overwhelm it with keywords, and Stable Diffusion and other models are tuned differently. If you ask MidJourney for a "futuristic solder in armor," you'll get... something. If you ask for "Hardmesh retro futurist steampunk fallout 7 6 power armor head, hyper realistic, art gta 5 cover, official fanart behance hd artstation by jesper ejsing, by rhads, makoto shinkai and lois van baarle, ilya kuvshinov, ossdraws, that looks like it is from borderlands and by feng zhu and loish and laurie greasley, victo ngai, andreas rocha, john harris radiating a glowing aura global illumination ray tracing hdr"[0] you'll get something that is less obviously MidJourney. 0. I grabbed that prompt from lexica.art as something work-safe.
- BiteCode_dev 4y agoI find it hard too. I just read on reddit about https://aiprompt.io https://aiprompt.io and I find it useful to kickstart a new generation. It's a good way to start and discover technics by serendipity. It's not teaching you tricks, but it's great for beginners or when inspiration is lacking.
- O__________O 4y ago(Meta: Might be worth adding (Warning: NSFW) next to your link, which appears to just provide per selected random prompts for users.) Curious, if you were to imagine what a truly useful tool would be like for teaching how to create prompts?
- BiteCode_dev 4y agoI have no idea I'm just getting into this SD wonder.
- O__________O 4y agoAgain, there’s nude on the page you linked to; you need to edit your comment to warn users.