7 ms·
So long “prompt engineering,” we hardly knew ya
- orzig 3y agoI hate approximately everything about this article, but I'm glad that I took a second look through because this is a decent framework for the-thing-which-he-swears-isnt-prompt-engineering: --- * diagnosis * decomposition * reframing * constraint design. Diagnosis is discovering the problem that AI can solve. This is the human part of knowing that a problem exists. Learning to ask the right questions, look at the different ways that the problem can be seen. Decomposition is about splitting the big problems into bite-sized ones. Take the problem apart, examine it, and let AI help you determine your findings since it handles data so well. Instead of tackling the biggest problem, take it apart and work on the smaller parts to achieve small successes. Reframing is about shifting your perspective and seeking new interpretations. Extrapolating and recombining the parts of the problem in order to identify the meta components. Perhaps a new way of looking at the problem may find a solution hidden in plain sight. Constraint design is about setting boundaries for the solution. Knowing what to accomplish, and when to know it is done. Setting the length, style, and description of the audience can help AI understand its mission. But we have to know that first in order to instruct. --- As someone who asks GPT and junior developers for lots of things, there are a lot of similarities. I don't imagine that is going away, at least until we wire LLMs up to a huge amount of rapidly changing, cross-silo context so it could understand "Fix the monitoring that slowed our recognition of yesterday's bug". So being thoughtful isn't going away. The author agrees with that (see above), but doesn't make clear where he draws the boundary between "being thoughful" and "prompt engineering"
- Macha 3y agoThat's just systems analysis though. To call rediscovering that prompt engineering just because you're dealing with AI rather than programmers siloed away from business processes feels a little silly and just being justified to keep the title when as originally formulated by the people who first started using it, it was about learning the "magic words" for a specific LLM
- fnordpiglet 3y agoSee I always saw prompt engineering as the ability to author a prompt that elicits a specific behavior by the LLM, not simply the incantation to make to it say penis or talk like a sailor. The “make the AI be lewd” was a parlor trick, but the art of making it behave the way you want and to get the answers you want at the depth, breadth, and form you need is a skill and one I think isn’t going away in our lifetime. I’d further note that writing to influence humans to your intended response and understanding is a widely studied field by every study with multiple PhD and beyond disciplines associated. We even elect such people to lead us largely through their ability to prompt engineer humans. IMO - Folks seem to be reacting to the term “engineer” in the same way traditional engineers with engineering exams do to software engineers claiming to be engineers despite often graduating from a LAS school or not at all - I have a bunch of PE EE in my family and they treat my claim to be an engineer with extreme scorn.
- RC_ITR 3y agoThis article’s argument is interesting because it relies on two very questionable principles: 1) That as tools get better, operator specialization is less useful. I understand how this feels like it makes sense, because for for binary tasks like driving it’s true, but for creative tasks, I struggle to think of a single example where this is the case. Which leads to 2) Humans won’t figure out how to use these tools in increasingly complex/weird ways to create increasingly complex/weird outputs. I think in general, everyone has been so conditioned by the idea of singularity (which to be frank is a completely tangential concept to contemporary LLMs) that they refuse to see these things for what they are: tools built by humans to serve humans when operated by humans. Add as many layers of “self-prompting” as you want there, but a human still set the original intention and they will be the ones to judge the ultimate outputs.
- parpfish 3y agoi feel like people that get really into prompt engineering seem to forget that other people are working on building models that are even better. Do they really think that the tricks they developed to make gpt3.5 work well will still be necessary on gpt9 (or whatever new model replaces gpt)? if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff.
- JimtheCoder 3y ago"if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff." I think this is more for people, though, that want to maximize the use of AI in their own field, isn't it? The "knowledge worker enhancer"? I am not sure they really need to work on the actual AI stuff...
- soderfoo 3y agoIt felt like a mix to me. People seeking to enhance their work and the low effort high reward seeking hustle bros. The latter latches on to the surface level of any new tech but proceeds with swagger befitting a Nobel laureate.
- kaycey2022 3y agoYou know I'm something of a "Google User" myself
- pessimizer 3y agoI don't agree. There's a huge need for people to figure out how to integrate AI into larger things, and to establish ways we'll work with it - and at every step, including GPT 3.5 prompt engineering. There's a lot you can come up with if you just assume that good generative AI exists, and will probably get better. Different iterations of AI might as well be pluggable modules.
- golergka 3y ago> if you want a long-lasting career in internet, you need to work on the actual internet stuff, not just using the internet - a C developer that writes modem drivers at a telecom company in the 90s about html/perl developers building first interactive websites
- jemmyw 3y agoI think a lot of prompt engineering is magic thinking. It apparently worked once so you keep iterating on it without knowing if it has a positive or negative effect on any continued use. Some folks at work have got really into it and have some elaborate prompts to use before asking code related questions. I cannot see any different in their output vs mine when I just straight ask gpt4 questions.
- huslage 3y agoThe writer tears down prompt engineering while extolling the virtues of well-written prompts. Also GPT-4 isn't "learning new things all the time" in that way...it's getting updates, sure, but not constantly or in real time. These articles are pure nonsense.
- swyx 3y agoeveryone thinking Prompt Engineering will go away dont understand how close Prompt Engineering is to management or executive communications. until BCI is perfect, we'll never be done trying to serialize our intent into text for others to consume, whether AI or human. Where I DO think Prompt Engineering goes away is because codegen and code orchestration of LLMs rises up to take its place. Hence Prompt Engineer -> AI Engineer https://www.latent.space/i/131896365/the-role-of-code-in-the-evolution-from-software-to-software https://www.latent.space/i/131896365/the-role-of-code-in-the...
- minimaxir 3y agoAs someone who’s spent an unreasonable amount of time working with prompting for both text and text-to-image, I can confidently say prompt engineering does work for any nontrivial use cases (e.g generation with constraints). I don’t like that prompt engineering works, but it does, and proper use of it does result in objective improvements in generation quality. The latest ChatGPT model, gpt-3.5-turbo-0613 has better system prompt steerability, and with some prompt engineering I can get GPT-4 quality results out of it at a fraction of the cost. Ultrapopular tools like LangChain and AutoGPT are essentially just prompt engineering under the hood.
- binkHN 3y agoI concur; in using this for a side project, even a few words can get you from a fair ChatGPT completion result to an excellent one.
- tracerbulletx 3y agoEven if these models become AGI someday, there's still sort of "prompt engineering", because there is still social engineering. What and how you say something is always going to have an effect on the response.
- broast 3y agoPrompts are programs. They take input and produce an expected output. You can parameterize them and they can be used as functions and chained together. In that way, prompt engineering is software engineering.
- sib 3y ago>> Prompts are programs. They take input and produce an expected output. I think one key difference is that (most) programs that people are used to writing tend to be deterministic (when not intentionally random...) whereas LLM prompts pretty much always end up with an nondeterministic output.
- mrleinad 3y agoSo, the new thing is that we're going to call everyone an engineer? Carpenter -> Wood engineer Tailor -> Fabric engineer Ceramist -> Clay engineer See how dumb it looks? And maybe the field is too new to have a specific word for it, but calling it "engineer" diminishes the level of actual engineers.
- hazn 3y agoCalling carpenters wood engineers is pretty much warranted. There should be no engineering police that determines which jobs are allowed to carry the prestigious engineer title. Software engineering was also ridiculed for the longest time by “real” mechanical engineers.
- ghaff 3y agoAs a former mechanical engineer I’m not sure how common that was. Computer Scientist was/is a common alternative but that’s somewhat of an artifact of CS often being associated with math departments.
- throwaway42401 3y agoI don't see why we have to take and muddle someone else's word, "engineer" doesn't have inherently prestigious syllables, the word sounds good when someone says it at a party because the job itself is actually hard or whatever. If software development (or even "programming"!) is the same, the same thing will happen. If not, any other words we steal will lose their shine too and engineers will start calling themselves something else to get it back.
- cypress66 3y agoCarpenter and tailor are basically unique words. If you never heard of the word carpenter, you'd have no idea what it meant. It is very rare I think for completely new words to emerge nowadays. Prompt engineer is descriptive and you can guess its meaning.
- b1n 3y agoBut doesn't an engineer's authority come their skill-set and not from peoples perception of them or their job title? When you say diminished, are you are talking from a societal point of view? Prompt Engineering is a fun title in a fast moving and interesting field. Everyone's taking it far too seriously. I'm going off to become a Clay Engineer, that sounds fun!
- bsaul 3y agoAm i the only one that tried using gpt4 for serious development work but stopped pretty fast because of its obvious limitations ? Reading that article makes me wonder if we're even talking about the same thing.
- leaveitalone 3y agoI use it all the time. What kind of problems are facing?
- GenerocUsername 3y agoYou are not the only one, but also your experience is not the only experience with the tool. I have used several of 'the big' SaaS GPTs, and have gotten great use from them. There is absolutely a use for these tools, but they do take some small amount of skill to get good results. It all comes down to the context you can provide to steer the answer to what you really need. The better you can describe your current state and what you want from the answer, the better the answer will be.
- bsaul 3y agoWhen working on a project, context is everything. I feel like it'll only really make a difference once it's able to read the thousands of files in my project's folder. My only use so far is get inspiration for type naming, and very simple scripts that i'm too lazy to write myself. But this happens less than once a month.
- two_in_one 3y agoI'm using it for hobby development, which is more or less serious. Most useful cases I found: 1. API, concepts, etc docs with explanation. Much faster then googling then scrolling through tons of texts with blinking adds 2. Write simple things that I don't want to think about. Like in Python process all files in a directory, follow the links. Saves time. 3. Try things I don't know how to do. The recent was checking if user pressed a key without blocking in Python. Nontrivial, but possible. We went through several options till found the one which works on Ubuntu. So, it's useful, no regrets about subscribing. Funny thing I'm using it working on toy GPT
- Animats 3y agoI wonder what happened to those people who got 5-year "Knowledge Engineering" degrees from Stanford in the 1980s. They were trained on how to write rules for "expert systems", which turned out to not be very useful. Where are they now?
- puffyengineer 3y agoThis has to be the most incoherent article I've ever read.
- doctor_eval 3y agoI doubt that prompt engineering is going to go away, but it will probably change. In a couple of years, perhaps it will come to be known as “machine psychology”?
- Der_Einzige 3y agoThis article is garbage with a capital G I'm going to link my own snarky titled by actually okay-ish take on prompt engineering, emphasis on the actual engineering, to show that the author doesn't even know what prompt engineering is - https://gist.github.com/Hellisotherpeople/45c619ee22aac6865ca4bb328eb58faf https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c... Prompt engineering is not designing a cool prompt. It's when you start applying genuine techniques to do things that are not possible with tokens alone. For example: "What's the definition of {apple|orange}" where {apple|orange} is the mathematical average of those two words. This is prompt engineering. Right now, Prompt Engineering is basically in Stable Diffusion through Automatic1111, it's in libraries like microsoft Guidance or LMQL, and not a whole lot else.
- treprinum 3y agoThis is a weird article not understanding what prompt engineering is. Basically with GPT-3.5 one can say bye bye to fine-tuning in most cases as natural language directives (prompting) combined with chain of thought can now beat most fine-tunings. That only works with extremely large models though. One can use prompt engineering for things like "the person I like talks this way: ... Please talk like this person". Or one can use prompt engineering to simulate one-shot NER and collect all the data appearing in the conversation without any super complex ML model pipelines. I fail to see how this goes away once GPT-274745 can write its own prompts.
- yousif_123123 3y agoI don't think prompt engineering is the problem. I find myself with minimal education but a lot of trial and error being able to "engineer" a good prompt eventually if I know the problem and can validate answers. I think the big problem is actually finding problems you could use LLMs in. I think anyone that's played with them tends to have a good guess at whether they could or couldn't do something (although you need to really test to be sure, do some prompt engineering etc), but actually finding problems to work on that are within the realm of being solvable and useful is the hardest part imo.
- radicalriddler 3y agoPersonally, I look at prompt engineering the same way I see "software engineering" and think it's a bit of a mockery of the trend. Other industries have proper certifications for "engineering" that I.T. absolutely doesn't, yet someone who figured out how to center a div calls themselves a software engineer. https://www.theatlantic.com/technology/archive/2015/11/programmers-should-not-call-themselves-engineers/414271/ https://www.theatlantic.com/technology/archive/2015/11/progr... <- related article. I reckon if you're going to give devs the right to call themselves engineers just because they write code to solve business problems, you don't have much of a leg to stand to judge people whom write natural language into a software solution, and receive output that solves business problems.
- satisfice 3y agoThe style of this article is all arched-eyebrows and “keep up with me I see the future” BS. Clear away the faff and ignore the “AI is improving sooo fast ohh what a magic” remarks. What you’re left with is a tepid article about prompt engineering.