12 ms·
Prompt Engineering Guide: Guides, papers, and resources for prompt engineering
- dpflan 4y agoNice, thanks. I have limited knowledge, but let's choose one example, GPT, are people using GPT to create prompts for GPT?
- 88stacks 4y agoyes,I am doing it all the time.
- williamcotton 4y agoLarge Language Models Are Human-Level Prompt Engineers https://arxiv.org/abs/2211.01910 https://arxiv.org/abs/2211.01910
- msoad 4y agoThe whole prompt engineering thing feels like a temporary stopgap. Because ideally language models should be able to understand prompts from anyone without them trying to craft in a way that works for the model.
- dpflan 4y agoThat's what I'm wondering, if using LLM to create prompts for the LLM is a few steps away from integrating that into the LLM and unlocking recursive phenomenon and emergent behavior.
- imranq 4y agoThis is actually the premise behind Anthropic's Constitutional AI (https://arxiv.org/abs/2212.08073 https://arxiv.org/abs/2212.08073) where they have a set of principles that guide a workflow where the LLM creates the prompts and the critiques. They were recently given $300M by Google, so certainly you have a promising idea there
- dpflan 4y agoAh, so Google' response to the OpenAI+MSFT collaboration...
- krab 4y agoNot really. It's just a higher level programming language. You still need to know how to decompose a problem and structure the solution. On some level, also systems thinking as this stuff will get integrated together.
- msoad 4y agowhy can't the model do those things? As someone said, it's like Google-foo. It used to be the case where you had to give Google some context to disambiguate certain queries but now it is really good so I don't have to write anything crazy like site:wikipedia.com etc
- williamcotton 4y agoLarge language models will probably never be reliable computers. The solution is either providing translation examples (aka, in-context learning, few-shot) or fine tuning a model to return some symbolic method to tell an associated computing environment to evaluate a given bit of math or code.
- barking_biscuit 4y agoIn many ways this is sort of what humans do. They have something they don't yet know how to do and they go away and learn how to do it.
- williamcotton 4y agoIt definitely feels like I have to work to maintain the state of each algorithmic step when I'm multiplying two three digit numbers in my head. It's a lot easier if I can maintain that state on a piece of paper. I'm definitely not calculating the path of a projectile in a similar manner when I catch a ball. I'm definitely not "computing" a sentence when I read it in the same way that I compute the multiplication of those two three digit numbers!
- 4y ago
- Havoc 4y agoAgreed. Much like google fu there is still likely to be some skill involved, but calling it "engineering" seems a bit much
- CharlesW 4y agoAgreed, "prompt design" (for example) would've been a better characterization. I'm sure people are already putting "prompt engineering" on their resumes.
- layer8 4y agoGoogle also defines "engineering" as: 2. the action of working _artfully_ to bring something about. "if not for his shrewd engineering, the election would have been lost" (https://www.google.com/search?q=define%3AEngineering https://www.google.com/search?q=define%3AEngineering) Merriam-Webster has: 3 : calculated manipulation or direction (as of behavior) giving the example of “social engineering” (https://www.merriam-webster.com/dictionary/engineering https://www.merriam-webster.com/dictionary/engineering) Random House has: 3. skillful or artful contrivance; maneuvering (https://www.collinsdictionary.com/dictionary/english/enginee https://www.collinsdictionary.com/dictionary/english/enginee...) Webster's has: The act of maneuvering or managing. (https://www.yourdictionary.com/engineering https://www.yourdictionary.com/engineering)
- __loam 4y agoIn Canada you can't call yourself an engineer unless you've actually been certified as an engineer. In the US, engineering programs worth anything are typically ABET certified. We should stop throwing that word around like it doesn't have a centuries old profession associated with it. It cheapens what actual trained engineers do and have done to earn their training.
- fassssst 4y agoIt’s not, it’s what all these services are doing behind the scenes so users don’t have to.
- jlawson 4y agoI disagree because when I want to get results from flesh-and-blood humans, I still have to engineer my prompt very carefully (and often get it wrong!). This includes asking questions, and trying to direct someone to effectively complete a task. Prompt engineering is just communications skills as applied to AI instead of meat-minds.
- smoe 4y agoI think as of now one big difference is that when talking to a competent flesh and blood human, they will ask (or at least try) the right questions before giving an answer. E.g when talking with my accountant, she usually ask me a bunch of clarification questions and numbers, instead of just making a best effort, but confident sounding response with whatever initial context i gave. I myself don’t have to have the depth of knowledge to direct someone to complete a task step by step to get the right results.
- jlawson 4y agoVery interesting point. Perhaps a big step forward for chat AI interfaces is to make the AI capable of knowing when it needs to ask follow-up questions and then having it do so. Essentially, it helps you along with your prompt engineering. Perhaps this is what the prompt engineering tools are doing anyway. What we need to do is integrate the prompt engineering tools with the chatbot itself, so it can both help extract a good prompt from users and then answer that prompt in the same process. I think this is where we'll move towards relatively soon; it seems obvious when you say it.
- williamcotton 4y agoIf anything, we're all learning pretty quickly that a chatbot interface is probably not the ideal way for interacting with an LLM!
- JohnFen 4y agoThe underlying problem is that human language is imprecise. Even humans frequently misunderstand what each other is meaning when they say things, because very often there are multiple valid ways to interpret a statement.
- kleer001 4y agoIthkuil, however, is very precise :) https://en.wikipedia.org/wiki/Ithkuil https://en.wikipedia.org/wiki/Ithkuil
- simonw 4y agoI've seen this argument a bunch recently, and I firmly disagree with it. I just published this explaining why: https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-engineering/ https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-e...
- 13415 4y agoEven if it doesn't become obsolete as an activity the knowledge you build around it becomes obsolete faster than almost any other tech sector. Whatever you learn by trial and error now about one particular model will be completely outdated in two years from now.
- BiteCode_dev 4y agoIdeally people should be able to understand sentence from anyone without them trying to craft in a way that works for the person. But they don't.
- deleted 4y ago[deleted]
- ketralnis 4y agoWe don't have to do that to communicate with humans because they're aren't one-shot. Humans ask followup questions like "Polish as in people from Poland, or polish as in rubbing until it's shiny?". Or they know you both work for a translation company and you don't talk about shinifying objects very often. If you did have to send google-esque one shot queries to humans you'd probably settle on short hand like "polish shiny" or you'd opt to use "poland" specifically to avoid it. For most known-ambiguous terms you'd do this, the same way we say out loud "M as in Mancy" because we know the sound of the letter can be ambiguous sometimes. We have lots of these in English that you use all of the time without knowing it. In a smaller audience the local language gets even more specific, you probably disambiguate people with your friends like "programmer Bob, not carpenter Bob". It's not at all crazy that we'd develop another local language for communicating with computers even if that's not a traditional programming language
- Xelynega 4y agoExcept we do that for humans and it's still not really relevant to language models. If I were talking to a python programmer I could assume they knew what a for loop was, so could phrase a question requiring the context of one differently than I would for the layperson. Just like if I were designing inputs to one language model I could assume it's capabilities that are different from another. I don't think it's relevant to the performance of language models for that same reason though, we already have to design out queries with the thing being queried in mind so I don't see why we wouldn't for llms.
- selfhoster11 4y agoI’ve been able to make ChatGPT ask for follow-up clarifications if it’s unsure about the meaning of something. I didn’t explore this further, but the ability is right there and can be triggered with the right prompt text.
- Xelynega 4y agoI fail to see the distinction you're trying to make. Creating the input for the llm to get a specific result is crafting an input in a way that works for the model and it's something we even do between different people(you probably have different ways to ask your boss and spouse to do something).
- paxys 4y agoThis IMO is the key distinction between "true" AI/AGI and the current generation of LLMs. ChatGPT cannot understand what you are telling it. It does not have a brain and cannot speak conversational-level English. This is why the entire area of prompt engineering needs to exist. Whether it is a stopgap or not depends on if we can actually make these models understand what they are asked rather than just pattern match.
- didntreadarticl 4y agoI much prefer leisurely engineering
- nickpinkston 4y agoCame here for this joke - thank you :-)
- GartzenDeHaes 4y agoThis reminds me of Hitchhiker's Guide to the Galaxy. We've built systems that can answer any question, but now we need even more powerful systems to tell us what the question is.
- djyaz1200 4y ago42
- runlaszlorun 4y agoNo… no… that’s the answer. The question would be: 42?
- williamcotton 4y agoI've been developing a methodology around prompt engineering that I have found very useful: From Prompt Alchemy to Prompt Engineering: An Introduction to Analytic Augmentation: https://github.com/williamcotton/empirical-philosophy/blob/main/articles/from-prompt-alchemy-to-prompt-engineering-an-introduction-to-analytic-agumentation.md https://github.com/williamcotton/empirical-philosophy/blob/m... A few more edits and it's ready for me to submit to HN and then get literally no further attention!
- ed 4y agoHow is this different from e.g. a python agent like https://github.com/hwchase17/langchain-hub/blob/master/prompts/llm_math/prompt.json https://github.com/hwchase17/langchain-hub/blob/master/promp... ?
- williamcotton 4y agoUsing the terminology that I'm working with this is an example of a second-order analytic augmentation! Here's another approach of second-order analytic augmentation, PAL: https://reasonwithpal.com https://reasonwithpal.com Third-order, Toolformer: https://arxiv.org/abs/2302.04761 https://arxiv.org/abs/2302.04761 Third-order, Bing: https://www.williamcotton.com/articles/bing-third-order https://www.williamcotton.com/articles/bing-third-order Third-order, LangChain Agents: https://langchain.readthedocs.io/en/latest/modules/agents/getting_started.html https://langchain.readthedocs.io/en/latest/modules/agents/ge... The difference isn't in what is going on but rather with framing the approach within the analytic-synthetic distinction developed by Kant and the analytic philosophers who were influenced by his work. There's a dash of functional programming thrown in for good measure! If anything I filled a personal need to find a way to think about all of these different approaches in a more formal manner. I have scribbled on a print-out of the article on my desk: Nth Order - Existing Examples [x] (added just now) - Overview [] - data->thunk->pthunk []
- ComplexSystems 4y agoThis seemed interesting. So if I get your idea correctly, rather than talking to the chatbot directly, you first run your prompts through some algorithm which increases the chances of the AI getting what you are asking and giving a successful result?
- tekbog 4y ago<T> Engineering
- booleandilemma 4y agoThese guides are going to feel so dated ten years from now. It's like reading a guide on how to use a stylus for a palm pilot.
- pixl97 4y agoYes and no. Human like AI isn't going to be magic because humans are not magic. You are still going to have to comprehensively communicate with your AI so it understands your reference frame. If you walked up to the average programmer on the side of the street and threw a programming fragment at them, especially if its a difficult problem, they will have a whole load of follow up questions as to place the issue you want to solve. Coming at a human or AI with expectations and limitations first will almost always lead to a faster and better solution.
- time_to_smile 4y agoThe more seriously I see people take the topic of "prompt engineering" the more deeply depressed I become about this field. It feels like tutorials on tailoring the emperor's new clothes. At the same time I feel it's only a matter of time until my job interviews involve questions like "can you provide a few example of where you've used prompt engineering to solve a problem? Describe you prompt engineering process?" and I'm just not sure I can fake it that hard.
- freetinker 4y agoI feel exactly the same way. Let’s do our part and not allow “engineering” to be co-opted in this manner and completely lose all meaning. I’ll settle for “prompt trials”.
- behnamoh 4y agoI feel the same way about “data science” because there’s no “science” in it; could just call it statistics or whatever, no data science expert is actually called a “scientist” by the society, because we all know it’s not really science.
- JohnFen 4y agoThe popular confusion/conflation of "science" with "engineering" has long been a pet peeve of mine. They are very different things.
- Der_Einzige 4y agoIf you publish work in a journal or conference that's meaningfully important in your field you are a scientist.
- YurgenJurgensen 4y agoSeems like you forgot that Mathematics is a field that has journals and conferences.
- 4y ago
- williamcotton 4y agoOne recent paper to be added to this list of Approaches/Techniques is: Toolformer: Language Models Can Teach Themselves to Use Tools https://arxiv.org/abs/2302.04761 https://arxiv.org/abs/2302.04761 The authors show a valuable method of using prompts to generate training data to be used for fine-tuning. Check out Appendix A.2 in the paper for the example prompts.
- ajhai 4y agoIf anyone is looking for an easier way to manage their prompts in production (versioning, templating, caching and testing), please checkout https://trypromptly.com https://trypromptly.com. We built this out of our experience using Open AI's completions API at https://makerdojo.io https://makerdojo.io which is now powered by Promptly. (Testing and sharing are WIP)
- cloudking 4y agoInterestingly enough the best results I get from ChatGPT for writing code come from prompting it in a "pseudo code" style, with variables and logical instructions written in plain English.
- adamrezich 4y ago"engineering" is an interesting way to frame "throwing things at the wall and seeing what sticks"
- AlexCoventry 4y agoYeah, this seems a lot like calling management "mandate engineering" or something. You don't really understand the processes by which it's working, so referring to anything related to a using LLM to get useful outputs as "engineering" seems a bit high-flown.
- pixl97 4y agoUm, I think you may be slightly confused about many physical processes and how they are engineered. There are plenty of problems where we understand that if we do A between the bounds of (X and Y) then everything works, and if we go out of those bounds things break. When building the process you don't have to understand why A works at all as long as you stay in those bounds, you just have to design the process either not to exceed them or to stop and give a warning if it does.
- AlexCoventry 4y agoI'm engineering this conversation with you right now.
- pixl97 4y agoWelcome to Sorites Paradox
- int_19h 4y agoYeah, but we already did it with software engineering.
- runnerup 4y agoRespectfully but realistically, this better fits “Prompt Analyst” than “Prompt Engineer”.
- mlsu 4y agoThis morning, I brushed and flossed, which makes me a Plaque Removal Engineer. I then used my skills as a Room Tidyness Engineer to make the bed. After that, I engineered the harness onto my dog and took her on a walk: Canine Fitness Engineer. I engineered the water to a higher temperature using the kettle, and poured it over some coffee grounds to create a chemical reaction in my morning brew: Yeah, that's right, Caffeine Engineer. After this incredibly productive morning, I got in the car and drove to my job as a computer programmer.
- runnerup 4y ago…”respectfully”
- BiteCode_dev 4y agoWe found that having pre-determined sections are indeed very useful. One for context, one for rules, one for output format, one for input data, etc. Also, you can as for a structured output (gpt spits out JSON fine) but you need to be super explicit about each field possible values and the conditions they appear in. Templating prompts with jinja goes a long way for testing. And you will need a lot of testing, if for nothing else than remove variability in answers. It's funny because people keep saying gpt will removing all values from having writer skills, but being a good writer helps a lot with crafting good prompts.
- williamcotton 4y agoI've been writing prompts with the intent of producing JSON that contains JS functions to evaluate. Since the response needs to be parsed and evaluated in a specific structure it means that changes need to be made on the evaluation side as well as the prompt template side. So I've been writing the translation examples (few-shot examples [not my favorite term]) as TypeScript, transpiling to JS and compiling with other examples and a prelude, and using the same method to build the translation examples as to build future prompts. It saves a lot of silly mistakes! JavaScript has less tokens than TypeScript, so it seems more economical to convert to JS beforehand instead of passing TS to the LLM! I wouldn't be surprised if TS resulted in better solutions, though... add it to the endless list of things to test out...
- BiteCode_dev 4y agoThe problem with that approach is that is very hard to verify that the output is consistent. If you get pure json, you can check it against a schema, and regularly check that chatgpt is not outputting BS (which is does easily). With turing complete code, it's way harder.
- williamcotton 4y agoThe approach I've been using is that I test against a corpus of questions with known answers. Using sample-and-vote, that is asking for some number of variable completions with a non-zero temperature and treating the explicitly computed solutions as votes, has smoothed over issues with basic syntax and logic errors. This is pretty similar to the approach used in the Toolformer paper, other than sample-and-vote, which I believe is novel.
- monkeydust 4y agoLangchain is a fantastic library to use but also educate yourself on the dark art of prompt engineering.
- flowersjeff 4y agoReally?... Isn't the 'underlying' skill ( and it was hard for me to write that last word )... being able to communicate? Are we really going to call this anything but the ability to write one's thoughts intelligently?
- simonw 4y agoI just published this, as a counter to the "prompt engineering will be made obsolete as AIs get better" argument that I keep seeing: https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-engineering/ https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-e...
- pixl97 4y agoWe need a name for this, something like genie effect, but that doesn't sound creative enough. But working of the fables of the genie, we have and all powerful wish machine where the user mistakes they are telling the genie with intention of their thoughts, whereas the genie is looking at all possible decodings of the words they are saying without knowing the users thoughts. If you do not verbalize exactly what you want (and have done some thinking about the ramifications of what you want) you might end up squished under a pile of money, or some other totally ridiculous situation. Unless you share your entire life with the AI/genie you will always have to give more detail than you think. Hell, in relationships where people live together for years there is always the possibility for terrible misunderstandings when communication breaks down.
- simonw 4y agoI've used an analogy of magic and spell casting to prompting in the past, but I've since been convinced that it's risky to describe it like that because it implies that this stuff is all-powerful and impossible to understand: https://simonwillison.net/2022/Oct/5/spell-casting/ https://simonwillison.net/2022/Oct/5/spell-casting/
- devmunchies 4y agoIt seems to be a theme with AI. Scale.ai (no affiliation, not a customer) has a product called Spellbook https://scale.com/spellbook https://scale.com/spellbook
- simonw 4y agoYeah, I've definitely had informal conversations with AI experts where they've talked about collecting spells for their spell books!
- gregsadetsky 4y agoSuper grateful for this guide! If there's anyone here who's become good at perfecting llm prompts & available for a freelance contract -> please contact me (my email is in my profile) It's easy-ish to get gpt to generate "good enough" text, obviously. Like any tool, what's interesting and complicated are the use cases when it doesn't follow instructions well / gets confused, or you're looking for a more nuanced output.
- deleted 4y ago[deleted]
- cwkoss 4y agoI've seen a boom of people selling ebooks, etc. with "46 Revolutionary ChatGPT prompts" with the idea that someone found some cool behavior that could be economically useful. However, I think this kind of 'engineered prompt' sharing is about as useless as sharing the engineering calculations for building a bridge over a particular river : none of those calculations are generalizable and all must be redone for each future placement. Chances are you'll need to tweak any purchased prompt to fit your own use cases, and if you are capable of tweaking purchased prompts, why can't you just chat with the bot for an hour or two and build a working prompt for yourself? I think the value of "prompt engineering" as a skill is all about adaptability and testing and verification processes: what is the process of going from idea to functioning prompt for a particular use. Silver bullet prompt fragments are cool, but not really products - the only people buying these are suckers who don't understand how to use chatGPT. Because chatGPT will happily help you make these prompts for free.
- saurik 4y agoFWIW, I watched a relatively trivial video on using ChatGPT a couple nights ago that had like 6 prompts, and while I had seen people do some of them before it was still useful: it showed me how people were using the project and massively accelerated getting to useful results. People buy books of suggestions for how to do literally everything from song writing to project management... is it really surprising that how to get the most value out of talking to an AI--something many people might find a bit strange and unnatural--is something that you could help people be better at? If the same eBook had been "46 revolutionary prompts for your next team brainstorming meeting" or "46 revolutionary ice breakers for your next party" would we even bat an eye?
- ComplexSystems 4y agoDo you have a link to the video?
- saurik 4y agohttps://youtu.be/EYjG6i53-xk https://youtu.be/EYjG6i53-xk Honestly, my favorite moment has nothing to do with the thesis of the video: it is when she says "zero-shot chain of thought" and stops talking for a moment to comment about how that rhymes, as it definitely only rhymes because of her accent; but I'm really into linguistics and, in particular, phonology, so, for me, that kind of moment is magical. But like, the point is: if you watch that video and replace the idea of you talking to ChatGPT with talking to one of your coworkers, this is a totally legitimate video someone would make. You can't quite just use a book of brainstorming ideas, though, with ChatGPT, as it has some fundamental limitations to how it understands certain kinds of concepts or language, and so seeing "oh that path worked well" is practically useful.
- NicoleJO 4y agoThanks -- this will help out a LOT in copyright infringement cases!
- travisgriggs 4y agoIn another universe the Just In Time mantra infected how we did engineering, and became known as "Prompt Engineering". But in this universe, it took a different turn. Words are a funny thing. Ironic.
- imranq 4y agoEssentially what large language models allow us to do is move through an n-dimensional space using only words. Somehow we need Google Maps for semantic spaces, but to do that we need to have a strong definition of what a "semantic space address" is and what "semantic roads" look like
- barking_biscuit 4y agoThis is a cool way to conceptualize it. When conversing with humans, you can explore various parts of the semantic space, but the process also involves a range of top-down and/or bottom-up mechanisms that facilitate a kind of 'hidden navigation'. LLMs, on the other hand, lack this capacity but with the right prompts, one can simulate it by knowing how to steer, which is sort of what prompt engineering feels like to me.
- 13415 4y agoThe prompt engineers will be the first self-proclaimed engineers to be replaced by AI.
- celestialcheese 4y agoI've found that context is everything to getting consistently good output, and augmenting your prompts with known truths with SERPAPi, embeddings and a vector db, brought really flakey results into the >90% accuracy threshold. As an aside - does anyone have good tools or methods for testing and evaluating prompt quality over time? Like performance monitoring in the web space, but for prompt quality. The techniques to use LLMs as evaluation tools of themselves always has seemed flakey when I've tried it, I'd like to use a more grounded baseline. For example, if you have a prompt that says "What is the weather today in {city}?", you can run it against a list of cities and expected outputs (using a lookup to some known truthful API). That way, when you make changes to the prompt, you can compare performance to a baseline.
- 30minAdayHN 4y agoWe personally came across the evaluation problem while building MakerDojo[1]. My current workflow is to run manually on 50 different test cases and compare the results against the previous versions of the prompt. This is extremely time consuming. And to be honest, I no longer test for every little change. Some more contect - as a way to support MakerDojo[1], we are building TryPromptly[2] - a tool to do better prompt management. In that tool, we are building the ability to create a test suite, run the test suite and compare the results. At least knowing for which test case the results varied and reviewing them would go a long way. Here is the format we are thinking: https://docs.google.com/spreadsheets/d/1kLBIb7W0jrY-IkNPqJsNeh8CdGcJw7E-1-fulKZJrv0/edit?usp=sharing https://docs.google.com/spreadsheets/d/1kLBIb7W0jrY-IkNPqJsN... In addition, we are about to launch after test suite is to have live A/B tests in the production based on user feedback. Users can upvote or downvote their satisfaction with the results and that will inform you which version of the prompt yielded better results. If you have other ideas on how to test them better, it would be super helpful to us. [1] MakerDojo - https://makerdojo.io https://makerdojo.io [2] TryPromptly - https://trypromptly.com https://trypromptly.com
- leobg 4y agoPrompt engineering. The pick-up lines of the NLP world…
- stevofolife 4y agoOn a relevant note, does anyone know any good prompt guides for vision models like Stable Diffusion, Midjourney, or Dall-E?
- menacingly 4y ago"Prompt Engineering" is a fictional attempt to create space for human intermediaries, and arguments against this are weak and pointless. Disregarding where it is _today_, the goal obviously is for natural language to produce the desired output. It happens to fail at that today, but poking around at whatever arbitrary, unpredictable mutations to the input will get you there (without explanation) isn't a skill. Even the becoming-cliche "humans don't know what they want" argument is its own counterargument. You're right, humans do not know how to precisely ask for what they want, and yet manage today to navigate around this all the time without learning a new way to communicate.
- karaterobot 4y agoI'm not sure how you're defining skill, but it seems like you can get better at prompt engineering, and some people are consistently better at it than others. I'd call it a skill, just like framing a search query in Google is a skill. Sorry if that's weak and pointless.
- menacingly 4y agoyou can get better at astrology, I'd still advise against carving out a niche there. Claiming you can tame a hurricane of untraceable parameters with guesswork, on rapidly changing systems whose designers have the goal of obviating the need to format queries in magical ways, is dicey at best.
- karaterobot 4y agoI think you're basically saying that this thing is a) a black box, and b) more complex than any human can pretend to understand, even if we cracked open the box and looked inside. But, that does not close off the possibility of learning what ways we can poke and prod it to affect the results. It would be interesting to run an experiment to test whether some people are consistently better at generating desirable results from an AI model. My money would be that they can, even at this early stage of "prompt engineering" as a discipline, let alone 5 or ten years from now. You may also be saying "don't call it engineering, it feels more like black magic," a position I would be sympathetic to. But, I think a lot of realms of engineering deal with uncontrollable elements we don't understand and have to just deal with, with increasing levels of control as our disciplines become more sophisticated.
- quadcore 4y agoTangential, has someone tried to have two chatGPTs chatting together? Like the good old joke which consists of dialing two random persons and have them discuss together.
- Towaway69 4y agoWasn't COBOL a first attempt in having a natural language interface to the machine? Is this the next attempt to make the human-machine interface simpler? Why is it that we create something with which we can't communicate but that we expect to solve all our problems.
- malwrar 4y agoI’m someone who hasn’t jumped onboard this space—isn’t this “prompt engineering” concept just for OpenAI’s ChatGPT or is there some broader reason to expect prompt engineering to be relevant? I’m extremely hesitant to rely on ChatGPT or its current interface as a tool since it exists at the mercy of a massive profit-hungry corporation. It feels too early to start inventing formal concepts against a single black box API.
- zoover2020 4y agoIs is relevant for all LLMs, for instance if you're asking Stable Diffusion to generate an image of a castle. Composing the right prompts here will allow you to obtain actually what you want versus, let's say, a castle as drawn by a toddler (okay maybe that's what you want, but that's not the point)
- williamcotton 4y agoOne of the emergent properties of large language models is that they allow for "in-context learning" by providing "zero-shot", "one-shot", "few-shot" training examples. This seems to be consistent across all sorts of transformer based models with language capabilities. Certain kinds of prompts result in certain kinds of completions and "prompt engineering" is a collection of methods related to encourage a specific completion. For example, prompts that do not contain enough facts about a desired question will result in "hallucinations", whereas prompts that have additional context programmatically added to the question will result in more reliably factual completions. I find it useful to think about this grounded in terms of the analytic-synthetic distinction from analytic philosophy. The above guide is an excellent resource! You should read it!
- malwrar 4y agoI’ll give it a read then! Good to hear it’s more generic than I thought.
- paulosalem 4y agoI shared my tutorial (with code) on the topic before, but here's it again (just sent a PR to this repo too). https://medium.com/data-science-at-microsoft/building-gpt-3-applications-beyond-the-prompt-504140835560 https://medium.com/data-science-at-microsoft/building-gpt-3-... In short, Prompt Engineering is just one of the pieces in building a GPT-3/LLM-based solution. In fact, I'd say a whole new set of Software Engineering best practices is necessary and will gradually emerge. I gave one such approach that has been useful to my related projects.
- alphanullmeric 4y agoPrompt “engineering” reeks of the A in STEAM
- jeffrallen 4y agoPrompt "engineering" is about as engineering as software "engineering".