9 ms·
Prompt engineering
- zerop 3y agoSuch a useful information. If they published it no, I wonder why it wasn't there.
- deleted 3y ago[deleted]
- brandall10 3y agoAs far as I can tell, this is the same guide that has been up for awhile now. Is there something new or notable here that explains why it's climbing up the HN page?
- swyx 3y agoyes it has been up for a while. looked for it in hn search but couldnt find, i guess this is the first time someone posted it here
- deleted 3y ago[deleted]
- verdverm 3y agoThere are better guides out there too - https://www.promptingguide.ai/readings https://www.promptingguide.ai/readings - https://github.com/dair-ai/Prompt-Engineering-Guide/tree/main https://github.com/dair-ai/Prompt-Engineering-Guide/tree/mai... - https://github.com/microsoft/promptbase https://github.com/microsoft/promptbase (this one is less of a guide, but is likely the current SoTA)
- mkmk 3y agoI also strongly endorse https://www.udemy.com/course/prompt-engineering-for-ai/ https://www.udemy.com/course/prompt-engineering-for-ai/, if you're into the Udemy-style of learning.
- seydor 3y agoprompt engineering shouldnt exist. everyone knows how to use language. if people need to learn a new language to communicate with the machine then the idea of language modeling has failed. plain language, with appropriate GUI to upload information, or other kinds of interactions should be enough to make the system obvious to use without any tweaks
- MattRix 3y agoThis is a guide for working with the LLMs we have now, not some perfect future models. The reality is that certain prompt engineering techniques allow you to get better results. On top of that, humans still require training and instructions for how to write and speak to get the most impact out of their words even when interacting with other humans. The reality is that certain communication techniques are more effective than others, and not always in ways that are intuitive or obvious.
- geph2021 3y agoI think you're missing how this technology works. The basis for these models is training data. And to train anything, you need to label the data. So prompt engineering is simply using the lexicon, terminology and language labeling choices that were in the training data labeling. I suppose the models can conflate different words and terms, but to some degree the more you do that, the less precise or specific you can be in your prompt, and the generated result.
- kevindamm 3y agoIt's less like "you're holding it wrong" and more like "look, when you hold it like this the waves are different!" .. you can be pretty successful with basic language prompts but you can also be deliberate in the way you give instruction. Much like you can achieve different results with real people if you present your statements with some attention to the intended audience.
- minimaxir 3y agoPrompt engineering exists because a) LLMs are trained to optimize for statistically average accommodation of the dataset and b) Sturgeon's law: "ninety percent of everything is crap". Therefore, LLMs out-of-the-box will give worse-than-ideal results by design. The initial proof that prompt engineering worked was around the VQGAN + CLIP days, where simply adding "world-famous" or "trending on ArtStation" was more than enough to objectively improve generated image quality. The workaround to prompt engineering is RLHF/alignment of the LLM, but everyone who has played around with ChatGPT knows that isn't sufficient.
- bhouston 3y ago[flagged]
- shashashasha___ 3y agonothing you said seems related to the article at hand. are we pushing for a different agenda with this type of unrelated comments?
- hotdogscout 3y ago[flagged]
- lvh 3y agoHow do you know this person (Tal Broda) made this? It doesn't seem to list an author.
- bluefishinit 3y ago[flagged]
- hotdogscout 3y ago[flagged]
- bhouston 3y agoI have corrected the comment.
- unsupp0rted 3y agoI wish my colleagues wrote questions to me the way OpenAI expects me to write questions to ChatGPT. Provide context? Too busy. Write step by step? Delimit the question from the context? Why not just copy-paste an error message from somewhere then write below it "please advise"?
- drunkpotato 3y agoWhat luxury! I’d kill for a copy-pasted error message! Usually I just get “it didn’t work.”
- extr 3y agoPeople notify you that something didn't work? What a treat! For me the work just doesn't get done, and you have to ask why before you get an explanation that there is some technical blocker.
- zurfer 3y agoPeople tell you that something did not work, after you ask them? I need to manage an email sequence with 8 follow ups to get explanations.
- unsupp0rted 3y agoI've built email notifications into the major stuff, so on failure, my bot sends them an email with the error message. I've got my address in the reply-to. So they reply "please advise" and hit send. For the minor stuff... a copy-pasted error message from the UI is luxury, you're right.
- ofrzeta 3y agoI don't know about your work but there are tools/libraries to instrument your applications. For native applications there are things like ABRT, for web applications there's Sentry and a lot of other tools.
- deleted 3y ago
- dmortin 3y agoIt feels like talking to a reluctant employee who does his job halfheartedly and requires elaborate explanations to do an acceptable job.
- SOLAR_FIELDS 3y agoConsidering the cost of that reluctant employee vs the cost of this machine that seems like a great deal
- m12k 3y agoIt's like having a very fast, enthusiastic, eloquent, but also quite sloppy junior employee as an assistant.
- tosh 3y agoMost (all?) of the strategies described here also work with other language models. Prompt engineering fundamentals are useful (think: get more out of the model at hand) but also transferable. The deeplearning.ai course by Andrew Ng in collaboration with OpenAI has similar content: https://learn.deeplearning.ai/courses/chatgpt-prompt-eng https://learn.deeplearning.ai/courses/chatgpt-prompt-eng
- MattRix 3y agoMost of these strategies will get better results when working with other humans as well!
- fhqwhgads 3y agoYES THIS!!! I always say they're human roleplaying machines. Pretend it's a human, do the same thing you would do with a human on your best day and you'll get better results. Prompt engineering for me is about empathy in a way, learning to understand where the model's attention goes and leaning into that.
- bluefishinit 3y ago[flagged]
- hotdogscout 3y ago[flagged]
- cornercasechase 3y ago[flagged]
- hotdogscout 3y ago[flagged]
- cornercasechase 3y ago[flagged]
- hotdogscout 3y ago[flagged]
- nlh 3y agoI know for mass adoption LLMs need to support natural language input. But we've done a reasonably good job (note: source for endless arguments here ;) over the past ~80 years of developing a very precise system for inputting exactly what we want a system to do in the form of programming languages. I'm curious whether any of the leading models - LLMs, image generation models, etc. have taken this into consideration. Particularly in more precise I/O domains (image generation comes to mind), it seems like a structured input format where we remove the entire problem space of natural language prompt -> user intent would make things dramatically easier to get the output we want.
- ta988 3y agoWhat LLMs are really good at is guessing from fuzziness which is something formal languages are usually bad at. I can often ask for things I don't know much about in the wrong way and it gets to a response that shows where I was wrong and what may have misled me.
- asim 3y agoAgreed. This is really about defining a command and query language that's much more like the commands in a terminal or a cli. I think the fact that we're moving to this verbose approach is a sort of anti pattern and we'll see levels of abstraction or different methods to reduce it down once again.
- openaikillers 3y ago[dead]
- zzzzzzzza 3y agosounds similar to semantic kernel
- cornercasechase 3y ago[flagged]
- nradov 3y agoThere is no such thing as prompt "engineering". It's basically just trying different approaches until you come up with something that subjectively seems good enough. Nothing wrong with that, but let's not confuse the issue by labeling it as engineering.
- fl7305 3y ago> It's basically just trying different approaches until you come up with something that subjectively seems good enough. > let's not confuse the issue by labeling it as engineering. In my view, "trying different approaches" is a good description of engineering throughout history. Sure, it's excellent if you can base your engineering on a detailed physical model that lets you mathematically optimize a solution based on your boundary conditions. But compare that with metallurgy before we had atomic models. It was a process of trial and error. "Let's add small amounts of different alloy metals and see which ones makes the metal harder / more pliable / stainless / etc". That's still engineering to me. If anything, it could also be called science.
- nradov 3y agoWhat you're describing is artisanal or craft work. It's a crucial aspect of human endeavor but it's simply not engineering. Real engineering requires a foundation in accepted scientific theory and a consistent body of knowledge. Call it "prompt crafting" or something like that.
- fl7305 3y agoWas there an "accepted scientific theory" on metallurgy in the 1800s when gigantic metal ships were built? Were they not designed by engineers?
- nradov 3y agoYes, scientific knowledge of metallurgy based on analytical chemistry was fairly well established by the 1880s when the first successful large steel ships were built. It's impossible to produce large volumes of steel with consistent properties on an artisanal basis because the raw materials are inconsistent. Ship designers had also developed sophisticated mathematical techniques for calculating the strength of metal structures, and optimizing for weight and cost. They certainly weren't just riveting pieces of steel together by intuition and hoping it would float. I do consider those naval designers from the 1880s onward to be true engineers in the modern sense of the word. (At the time, engineers were mostly steam engine operators, so the meaning has changed since then.) Prior to the 1880s, large ships were generally composite wood and cast iron construction. While there was an aspect of engineering involved it didn't require the same level of theoretical knowledge and design was more artisanal. But that's a gray area.
- 1970-01-01 3y agoDo we understand why prompt engineering is still necessary? Why it is unable to correctly determine ("understand") what output the user wants from unstructured input?
- deleted 3y ago[deleted]
- verdverm 3y agoConsider how this works with humans. Often you don't have enough input, context, or information to provide the desired answer or outcome. Humans typically realize this and ask questions, we do this so much you typically don't take note of it. LLMs have yet to do this in my experience.
- 1970-01-01 3y agoThis makes sense to me. LLMs would benefit tremendously by using clarification prompts. Instead they spew output with whatever confidence level their creators deem is good enough.
- M4v3R 3y ago> LLMs have yet to do this in my experience I think the key missing ingredient of current AI systems is the lack of internal monologue. LLMs are capable of asking questions, but currently you need explicitly prompt it to deconstruct a problem into steps, analyse these text and decide whether a question is warranted. You basically need to verbalise our normal thought process and put it in the system prompt. I imagine that if LLM could do few passes of something akin to our inner monologue before giving us a response they would do a lot better on tasks that require reasoning.
- verdverm 3y agoThis is being worked on, look into Chain/Tree of Thought applications What is missing for me is it recognizing that it lacks enough information to provide a sufficient response, and then asking for the missing information. - typically, it responds with a general answer - sometimes it will say it can give a better answer if you provide more information (this has been increasingly happening) - however, it does not ask for specific information or context, it doesn't ask what if, or if/else, kinds of problem decomposing questions I do expect these things to improve as we are reaching the limit of raw training data & model sizes. We're primarily in the second order improvements phase now for real applications. (there are still first order algo improvements happening too)
- minimaxir 3y agoThese examples are for more simple prompt engineering demos. With the ChatGPT system prompt, you can give the model a large and complex set of rules to account for and recent models of ChatGPT do a good job of accommodating them. Some of my best system prompts are >20 lines of text, and all of them are necessary to get the model to behave. The examples are also too polite and conversational: you can give more strict commands and in my experience it works better. There's also function calling/structured data support which is technically prompt engineering and requires similar skills, but is substantially more powerful than using the system prompt alone (I'm working on a blog post on it now and it unfortunately it is going to be a long post to address all of its power). Here's a fun demo example which compares system prompts and structured data results: https://github.com/minimaxir/simpleaichat/blob/main/examples/notebooks/schema_ttrpg.ipynb https://github.com/minimaxir/simpleaichat/blob/main/examples...
- fragmede 3y agoMind sharing a system prompt of yours? 20+ lines sounds useful
- minimaxir 3y agoUnfortunately those were for specific work use-cases so I can't share them but the tl;dr is that every time the model does something undesired, even minor I add an explicit rule in the system prompt to handle it, or some few-shot examples if the model is really bad at handling it. That list can balloon quickly.
- verdverm 3y agoHere's a big one I needed to get ChatGPT to do something more sophisticated with a JSON object response (predates functions and all that) https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/prompts/dm.cue https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro... It no longer worked after a model update some time ago, haven't tried recently. I found codellama to be much better for this and require fewer instructions, an anecdotal validation for smaller, focussed models
- throwup238 3y ago
- supafastcoder 3y agoWe’ve trained a whole generation of people to “prompt” Google to get what they need from the internet with a few keywords (often autocompleted before you’re done typing), so asking them to now start writing elaborate prose to get what they want is just going to take a lot of time. I also think this is a temporary phase where we’ll converge back to keyword autocompletion soon (or other more efficient ways of interaction).
- emodendroket 3y agoOne thing I've noticed is people other than me are more likely to phrase Google searches in natural language anyway rather than using keywords.
- al_borland 3y agoEvery time I see someone do this I start wondering how common it is. Is saying to Google, "Please show me the finial score of last nights Detroit Pistons game." more common than my "Pistons game" query that gets me the same result? I've also seen this with various voice assistants.
- emodendroket 3y agoSeems like maybe Ask Jeeves just got undercut by regular search engines learning to ignore irrelevant query words
- aseipp 3y agoMy best friend does this and it both drives me insane sometimes but also blows my mind. It's how he has always used Google search. He does not use keywords at all, he simply states the generalized phrase out loud. Like if we're talking about a movie with Dean Winters in it, and I say "You know, it's the guy from those auto insurance commercials who would pretend to be a little girl in a driving accident." And he goes, "Hey Google" to his phone -- "Funny talented actor who pretends to be little girl in a funny auto insurance commercial" and "Dean Winters" is the first result or whatever.
- prvc 3y agoFrankly, this shouldn't be necessary. There are so many easy gains to be had in implementing an LLM-based chat app which do not require any theoretical advances compared to what we have now. All that's needed is a bit of elbow grease from the implementers.
- spenczar5 3y agoCould you give an example?
- prvc 3y agoPlanning can be implemented (at high computational cost) trivially by generating hidden responses. New models can be trained to natively query "authoritative" sources of information, such as databases and computer algebra systems. New models can be used to transform prompts into more effective ones (along the lines of TFA).
- ianbicking 3y agoI don't know... some of these are about being clear about what you want, and would work with people just like they work with the LLM. Or a lot of what happens in a conversational chat interface is what could happen in a one-shot full prompt; and maybe that's fine for a casual user but if you are programming something you should put in the effort to get that initial prompt right so the conversation isn't as necessary. I do agree about planning; one of the disappointments of Custom GPTs (among many!) is that you can't do this planning without letting it all hang out for the end user. That is, it would be great if you could tell the Custom GPT to put its plans inside <plan>...</plan> tags and have those filtered out (or at least hidden by default; they shouldn't be _secret_, but they are distracting). But even so in that case deciding that you need a plan, and what kind of plan, is something that can and probably should go in the prompt. Not all "plans" are the same, just as not all "summaries" are the same – and part of prompt engineering is getting past these rather lazy descriptions and being specific. Most summaries are a kind of extraction, and asking for a "summary" is deferring to the LLM to figure out what information is interesting entirely based on its sort-of-common-sense assessment. You can always do better than that! Plans are similar, it's an opportunity to give the LLM a template for planning, to specify goals, things to watch out for, etc. You can usually do better than "think step by step".
- thallavajhula 3y agoPrompt engineering in a way feels like the advanced search querying on Google. Chat bots work fine for most of the basic questions. It gets tricky to get more accurate information when the requested info is a little more complicated. Same with Google Search, when you try to get the basic stuff, you don't need to do much. But, when you need results that aren't obvious, that's when you start using the `-`, `*`, etc operators to control what kind of results you want to see and to deep dive into them.
- openaikillers 3y ago[dead]
- nextaccountic 3y agothere was a time that google did something that doesn't scale: you could pay google to employ some employee to manually search something in the web for you i never used this service but people said it was magic, because those employees really really knew how to get the most out of a web search i suppose it was retired because the plan was always to make the search itself better however in the last years or decade i have noticed a regression, in that i can't find things i was sure i would in some ways this mirrors the chatgpt regression in quality due to constrained compute resources or something like that
- rubslopes 3y ago> there was a time that google did something that doesn't scale: you could pay google to employ some employee to manually search something in the web for you That's so interesting, I had no idea! What year are we talking about?
- SushiHippie 3y agohttps://en.wikipedia.org/wiki/Google_Answers https://en.wikipedia.org/wiki/Google_Answers Edit: woah you can even read the questions and answers 17 years later! http://answers.google.com/answers/ http://answers.google.com/answers/
- intrepidsoldier 3y ago"large language models (sometimes referred to as GPT models)" lol
- verdverm 3y agowhy "lol" Generative Pre-trained Transformer (GPT) is the primary or core implementation for LLMs
- ugh123 3y agoI like the use of the "Worse/Better" table. It gives people clear examples of amount of specificity (one of my least favorite words to say) needed for common knowledge tasks where the actual need and it's presentation have not been described yet. There should be a lot more of these. Novice users should be able to adapt those to their own needs easier and craft better prompts rather than completely "thought generating" their own.
- revskill 3y agoPrompt Engineering to me is the best way to learn how to ask a good question. It's such a great mentor.
- verdverm 3y agoLike humans, it takes time to learn how to communicate with an LLM. Also like humans, each LLM needs something a little different
- dsco 3y agoId love to hear if someone here has experience in pushing GPT to actually not omit code and write out the entire thing you’ve requested. I often need to push it and prompt it to “_WRITE OUT ALL OF THE CODE_” like a demanding Karen.
- arthurcolle 3y agoyep this is a bug that they say they are trying to fix. For now use gpt-4-32k on Azure
- 3xnl 3y agoWhy is this called prompt engineering not prompt something else? I feel like the word engineering is being abused
- iwontberude 3y agoI think if you were more into sales engineering it would all just make sense.
- minimaxir 3y agoThe metaphor is more based on social engineering.
- verdverm 3y agoHaving to iterate on the prompt to get good, consistent results on a variety of inputs definitely feels like an engineering task
- 3xnl 3y agoNot really. Though I know these days people use Engineer for all sorts of things
- notatoad 3y agothis could be wrong and i've missed some of the timeline, but from what i've seen "prompt engineering" started out as a sarcastic joke on twitter about how software engineering roles were going to be reduced to prompt engineering. and then people took the term and started using it seriously.
- danjc 3y agoLLM's are teaching us to communicate clearly.
- consumer451 3y agoYes. Or more generally, one of the more immediate benefits of the quest for artificial intelligence is the understanding of human intelligence that we gain along the way. (insert friends made along the way meme, but truly profound)
- verdverm 3y agoOne of my big hopes is that LLMs help us to become better communicators with each other. - prompt engineering for clarity (and focus?) - results (good examples of quality replies) - assistant (help me say this better) where better could be a lot of things, depending on the context, here I'm mainly meaning in how we treat each other through communication (politeness, contentiousness, how we behave on social media), like giving nudges to be nicer
- sorokod 3y agoIn the same way water "teaches us" not to drown.
- openaigenocide 3y ago[dead]
- openaihate 3y ago[dead]
- openainazis 3y ago[dead]
- openaizionists 3y ago[dead]
- darkerside 3y agoWrite clear instructions Provide reference text Use external tools Split complex tasks into simpler subtasks Give the model time to "think" Test changes systematically As these best practices solidify, why are they not being built into the UI or product itself for these tools? Seems trivially straightforward besides the last one. For the first one, add an optional persona field and allow query construction in pieces before sending over the wire. Permanently pre-prompt the model to always ask itself how long to "think" before answering, and ask itself intermediate questions if it's nontrivial.
- russellbeattie 3y agoPrompt engineering won't be around forever. I think of LLMs as being like early computing systems, where you had to work around the limitations imposed by the CPUs, memory and other hardware. Back then they had to implement workarounds like binary math tricks, etc. It was a pain, but that's what you had to do. Eventually the hardware got better, the amount of low level effort was reduced and programming languages got easier to use. No one needs to write assembly any more. LLMs are on a similar path. Right now, we have to work with the limitations imposed by the current state of LLM functionality. As the technology matures, we won't need to worry about wording input as much.
- openaikills 3y ago[dead]
- pknerd 3y agoTBH, Often I asked chatGPT to suggest a prompt for a domain I am not good at. For instance, I asked chatGPT to give me a prompt that can help me to give a market(stock) overview at day end with all key insights. chatGPT came up with a good prompt which I then used on Google Bard(I do not have gpt4 subscription hence no access to the latest data). Bard came up with a good 5-7 lines paragraph of text having all key insights of NASDAQ. I later asked Bard to return the key points in JSON format and it obeyed me.
- openaikillers 3y ago[dead]
- WhitneyLand 3y agoI ordered a cheeseburger in Spanish and the server looked at me funny when I said: ”hamburguesa con queso sin pepinillos…” I’m always interested in how to improve, especially since ChatGpt and Google Translate both suggested that translation so I asked why. She said I’m not sure, it just doesn’t sound right. I came back the next day after practicing with this prompt: ”When translating into Spanish, tailor it for Mexican-Americans living in Dallas, Texas. Leave certain words as English as necessary to produce the most idiomatic, culturally relevant, and understandable result.” Ordered this time with the phrase ”Cheeseburger sin pepinillos." She said yes, that’s better.
- smith7018 3y agoShe couldn't tell you "we say 'cheeseburger' instead of 'hamburguesa con queso?'" This is a good anecdote on why LLMs are great for translation but a strange example lol
- WhitneyLand 3y agoShe was not bi-lingual either and I didn’t want to intrude too much on someone’s work time with my random tech nerd curiosities
- openaikillers 3y ago[dead]
- jqr- 3y agoI'm almost sure "pepinillos" is what's throwing her off. That's their proper name, but they're nor common in Mexican cuisine and therefore not part of the lingo. If ordering at an American place in Mexico, you would call them "pepinos". Back to your example, both of these sound natural to me: "Una hamburgesa con queso sin pickles" "Un/a cheeseburger sin pickles" Here the gendered noun can go either way since it's not clear if cheeseburger is a male or female noun. You: "Una hamburgesa sin pickles" Them: "Con o sin queso?" You: "Sin". Source: I'm almost your target audience.
- 3y ago
- CrypticShift 3y agoI've been hesitant lately to dedicate a lot of time to learning how to perfect prompts. It appears every new version, not to mention different LLMs (Google's here [1]), responds differently. With the rapid advancement we're seeing, in two year or five, we might not even need such complex prompting as systems get smarter. [1] : https://ai.google.dev/docs/prompt_intro https://ai.google.dev/docs/prompt_intro
- sanderjd 3y agoI know I'm the dummy here because people are doing useful stuff with these techniques, but I don't think I'll ever shake the feeling that this can't possibly be the way forward, that it can only possibly be a short-lived local maximum. Doesn't this all seem ... kind of silly?
- barfingclouds 3y agoThat’s a good point. I remember in elementary school, our librarians were teaching us how to find information online using some really technical search engine. Maybe called like jstor or something else? They hated google and said you can’t properly filter with it the way you could with theirs. But years later I obviously never heard of the previous one again and used google regularly
- maxbrydak 3y agoJSTOR and other similar are really cool when looking for academic papers tbh
- ianbicking 3y agoI am frequently frustrated when I see people doing "studies" of LLMs where they don't put in the prompt engineering work. I came upon an example [1] recently where someone compared GPT-4 to Gemini Pro and Claude 2. The results hardly matter to me because they didn't put in the prompt work: they didn't give the model space to think (demanding it return only true/false), and they didn't give it higher-level instructions about the categorization, only vague instructions and a couple examples. I think this often happens in order to be "objective" about the evaluation. I can see how it feels like cheating to coax the model to produce the answer you want. But... it's not! An off-handed prompt isn't more objective than a crafted prompt. You just haven't investigated its biases and flaws. This lazy assessment is common everywhere, of course. It's one of the reasons bias gets into testing so easily: you setup a test and you assume that it is objective because you give everyone the same test with the same rubric. But if the subjects don't understand your terminology, or the proctor doesn't understand the subjects' terminology, it's easy to mistake misunderstanding for something else (intelligence, opinion, whatever you are testing for). Systems based on communication need feedback loops, and that's just to get to the _starting point_. Prompt engineering is one of those feedback loops. [1] https://www.vellum.ai/blog/best-at-text-classification-gemini-pro-gpt-4-or-claude2 https://www.vellum.ai/blog/best-at-text-classification-gemin...
- IKantRead 3y ago> I can see how it feels like cheating to coax the model to produce the answer you want. But... it's not! If it's for a single example, it is absolutely cheating. As an AI engineer this is a particular point of frustration where people complain because a large system can't return the result they want, when they were able to get the answer they wanted on their own with a lot of prompt hacking. Each prompt is basically a point in latent space, and if you're "tweaking" the prompt what you're really doing is just re-rolling the dice until you land in a neighborhood closer the answer you want. You're not better at prompting, you just got lucky and are confusing that for insight. Now if you're specific prompting trick works across a suite of evaluations, then you are probably on to something. But what people are doing in most cases is equivalent to performing some ritual before pulling the handle on a slot machine and then, when they finally win, claiming that they finally stumbled upon the correct ritual.
- say_it_as_it_is 3y agoPeople have been working on very elaborate "super prompts" to drive custom GPT development on OpenAI. Non-programmers have spent hours copying and pasting super prompts together with the hopes that they will cash in on OpenAI GPT Store when it opens, without interest in open sourcing these prompts. Unfortunately, they left back doors wide open and have been.. Pwned.. by ChatGPT-savvy users who have gladly open-sourced the super prompts: https://github.com/linexjlin/GPTs.git https://github.com/linexjlin/GPTs.git
- jorblumesea 3y agoIsn't the need for prompt engineering in some ways admitting some kind of user interface failure? Feels like there should be an abstraction layer on top of this.
- k__ 3y agoPrompt design/engineering is more complex than most people think. If you want content that doesn't look like the crap that currently floods the web, you need to understand how to talk to a model AND have enough domain knowledge to articulate what you actually want.
- lsy 3y agoWhat makes LLMs somewhat unique as a software product is that there is little-to-no separation between input and instruction. In most cases, the user's input can also be considered part of the "prompt". This leads to the well-known "prompt injection" "vulnerability" which is really just a byproduct of the fundamental inability (and indeed undesirability) for the model to distinguish instruction from input (undesirable because the value and flexibility comes from allowing the user, rather than a programmer, to specify an action). On top of it though, it introduces a sort of disciplinary sloppiness around whether the program can be reasoned about. It's assumed that prompt `P` works for whatever input `I`, but the concatenated `P+I` is really the full input to the program that produces a desired output. But the only way to be confident about the program's behavior is to exhaust the input space, as no `P+A` tells you anything about how `P+B` will behave. This makes it difficult to leverage an LLM in any process where the desired result is 1. unknown and 2. matters. If the result is unknown it's not clear how to determine mistakes or correct them if they're made. And if the correctness of the result matters it's courting disaster to connect it to a process which is not able to be reasoned about. I think that's why LLMs are primarily being used to assist ideation (which is cool!) or "spammy" use cases like third-tier customer service or listicle generation, and haven't yet broken into any use case where they need to be reliable for complex tasks.
- yellow_postit 3y agoThe reason we mostly default to separating concerns of data and code is because of all the headaches it avoids. One item on my research wishlist is to bring more constraints like this provable to language models.
- maxglute 3y agoIssue is language fails human to human interactions all the time. I would be so bold as to say most people are poor communicators, a machine is not going to read ones mind and intentions any better than fellow humans. It's why military has BLUF communication style to convey information in concise, simple, predictable way. If anything prompt engineering should exist if only to improve humans ability to communicate with other humans.
- lsy 3y agoOn the "humans need prompt engineering too": well, kind of. I hope that we are not all working on the precise verbiage to get the plumber to fix our toilet correctly. There are also actions we expect others to perform without any communication whatsoever. Human actions take place in a social web that includes incentives and accountability as well as expectations on the part of interlocutors around what level of detail is required for communication. Sometimes this very clearly breaks down. But the fact that interpersonal communications sometimes require elaboration or precision doesn't negate the differences between an LLM's outputs and the actions of people who are engaged in active coping with the world around them.
- Der_Einzige 3y agoReal prompt engineering (emphasis on engineering) exists, you just don’t know about it because it only exists for open source models: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865ca4bb328eb58faf https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...
- secret-noun 3y agoSometimes, I fantasize about what I would say if a genie in a bottle presented me with three wishes. "I want a billion dollars." But what if: - The money is in a worthless currency - The money is stolen and must be forfeited - The money will be given, but on my deathbed. - The money is in 1 cent coins It's difficult to state what I'm looking for because there are side effects and interpretations that I can't even imagine, and even if I could, language is imperfect.