4 ms·
These examples are for more simple prompt engineering demos. With the ChatGPT system prompt, you can give the model a large and complex set of rules to account
by minimaxir 3y ago
These examples are for more simple prompt engineering demos. With the ChatGPT system prompt, you can give the model a large and complex set of rules to account for and recent models of ChatGPT do a good job of accommodating them. Some of my best system prompts are >20 lines of text, and all of them are necessary to get the model to behave.
The examples are also too polite and conversational: you can give more strict commands and in my experience it works better.
There's also function calling/structured data support which is technically prompt engineering and requires similar skills, but is substantially more powerful than using the system prompt alone (I'm working on a blog post on it now and it unfortunately it is going to be a long post to address all of its power). Here's a fun demo example which compares system prompts and structured data results: https://github.com/minimaxir/simpleaichat/blob/main/examples/notebooks/schema_ttrpg.ipynb https://github.com/minimaxir/simpleaichat/blob/main/examples...
- fragmede 3y agoMind sharing a system prompt of yours? 20+ lines sounds useful
- minimaxir 3y agoUnfortunately those were for specific work use-cases so I can't share them but the tl;dr is that every time the model does something undesired, even minor I add an explicit rule in the system prompt to handle it, or some few-shot examples if the model is really bad at handling it. That list can balloon quickly.
- verdverm 3y agoHere's a big one I needed to get ChatGPT to do something more sophisticated with a JSON object response (predates functions and all that) https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/prompts/dm.cue https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro... It no longer worked after a model update some time ago, haven't tried recently. I found codellama to be much better for this and require fewer instructions, an anecdotal validation for smaller, focussed models
- throwup238 3y ago> The examples are also too polite and conversational: you can give more strict commands and in my experience it works better. The way that works best for me is "It extracts ALL the entities from the text, it does this whenever its told, or else it gets the hose again"
- deleted 3y ago[deleted]
- electrondood 3y agoI found that far less prompt is required for something like ChatGPT. I've stopped writing well-formed requests/questions and now I just state things like: "sed to replace line in a text file?" "Django endpoint but CSRF token error. why?" (follow up) "now this: `$ERROR`" etc. It still just gives me what I need to know.
- mrich 3y agoAgreed, you can use ChatGPT similarly to Google now. Except you do not have to parse and filter the results, plus there are no ads.
- firejake308 3y agoCan't tell if you're kidding or not
- behnamoh 3y agoWhich phrasing do you thinks work better? 1. "You are blah blah blah. You <always> respond to the user's questions using the information provided to you..." 2. "You are blah blah blah. You <should> respond to the user's questions using the information provided to you..." Also, when dealing with Completion models, which do you think is better? 1. The following is a conversation between ASSISTANT and USER. ASSISTANT is helpful and tries to answer USER's queries respectfully. 2. The following is a conversation between YOU and USER. YOU are helpful and try to answer USER's queries respectfully. Even more still, what about these ones? 1. You're a customer of company <X>. What do you think about the following policy change which was shown on the company's website? 2. A customer visits company <X>'s website. Pretend you're this customer. What do you think the customer thinks about the following policy change which was shown on the company's website?
- minimaxir 3y ago> Which phrasing do you thinks work better? I like as a rule-of-thumb "You are blah blah blah. Respond to the user's text [insert style rule here]". Then following it up with an additional rules and commands such as "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." Yes, threats work. Yes, all-caps works. > Also, when dealing with Completion models, which do you think is better? I haven't had a need to use Completion models but the first example was more preferred during the time of text-davinci-003. > Even more still, what about these ones? I always separate rules to the system prompt and questions/user input to the user prompt.
- deleted 3y ago[deleted]
- jazzyjackson 3y agoIs the LLM predisposed to understand this prompt as instructions from a higher authority? ("You must do this, You will always do this.") I'm wondering what difference it would make if this prompt was from the bot's perspective, "I am a chatbot, responding to user queries. I will always respond in less than 100 characters. I am a good person, I'm just trying to be helpful."
- pknerd 3y agocan you share some resources which helped you to write such nested prompts?
- minimaxir 3y agoReally just a decade of technical writing and learning how to be extremely precise and unambiguous with language (half of that decade being in software QA, which helps even moreso)
- abrichr 3y agoThank you for the great library and examples! Can you please comment on how simpleaichat compares to https://github.com/outlines-dev/outlines https://github.com/outlines-dev/outlines ?
- minimaxir 3y agosimpleaichat is designed to be simple and is essentially an API wrapper for common generative use cases. outlines does a few more things with a bit more ambiguity/complexity. (e.g. it may use grammars which is a secondary useful aspect of function calling, but does add more complexity) Neither are better or worse, it depends on your business needs.
- abrichr 3y agoThank you for your perspective! (We are looking into both for https://github.com/OpenAdaptAI/OpenAdapt https://github.com/OpenAdaptAI/OpenAdapt)
- solardev 3y agoI love this! This time last year, nobody believed this was possible. Now we're teaching AI to write better essays, prompting them like schoolchildren. <3
- neom 3y agoInteresting you should say that, I was playing around with prompting last week and did one around a legal question. The first time I asked very concisely without much detail, and the answer it gave was poor. Then I re-wrote the question explaining who they are, why they are answering the question, etc etc. The answer seemed better so I showed it to a lawyer friend and they laughed and said "You re-wrote the question into a very standard bar exam prep style".
- solardev 3y agoI just love this idea of "emergent humanity". Makes me wonder how much of our own personality and speech is also just trained/culturized over our lifetime. Some of us also have bigger context windows than others :)
- Zelphyr 3y ago> necessary to get the model to behave Don't I know it. Despite my telling GPT-4 to ONLY respond as valid, well-formed JSON it keeps coming back with things like, "I'm not able to process external files but if I could, this is what the JSON would look like: []"
- enobrev 3y agoWith a recent project, I was _moderately_ successful by providing a jsonschema to follow for the response. I still had to sanitize the json a bit, but the fixes were minor and the resulting data otherwise fit the schema well.
- hhh 3y agowhy don’t you use the new JSON mode?
- minimaxir 3y agotl;dr the JSON mode is functionally useless and is made completely redundant by function calling / structured data if you really really need JSON output.
- kristjansson 3y agoIt'll do that. Just look for the largest substring that's valid JSON in the response.