3 ms·

These are good examples of how to leverage the system prompt, which is vastly underdiscussed as that is only available via the API or the Playground and not the

These are good examples of how to leverage the system prompt, which is vastly underdiscussed as that is only available via the API or the Playground and not the megapopular ChatGPT webapp. Even in LangChain it requires some hacking to get working and may not be stable across generations. I am releasing a minimal Python chat AI package interface this week which very heavily encourages use the system prompt for efficient generations that are also stable and can hand a variety of user inputs. The results have been very effective!

swyx 3y ago
in fact ALL the examples use the system prompt. one gets the impression that the completion api is softly being discontinued. this has been alarming for capabilities researchers who just want next token generation without the constraints of roles
- minimaxir 3y ago
  The only reason to use text-davinci-003 nowadays is when the ChatGPT API is overloaded and breaks, especially given the lower cost of the ChatGPT API.
  - dontupvoteme 3y ago
    There's potentially some interesting research potential there given that you can peak more behind the scenes to infer how different inputs result in different outputs without the black box of RLHF at play. For instance, if I want to generate some python code that uses a certain library and use the "write the code header + function def + docstring" approach with complete/insert/edit functionality how does the output change if 0. I vary the file header 1. I vary the presence of other functions in the input 2. I larp as the creator of the lib, a famous programmer, John Smith, Hans Mueller, Ivan Ivanovich Ivanovsky 3. (In combination with 2) - I prompt it in another language 4. I choose GPL vs BSD vs Apache vs other licenses 5. I specify exact python and module versions (right now it hallucinates functions which I don't have a lot, which is quite annoying) It was trained on code and I don't like being abstracted away from code itself if I can avoid it/ I don't know how long davinci will be around as it strikes me as a risk to openAI - it may be being datamined as we speak for use in a legal case against them in the future, e.g. to show more direct evidence of having trained on data which they shouldn't have. Practically speaking I will sometimes run a request in parallel between davinci, chat API and the web interface and compare the results.
    - deleted 3y ago
      [deleted]
  - swyx 3y ago
    that's untrue; this is what i am trying to communicate. all the post davinci 003 apis are heavily RLHFed and instruction tuned, preventing further capabilities research outside the bounds of chat. in other words, there is a smol contingent of people who believe Chat Is Not The Final Form of generative text and they are slowly getting shut out from researching and putting in production different applications if they do not work at a large model lab (Anthropic also has similar chat expectations on their API) see also https://twitter.com/deepfates/status/1638212305887567873?lang=en https://twitter.com/deepfates/status/1638212305887567873?lan...
  - nomel 3y ago
    They have very different use cases. One is for completion, and one is for conversation.
    - minimaxir 3y ago
      With system prompt tricks, as noted in this article, you can force ChatGPT to behave for completion with often better results than text-davinci-003 in my experience.
      - nomel 3y ago
        Do you have an example? Most (all?) of the examples provided are with GPT4 with system prompts, not ChatGPT.

swyx 3y ago

in fact ALL the examples use the system prompt. one gets the impression that the completion api is softly being discontinued. this has been alarming for capabilities researchers who just want next token generation without the constraints of roles

minimaxir 3y ago
The only reason to use text-davinci-003 nowadays is when the ChatGPT API is overloaded and breaks, especially given the lower cost of the ChatGPT API.
- dontupvoteme 3y ago
  There's potentially some interesting research potential there given that you can peak more behind the scenes to infer how different inputs result in different outputs without the black box of RLHF at play. For instance, if I want to generate some python code that uses a certain library and use the "write the code header + function def + docstring" approach with complete/insert/edit functionality how does the output change if 0. I vary the file header 1. I vary the presence of other functions in the input 2. I larp as the creator of the lib, a famous programmer, John Smith, Hans Mueller, Ivan Ivanovich Ivanovsky 3. (In combination with 2) - I prompt it in another language 4. I choose GPL vs BSD vs Apache vs other licenses 5. I specify exact python and module versions (right now it hallucinates functions which I don't have a lot, which is quite annoying) It was trained on code and I don't like being abstracted away from code itself if I can avoid it/ I don't know how long davinci will be around as it strikes me as a risk to openAI - it may be being datamined as we speak for use in a legal case against them in the future, e.g. to show more direct evidence of having trained on data which they shouldn't have. Practically speaking I will sometimes run a request in parallel between davinci, chat API and the web interface and compare the results.
  - deleted 3y ago
    [deleted]
- swyx 3y ago
  that's untrue; this is what i am trying to communicate. all the post davinci 003 apis are heavily RLHFed and instruction tuned, preventing further capabilities research outside the bounds of chat. in other words, there is a smol contingent of people who believe Chat Is Not The Final Form of generative text and they are slowly getting shut out from researching and putting in production different applications if they do not work at a large model lab (Anthropic also has similar chat expectations on their API) see also https://twitter.com/deepfates/status/1638212305887567873?lang=en https://twitter.com/deepfates/status/1638212305887567873?lan...
- nomel 3y ago
  They have very different use cases. One is for completion, and one is for conversation.
  - minimaxir 3y ago
    With system prompt tricks, as noted in this article, you can force ChatGPT to behave for completion with often better results than text-davinci-003 in my experience.
    - nomel 3y ago
      Do you have an example? Most (all?) of the examples provided are with GPT4 with system prompts, not ChatGPT.