3 ms·

in fact ALL the examples use the system prompt. one gets the impression that the completion api is softly being discontinued. this has been alarming for capabil

by swyx 3y ago

in fact ALL the examples use the system prompt. one gets the impression that the completion api is softly being discontinued. this has been alarming for capabilities researchers who just want next token generation without the constraints of roles

minimaxir 3y ago
The only reason to use text-davinci-003 nowadays is when the ChatGPT API is overloaded and breaks, especially given the lower cost of the ChatGPT API.
- dontupvoteme 3y ago
  There's potentially some interesting research potential there given that you can peak more behind the scenes to infer how different inputs result in different outputs without the black box of RLHF at play. For instance, if I want to generate some python code that uses a certain library and use the "write the code header + function def + docstring" approach with complete/insert/edit functionality how does the output change if 0. I vary the file header 1. I vary the presence of other functions in the input 2. I larp as the creator of the lib, a famous programmer, John Smith, Hans Mueller, Ivan Ivanovich Ivanovsky 3. (In combination with 2) - I prompt it in another language 4. I choose GPL vs BSD vs Apache vs other licenses 5. I specify exact python and module versions (right now it hallucinates functions which I don't have a lot, which is quite annoying) It was trained on code and I don't like being abstracted away from code itself if I can avoid it/ I don't know how long davinci will be around as it strikes me as a risk to openAI - it may be being datamined as we speak for use in a legal case against them in the future, e.g. to show more direct evidence of having trained on data which they shouldn't have. Practically speaking I will sometimes run a request in parallel between davinci, chat API and the web interface and compare the results.
  - deleted 3y ago
    [deleted]
- swyx 3y ago
  that's untrue; this is what i am trying to communicate. all the post davinci 003 apis are heavily RLHFed and instruction tuned, preventing further capabilities research outside the bounds of chat. in other words, there is a smol contingent of people who believe Chat Is Not The Final Form of generative text and they are slowly getting shut out from researching and putting in production different applications if they do not work at a large model lab (Anthropic also has similar chat expectations on their API) see also https://twitter.com/deepfates/status/1638212305887567873?lang=en https://twitter.com/deepfates/status/1638212305887567873?lan...
- nomel 3y ago
  They have very different use cases. One is for completion, and one is for conversation.
  - minimaxir 3y ago
    With system prompt tricks, as noted in this article, you can force ChatGPT to behave for completion with often better results than text-davinci-003 in my experience.
    - nomel 3y ago
      Do you have an example? Most (all?) of the examples provided are with GPT4 with system prompts, not ChatGPT.

by swyx 3y ago

minimaxir 3y ago

The only reason to use text-davinci-003 nowadays is when the ChatGPT API is overloaded and breaks, especially given the lower cost of the ChatGPT API.

dontupvoteme 3y ago
There's potentially some interesting research potential there given that you can peak more behind the scenes to infer how different inputs result in different outputs without the black box of RLHF at play. For instance, if I want to generate some python code that uses a certain library and use the "write the code header + function def + docstring" approach with complete/insert/edit functionality how does the output change if 0. I vary the file header 1. I vary the presence of other functions in the input 2. I larp as the creator of the lib, a famous programmer, John Smith, Hans Mueller, Ivan Ivanovich Ivanovsky 3. (In combination with 2) - I prompt it in another language 4. I choose GPL vs BSD vs Apache vs other licenses 5. I specify exact python and module versions (right now it hallucinates functions which I don't have a lot, which is quite annoying) It was trained on code and I don't like being abstracted away from code itself if I can avoid it/ I don't know how long davinci will be around as it strikes me as a risk to openAI - it may be being datamined as we speak for use in a legal case against them in the future, e.g. to show more direct evidence of having trained on data which they shouldn't have. Practically speaking I will sometimes run a request in parallel between davinci, chat API and the web interface and compare the results.
- deleted 3y ago
  [deleted]
swyx 3y ago
that's untrue; this is what i am trying to communicate. all the post davinci 003 apis are heavily RLHFed and instruction tuned, preventing further capabilities research outside the bounds of chat. in other words, there is a smol contingent of people who believe Chat Is Not The Final Form of generative text and they are slowly getting shut out from researching and putting in production different applications if they do not work at a large model lab (Anthropic also has similar chat expectations on their API) see also https://twitter.com/deepfates/status/1638212305887567873?lang=en https://twitter.com/deepfates/status/1638212305887567873?lan...
nomel 3y ago
They have very different use cases. One is for completion, and one is for conversation.
- minimaxir 3y ago
  With system prompt tricks, as noted in this article, you can force ChatGPT to behave for completion with often better results than text-davinci-003 in my experience.
  - nomel 3y ago
    Do you have an example? Most (all?) of the examples provided are with GPT4 with system prompts, not ChatGPT.