4 ms·

This feels like an aggressive push to force codex users to adapt to GPT3.5/4. I'd imagine they will be integrating some of codex's features into GPT4 soon.

nunodonato 4y ago
It's ridiculous. Now we all have to implement "chat" even if we all want is code completion
- TylerE 4y ago
  [flagged]
  - drdeca 4y ago
    Consumers of the API
    - TylerE 4y ago
      Why would consumers need to implement it? Isn’t it just like, send prompt, get response? Why is it necessary to implement chat if you don’t care about chat? Is it used for scheduling or something?
      - sashank_1509 4y ago
        Its all to do with how the API is designed. Now the prompts need to be sent in the form of chat, and we need to worry about text being output from the model, and not just code. Basically, users need to comb through their codebase, change their API calls and see if nothing breaks. While gpt turbo is quite good, it was quite a headache to replace davinci calls with GPT Turbo, especially when I closely integrated it with a chain of prompts. I can only imagine it being more of a headache for codex users
        gkbrk 4y ago
        You already had to deal with text being output from the Codex models. I regularly ran into text from Python mailing lists when using the Codex API. Sometimes it writes code, loses coherency and types out pages and pages of Python mailing list comments and signatures. Any human text I get from the Chat completion API with the task of completing code is at least relevant to the code being written.
        nico 4y ago
        I tried a prompt like this to get it to spit code out: “Pretend you are a SQL expert. You convert requests written in English to Postgres SQL. Your output is always SQL only. If you need to respond with something other than SQL code, add it to comments within the SQL code you generate and prepend it with ‘Note: ‘. Your output will automatically be executed on a read-only Postgres database called ‘my_db’ and the result of that will be given to you. If you are still working on getting an answer, add ‘Working’ at the end of your message, or ‘Done’ if you for the result you needed to answer the request.” Request: How many API calls were made by clients last month?” You can keep that at the top of every request, while running the SQL code iteratively until it says Done.
        capableweb 4y ago
        While I agree that some changes have to be done in your code, preventing the model from outputting text is relatively easy, you just have to have a strong system message to set the model to give you the output you want.
        throwaway4aday 4y ago
        Until someone accidentally jailbreaks it by putting weird comments in their code. Somewhat /s but within the realm of possibility.
- brookst 4y ago
  No. I have an app that uses both. You do not have to implement chat. All you have to do is 1) call the chat completion API, 2) structure your prompt in JSON as system+agent+user, and 3) get the completion from a different JSON node. You don’t have to round trip or maintain context or anything.
  - QuadmasterXLII 4y ago
    I'm seeing that the chat api wants to always wrap its response in "```python ```" when it generates code. Is there a prompt trick to stop this, or should I just be stripping it manually

nunodonato 4y ago

It's ridiculous. Now we all have to implement "chat" even if we all want is code completion

TylerE 4y ago
[flagged]
- drdeca 4y ago
  Consumers of the API
  - TylerE 4y ago
    Why would consumers need to implement it? Isn’t it just like, send prompt, get response? Why is it necessary to implement chat if you don’t care about chat? Is it used for scheduling or something?
    - sashank_1509 4y ago
      Its all to do with how the API is designed. Now the prompts need to be sent in the form of chat, and we need to worry about text being output from the model, and not just code. Basically, users need to comb through their codebase, change their API calls and see if nothing breaks. While gpt turbo is quite good, it was quite a headache to replace davinci calls with GPT Turbo, especially when I closely integrated it with a chain of prompts. I can only imagine it being more of a headache for codex users
      - gkbrk 4y ago
        You already had to deal with text being output from the Codex models. I regularly ran into text from Python mailing lists when using the Codex API. Sometimes it writes code, loses coherency and types out pages and pages of Python mailing list comments and signatures. Any human text I get from the Chat completion API with the task of completing code is at least relevant to the code being written.
        nico 4y ago
        I tried a prompt like this to get it to spit code out: “Pretend you are a SQL expert. You convert requests written in English to Postgres SQL. Your output is always SQL only. If you need to respond with something other than SQL code, add it to comments within the SQL code you generate and prepend it with ‘Note: ‘. Your output will automatically be executed on a read-only Postgres database called ‘my_db’ and the result of that will be given to you. If you are still working on getting an answer, add ‘Working’ at the end of your message, or ‘Done’ if you for the result you needed to answer the request.” Request: How many API calls were made by clients last month?” You can keep that at the top of every request, while running the SQL code iteratively until it says Done.
      - capableweb 4y ago
        While I agree that some changes have to be done in your code, preventing the model from outputting text is relatively easy, you just have to have a strong system message to set the model to give you the output you want.
        throwaway4aday 4y ago
        Until someone accidentally jailbreaks it by putting weird comments in their code. Somewhat /s but within the realm of possibility.
brookst 4y ago
No. I have an app that uses both. You do not have to implement chat. All you have to do is 1) call the chat completion API, 2) structure your prompt in JSON as system+agent+user, and 3) get the completion from a different JSON node. You don’t have to round trip or maintain context or anything.
- QuadmasterXLII 4y ago
  I'm seeing that the chat api wants to always wrap its response in "```python ```" when it generates code. Is there a prompt trick to stop this, or should I just be stripping it manually