4 ms·

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which m

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack gpt-3.5-turbo to your use case even if it is hugely wasteful from a tokens point of view.

m3kw9 3y ago
What’s the diff with 3.5turbo with instruct?
- gamegoblin 3y ago
  One is tuned for chat. It has that annoying ChatGPT personality. Instruct is a little "lower level" but more powerful. It doesn't have the personality. It just obeys. But it is less structured, there are no messages from user to AI, it is just a single input prompt and a single output completion.
- thewataccount 3y ago
  the existing 3.5turbo is what you would call a "chat" model. The difference between them is that the chat models are much more... chatty - they're trained to act like they're in a conversation with you. The chat models generally say things "Sure, I can do that for you!", and "No problem! Here is". The conversation style is generally more inconsistent in it's style. It can be difficult to make it only return the result you want, and occasionally it'll keep talking anyway. It'll also talk in first person more, and a few things like that. So if you're using it as an API for things like summarization, extracting the subject of a sentence, code editing, etc, then the chat model can be super annoying to work with.
ClassicOrgin 3y ago
I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.
- gamegoblin 3y ago
  I'm assuming they will price it the same as normal gpt-3.5-turbo. I won't use it if it's more than 2x the price of turbo, because I can usually get turbo to do what I want, it just takes more tokens sometimes. Have you tried getting your formatted JSON out via the new Functions API? I does cure a lot of the deficiencies in 3.5-turbo.
  - mrinterweb 3y ago
    From what I can find, pricing of GPT-4 is roughly 25x that of 3.5 turbo. https://openai.com/pricing https://openai.com/pricing https://platform.openai.com/docs/deprecations/ https://platform.openai.com/docs/deprecations/
    - gamegoblin 3y ago
      In this thread we’re talking about gpt-3.5-turbo-instruct, not GPT4
      - mrinterweb 3y ago
        Sorry about that. Got my thread context confused.
- merpnderp 3y ago
  You need to use the new OpenAI Functions API. It is absolutely bonkers at returning formatted results. I can get it to return a perfectly formatted query-graph a few levels deep.
  - jimmySixDOF 3y ago
    There is also Code Interpreter now in plugin beta so should influence it's ability to output proper formats without hallucinations.
- senko 3y ago
  With the latest 3.5-turbo, you can try forcing it to call your function with a well-defined schema for arguments. If the structure is not overly complex, this should work.
Zpalmtree 3y ago
I'm hoping gpt-3.5-turbo-instruct isn't super neutered like chatgpt. davinci-003 can be a lot more fun and answer on a wide range of topics where ChatGPT will refuse to answer.
- rmorey 3y ago
  such as?
byt143 3y ago
What's the difference between chat and instruction tuning?
- tudorw 3y ago
  no expert, but from my messing around I gather the chat models are tuned for conversation, for example, if you just say 'Hi', it will spit out some 'witty' reply and invite you to respond, it's creative with it's responses. On the other hand, if you say 'Hi' to an instruct model, it might say something like, I need more information to complete the task. Instruct models are looking for something like 'Write me a twitter bot to make millions'... in this case, if you ask the same thing again, you are somewhat more like to get the same, or similar result, this does not appear so true with a chat model, perhaps a real expert could chime in :)
- msp26 3y ago
  System/assistant/user prompting

m3kw9 3y ago

What’s the diff with 3.5turbo with instruct?

gamegoblin 3y ago
One is tuned for chat. It has that annoying ChatGPT personality. Instruct is a little "lower level" but more powerful. It doesn't have the personality. It just obeys. But it is less structured, there are no messages from user to AI, it is just a single input prompt and a single output completion.
thewataccount 3y ago
the existing 3.5turbo is what you would call a "chat" model. The difference between them is that the chat models are much more... chatty - they're trained to act like they're in a conversation with you. The chat models generally say things "Sure, I can do that for you!", and "No problem! Here is". The conversation style is generally more inconsistent in it's style. It can be difficult to make it only return the result you want, and occasionally it'll keep talking anyway. It'll also talk in first person more, and a few things like that. So if you're using it as an API for things like summarization, extracting the subject of a sentence, code editing, etc, then the chat model can be super annoying to work with.

ClassicOrgin 3y ago

I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.

gamegoblin 3y ago
I'm assuming they will price it the same as normal gpt-3.5-turbo. I won't use it if it's more than 2x the price of turbo, because I can usually get turbo to do what I want, it just takes more tokens sometimes. Have you tried getting your formatted JSON out via the new Functions API? I does cure a lot of the deficiencies in 3.5-turbo.
- mrinterweb 3y ago
  From what I can find, pricing of GPT-4 is roughly 25x that of 3.5 turbo. https://openai.com/pricing https://openai.com/pricing https://platform.openai.com/docs/deprecations/ https://platform.openai.com/docs/deprecations/
  - gamegoblin 3y ago
    In this thread we’re talking about gpt-3.5-turbo-instruct, not GPT4
    - mrinterweb 3y ago
      Sorry about that. Got my thread context confused.
merpnderp 3y ago
You need to use the new OpenAI Functions API. It is absolutely bonkers at returning formatted results. I can get it to return a perfectly formatted query-graph a few levels deep.
- jimmySixDOF 3y ago
  There is also Code Interpreter now in plugin beta so should influence it's ability to output proper formats without hallucinations.
senko 3y ago
With the latest 3.5-turbo, you can try forcing it to call your function with a well-defined schema for arguments. If the structure is not overly complex, this should work.

Zpalmtree 3y ago

I'm hoping gpt-3.5-turbo-instruct isn't super neutered like chatgpt. davinci-003 can be a lot more fun and answer on a wide range of topics where ChatGPT will refuse to answer.

rmorey 3y ago
such as?

byt143 3y ago

What's the difference between chat and instruction tuning?

tudorw 3y ago
no expert, but from my messing around I gather the chat models are tuned for conversation, for example, if you just say 'Hi', it will spit out some 'witty' reply and invite you to respond, it's creative with it's responses. On the other hand, if you say 'Hi' to an instruct model, it might say something like, I need more information to complete the task. Instruct models are looking for something like 'Write me a twitter bot to make millions'... in this case, if you ask the same thing again, you are somewhat more like to get the same, or similar result, this does not appear so true with a chat model, perhaps a real expert could chime in :)
msp26 3y ago
System/assistant/user prompting