5 ms·

ChatGPT recreated with GPT-3.5?

40 points by karfly 4y ago

karfly 4y ago
We all love chat.openai.com, but... It's TERRIBLY laggy, has daily limits, and is only accessible through an archaic web interface. This repo is ChatGPT re-created with GPT-3.5 LLM as Telegram Bot. And it works great. In addition it supports special modes like "Code Assistant" and "Movie Expert".
- adamgordonbell 4y ago
  This just looks like some prompts to make it chattier though. Good idea but not exactly chatGPT is it? Does it get the context of the conversation so far resent to it for example?
  - karfly 4y ago
    1. You’re right, it’s not ChatGPT exactly. ChatGPT = GPT-3.5 + finetuning on conversational data + alignment (RLHF). But I found out, that GPT-3.5 + proper prompting gives comparable results and works really good. 2. The context of the conversation is preserved by adding whole (or some parts of) dialog to the prompt.
    - kelsolaar 4y ago
      Given it is not ChatGPT and it is especially missing RLHF, I would suggest amending the repo name and description. Maybe “ChatGPT-like” would be more appropriate here.
moglito 4y ago
Thanks for creating this, but ChatGPT is free and this replacement is not, right? (one needs to pay for API credit from OpenAI)
- deleted 4y ago
  [deleted]
- schattschneider 4y ago
  You're right that the usage is not free. The readme also lists this feature: > Track $ balance spent on OpenAI API
- karfly 4y ago
  It's true – you need to pay for OpenAI API. With my typical usage frequency of ChatGPT it's going to cost me about 3$/month.
d_watt 4y ago
Looking at the code, it looks like you're maintaining context by re-serializing all previous responses as "ChatGPT" and "User." Out of curiosity, do you know that this is how chatgpt is done, or your take on it? I assumed from this tweet, there was more of a "special sauce" for maintaining chat context: https://twitter.com/OpenAI/status/1615160228366147585?s=20&t=kQ0kODuywQBqTQCy8duE-g https://twitter.com/OpenAI/status/1615160228366147585?s=20&t...
vacuumcl 4y ago
The text-davinci-003 model is not GPT3.5, and this does not really replicate ChatGPT since that one is fine tuned based on human feedback. Edit: I stand corrected, sorry for Dunning - Krugering.
- d_watt 4y ago
  It is GPT 3.5. It's not ChatGPT, though. From https://beta.openai.com/docs/model-index-for-researchers https://beta.openai.com/docs/model-index-for-researchers > GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series: code-davinci-002 is a base model, so good for pure code-completion tasks text-davinci-002 is an InstructGPT model based on code-davinci-002 text-davinci-003 is an improvement on text-davinci-002
- karfly 4y ago
  You're totally true. But it turned out, that proper prompting (like packing dialog context into the prompt) worked great. I spent many hours with orig ChatGPT and with this recreated version. The main difference I have found is that the recreated ChatGPT is more inclined to ask questions on questions (maybe can be fixed with more prompt engineering). I didn't find any major differences in the quality or usefulness of the answers.
tehsauce 4y ago
Prepending “Hi I’m ChatGPT” to the prompt of another model is hardly a recreation :/
bg24 4y ago
ChatGPT like assistants are going to be commodity that you can use on your devices, applications, home. Amazing times ahead in the next 2 years.
bilater 4y ago
Nice! I did something similar for WhatsApp: https://www.gpthotline.com/ https://www.gpthotline.com/
xg15 4y ago
Ok, so OpenAI says that ChatGPT is GPT-3.5, but with extensive fine-tuning applied, based on a complex multi-stage feedback process with human evaluators. But at the same time, you can apparently just take the "raw" GPT-3.5, give it a prompt to behave like an assistant and get comparable results? So was the whole RLHF process just cargo cult?
- karfly 4y ago
  IMO all RLHF stuff is mainly about aligning model not to reply with offensive and inappropriate answers, but NOT about making model's answers better.
  - xg15 4y ago
    Ah, that makes sense. Thanks!
sauwan 4y ago
Amazing, nice work! How hard would this be to port to MS Teams rather than Telegram?

40 points by karfly 4y ago

We all love chat.openai.com, but... It's TERRIBLY laggy, has daily limits, and is only accessible through an archaic web interface. This repo is ChatGPT re-created with GPT-3.5 LLM as Telegram Bot. And it works great. In addition it supports special modes like "Code Assistant" and "Movie Expert".

adamgordonbell 4y ago
This just looks like some prompts to make it chattier though. Good idea but not exactly chatGPT is it? Does it get the context of the conversation so far resent to it for example?
- karfly 4y ago
  1. You’re right, it’s not ChatGPT exactly. ChatGPT = GPT-3.5 + finetuning on conversational data + alignment (RLHF). But I found out, that GPT-3.5 + proper prompting gives comparable results and works really good. 2. The context of the conversation is preserved by adding whole (or some parts of) dialog to the prompt.
  - kelsolaar 4y ago
    Given it is not ChatGPT and it is especially missing RLHF, I would suggest amending the repo name and description. Maybe “ChatGPT-like” would be more appropriate here.

moglito 4y ago

Thanks for creating this, but ChatGPT is free and this replacement is not, right? (one needs to pay for API credit from OpenAI)

deleted 4y ago
[deleted]
schattschneider 4y ago
You're right that the usage is not free. The readme also lists this feature: > Track $ balance spent on OpenAI API
karfly 4y ago
It's true – you need to pay for OpenAI API. With my typical usage frequency of ChatGPT it's going to cost me about 3$/month.

d_watt 4y ago

Looking at the code, it looks like you're maintaining context by re-serializing all previous responses as "ChatGPT" and "User." Out of curiosity, do you know that this is how chatgpt is done, or your take on it? I assumed from this tweet, there was more of a "special sauce" for maintaining chat context: https://twitter.com/OpenAI/status/1615160228366147585?s=20&t=kQ0kODuywQBqTQCy8duE-g https://twitter.com/OpenAI/status/1615160228366147585?s=20&t...

vacuumcl 4y ago

The text-davinci-003 model is not GPT3.5, and this does not really replicate ChatGPT since that one is fine tuned based on human feedback. Edit: I stand corrected, sorry for Dunning - Krugering.

d_watt 4y ago
It is GPT 3.5. It's not ChatGPT, though. From https://beta.openai.com/docs/model-index-for-researchers https://beta.openai.com/docs/model-index-for-researchers > GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series: code-davinci-002 is a base model, so good for pure code-completion tasks text-davinci-002 is an InstructGPT model based on code-davinci-002 text-davinci-003 is an improvement on text-davinci-002
karfly 4y ago
You're totally true. But it turned out, that proper prompting (like packing dialog context into the prompt) worked great. I spent many hours with orig ChatGPT and with this recreated version. The main difference I have found is that the recreated ChatGPT is more inclined to ask questions on questions (maybe can be fixed with more prompt engineering). I didn't find any major differences in the quality or usefulness of the answers.

tehsauce 4y ago

Prepending “Hi I’m ChatGPT” to the prompt of another model is hardly a recreation :/

bg24 4y ago

ChatGPT like assistants are going to be commodity that you can use on your devices, applications, home. Amazing times ahead in the next 2 years.

bilater 4y ago

Nice! I did something similar for WhatsApp: https://www.gpthotline.com/ https://www.gpthotline.com/

xg15 4y ago

Ok, so OpenAI says that ChatGPT is GPT-3.5, but with extensive fine-tuning applied, based on a complex multi-stage feedback process with human evaluators. But at the same time, you can apparently just take the "raw" GPT-3.5, give it a prompt to behave like an assistant and get comparable results? So was the whole RLHF process just cargo cult?

karfly 4y ago
IMO all RLHF stuff is mainly about aligning model not to reply with offensive and inappropriate answers, but NOT about making model's answers better.
- xg15 4y ago
  Ah, that makes sense. Thanks!

sauwan 4y ago

Amazing, nice work! How hard would this be to port to MS Teams rather than Telegram?