4 ms·

Thanks, I've saw this leaked chat prompt and that gave me some ideas. So under the hoods, ChatGPT is just a model like Llama where they prepend every user inpu

Thanks, I've saw this leaked chat prompt and that gave me some ideas. So under the hoods, ChatGPT is just a model like Llama where they prepend every user input with a context that makes it behave like a chatbot?

thot_experiment 4y ago
(I don't work at OpenAI so take it with a grain of salt) Yes and No they are similar. It is basically just a fancy autocomplete like llama, but I believe it's specifically been trained on chat content, or at least finetuned on such, and it probably uses a more chat focused labeling scheme on the training data as well to help it perform well on that specific task and be conversational.
- tarruda 4y ago
  To me it is really mind blowing that these properties (coding, emulate chatbot) emerge from just from feeding these neural nets with text data. Thanks for the info.