4 ms·

So now prompting GPT3 with some context counts as "training ChatGPT"? What?

• WA 4y ago
  I also found this description utterly lacking. However, OpenAI actually has a thing called fine-tuning the model. You can use it to train it with your own data. Maybe the author did exactly that? But I couldn’t find how to do this via the Playground. https://beta.openai.com/docs/guides/fine-tuning https://beta.openai.com/docs/guides/fine-tuning
  • dotancohen 4y ago
    She didn't actually fine tune or train, and other Twitter users called her out on that. Her settings in the screenshots basically boil down to this (also culled from a screenshot): openai.Completion.create( model = "text-davinci-003", prompt = "The following is a conversation with Present and Young.\nYoung has written the following journal entries:\n[Diary entries here]\n[Your question here]", temperature = 1, max_tokens = 600, top_p = 1, frequency_penalty = 0, presence_penalty = 0.6, stop = ["Present:", "Young:"], )
• lelandfe 4y ago
  I told ChatGPT to act as a novel-writing Unix command with a flag to provide text to emulate without affecting the ongoing story's narrative. And it totally worked! I asked it to summarize the story so far afterwards and it excluded the few paragraphs of "training" text I gave it, but then used the sentence structure and words I included in the text in future novel parts. "Training" has a precise meaning in this field, but perhaps it has different connotations with something like ChatGPT which can process the entire conversation.
  • zwaps 4y ago
    In context learning is probably best framed as priming the model towards already learned relations. The model was trained previously. Machine Learning Street Talk and Yannik Kilchers coverage of this years Neurips conference give some good broad level overviews of some papers in this regard