2 ms·

What is your hypothesis for the layout/diagram of models feeding the gpt-4 output for chatgpt/gpt-4 vs api/gpt-4?

• gwern 3y ago
  Components which are likely there and which could be affecting quality: the invisible-to-the-user 'system prompt' (probably with few-shot examples), retrieval from history (not necessarily exclusively your own, as a way to augment few-shot), cascaded models with a very cheap 'turbo' model to try to answer first, possibly regular finetunes of the main chat model (note that OP only specifies the API model hasn't changed, but the live chat models seem to change frequently), and a post-generation filtering model trying to reject offensive outputs. We know MS Bing Sydney is doing at least 3 of those (prompt, cascade with Megatron, and finetuned rejection classifier for post-output filtering) on top of its GPT-4-finetune, so it's not a stretch to figure that OA is doing similar things.