3 ms·
> I thought "as an AI language model" was OpenAI's design, and they deliberately instilled this specific phrase into GPT via RLHF. It's not something LLMs say "
by taneq 3y ago
> I thought "as an AI language model" was OpenAI's design, and they deliberately instilled this specific phrase into GPT via RLHF. It's not something LLMs say "naturally," and The Pile by itself shouldn't make an LLM preprend certain answers with "as an AI language model." The paper does not say they did any RLHF themselves, or even any prompt engineering about making the LLM respond like an AI chatbot. LLMs don't "know" they're AI unless they've been prompted to that effect.
This is correct. OpenAI's GPT models say this because they've been trained to do so via RLHF. Newer language models typically say this because their training set is partially generated by OpenAI's models. It's quite fascinating to ask LLaMA finetunes about themselves and see which ones respond with "As an AI language model..." or even outright say they were created by OpenAI.
- nicklecompte 3y agoYes, see causalmodel's comment. I missed that the researchers trained the AI on The Pile but did some post-conditioning on MOSS, which contains GPT-3.5 output. To be clear: it's fine for commercial / personal / etc LLMs to train on GPT. But it's not so fine when research LLMs do it: those benchmarks don't mean very much if you're training the AI on GPT's answers to the benchmarks. Test contamination is bad enough in LLM research; using datasets like MOSS in academic papers is just reckless.