5 ms·
I don't actually do LLM research or engineering myself - I hate Python :) So I am ignorant about a lot of the nitty-gritty in actually training and deploying an
by nicklecompte 3y ago
I don't actually do LLM research or engineering myself - I hate Python :) So I am ignorant about a lot of the nitty-gritty in actually training and deploying an LLM. There's a lot in this paper I haven't read closely. But this struck me as suspicious:
"Building on Kocon et al.’s insights [37], we explored the zero-shot learning capabilities of the scaled-up AM-SNet model with 1.5 billion parameters, trained on The Pile dataset [35]. Zero-shot learning tasks, which test the model’s generalization to unseen tasks without specific training, are crucial for
language models’ practical application."
...but a good chunk of the text generation examples begin with "As an AI language model..."
I may be mistaken: I thought "as an AI language model" was OpenAI's design, and they deliberately instilled this specific phrase into GPT via RLHF. It's not something LLMs say "naturally," and The Pile by itself shouldn't make an LLM preprend certain answers with "as an AI language model." The paper does not say they did any RLHF themselves, or even any prompt engineering about making the LLM respond like an AI chatbot. LLMs don't "know" they're AI unless they've been prompted to that effect.
[Edit: I missed this in Section 3, "Subsequent fine-tuning was conducted on the MOSS finetune dataset, which is tailored for dialogue systems." See causalmodels's helpful comment below. What I wrote below is wrong yet not totally off-base. The research wasn't "clumsily faked," but the post-training conditioning was certainly corrupted with GPT-3.5 output, which IMO makes the natural language generation analysis quite flawed - and in general should make us wonder how "zero-shot" these examples really are.]
I am suspicious this part of the research was clumsily faked. Happy to be proven wrong: maybe there's something in The Pile that explains it, or part of the modern LLM toolchain does some pre-training to make things more ChatGPT-like. Maybe the researchers actually trained it on GPT-4 and didn't mention that in the paper.
- webmaven 3y ago> I don't actually do LLM research or engineering myself - I hate Python :) What programming language(s) do you like?
- nicklecompte 3y agoIdris, Coq, F#/OCaml, Scheme. C# is also nice, and has the significant advantage that people will pay me to write it.... I was being a bit facetious, Python is perfectly fine for writing programs and software. The problem is the AI/ML dialect of Python. I don't like working in domains that involve big 3rd-party ecosystems and libraries[1]. I prefer doing something bespoke for a smaller organization, even if that means solving a smaller problem. [1] Interacting with tediously-but-necessarily detailed libraries and programming frameworks is an area where ChatGPT is genuinely useful - codegen which boils down to "translate this mathematical English into PyTorch." So at some point I might go back and actually program some of this stuff.
- pushfoo 3y agoDo you have recommendations for getting into ML-family languages and functional approaches, or even just F#? Although Python's ecosystem and plentiful training data for LLMs are useful, trying Elm made me want to find something better at types.
- taneq 3y ago> I thought "as an AI language model" was OpenAI's design, and they deliberately instilled this specific phrase into GPT via RLHF. It's not something LLMs say "naturally," and The Pile by itself shouldn't make an LLM preprend certain answers with "as an AI language model." The paper does not say they did any RLHF themselves, or even any prompt engineering about making the LLM respond like an AI chatbot. LLMs don't "know" they're AI unless they've been prompted to that effect. This is correct. OpenAI's GPT models say this because they've been trained to do so via RLHF. Newer language models typically say this because their training set is partially generated by OpenAI's models. It's quite fascinating to ask LLaMA finetunes about themselves and see which ones respond with "As an AI language model..." or even outright say they were created by OpenAI.
- nicklecompte 3y agoYes, see causalmodel's comment. I missed that the researchers trained the AI on The Pile but did some post-conditioning on MOSS, which contains GPT-3.5 output. To be clear: it's fine for commercial / personal / etc LLMs to train on GPT. But it's not so fine when research LLMs do it: those benchmarks don't mean very much if you're training the AI on GPT's answers to the benchmarks. Test contamination is bad enough in LLM research; using datasets like MOSS in academic papers is just reckless.
- causalmodels 3y agoThis got me curious, so I looked into it. The only post ChatGPT dataset was MOSS-3-SFT, which was used for dialogue fine tuning. MOSS-3 was generated using ChatGPT-3.5-turbo [1]. They explicitly call out their fine tuning data in footnote 13 so I wouldn't say it is faked per say, but it isn't a great look. [1] https://huggingface.co/fnlp/moss-moon-003-base#data https://huggingface.co/fnlp/moss-moon-003-base#data
- nicklecompte 3y agoGot it, thanks for looking into it more closely. I didn't actually finish reading the paper, and missed that this was in section 3: "Subsequent fine-tuning was conducted on the MOSS finetune dataset [36], which is tailored for dialogue systems." Agreed this almost certainly isn't faked, but the validity of using this dataset at all seems questionable.