3 ms·
It is my understanding that this is how “alignment” works. That is, openAI paid people to chat with their LLM to fine tune it and then other LLMs use chatgpt t
by wodenokoto 3y ago
It is my understanding that this is how “alignment” works.
That is, openAI paid people to chat with their LLM to fine tune it and then other LLMs use chatgpt to generate training data to align their models.
- visarga 3y agoThere are three ways 1. make your own RLHF dataset - like OpenAI and Open Assistant 2. exfiltrate data from a bigger/better LLM - Vicuna & family 3. use your pre-trained LLM to generate RLAIF data, no leeching - ConstitutionalAI, based on a set of rules instead of labelling examples
- cubefox 3y agoI wonder whether these approaches fit into the above categories: https://arxiv.org/abs/2305.13735 https://arxiv.org/abs/2305.13735 https://arxiv.org/abs/2305.11206 https://arxiv.org/abs/2305.11206