3 ms·
Yes but fine tuning for RL is not expected to be hard. You're essentially limited by how much human feedback is available, so it's very different from training
by zozbot234 4y ago
Yes but fine tuning for RL is not expected to be hard. You're essentially limited by how much human feedback is available, so it's very different from training the foundational model on random bulk data.
- simonw 4y agoAt this point I fully expect that someone will release a usable RLHF-fine-tuned language model that can run on consumer hardware, based on the methodology used for LLaMA (and other similar papers e.g. https://github.com/FMInference/FlexGen https://github.com/FMInference/FlexGen ), at some point in the next 6-24 months.
- monkeydust 4y agoWhat are the functional steps that need to happen to get to that point ?
- ntonozzi 4y agoHow are they different from models that already exist like FLAN-T5?
- stevenhuang 4y agoOf course, but even still the level of scale, clean data, and human supervision needed may be significant. It's reported OpenAI used an army of humans to generate question answer prompts and rate the model output. They kept the details closely guarded and only hinted at how they did RLHF and transitioned the architecture to self supervised learning.
- pps 4y agoI'm stupid in this area and have only a very broad understanding of "AI". But isn't this already happening in this project? https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant