14 ms·
> you can't train a model using human voters to vote during iterations Why not? Just comment multiple times on a post in different ways. Score outputs with mor
by Zambyte 3y ago
> you can't train a model using human voters to vote during iterations
Why not? Just comment multiple times on a post in different ways. Score outputs with more of a desired response higher than outputs with less desired responses. Scale that up site-wide on multiple sites, and it seems like you have a pretty powerful way to get human feedback...
- Jensson 3y agoThese models trains on billions of examples, having bots posting billions of posts on different social media sites every time you train a new model probably isn't a viable strategy. You would get banned real quick since most of those will be really low quality in the early stages.
- vineyardmike 3y agoSo I agree with your overall sentiment, BUT we already have good LLMs, so I think we’ve crossed the bridge from “bad low quality responses in the beginning” and I also suspect there’s more Social Media bots than we’d like to admit.
- Zambyte 3y agoI think we're simply talking about different things. Obviously you wouldn't want to start with a model of pure noise when interacting with real humans like that. I am describing using RLHF to fine tune existing models. That is a way to gamify training.