2 ms·
But it has human voters, you can't train a model using human voters to vote during iterations, and AI aren't good enough to replace human voters. Not sure if A
by Jensson 3y ago
But it has human voters, you can't train a model using human voters to vote during iterations, and AI aren't good enough to replace human voters.
Not sure if AI voters even can result in a model smarter than the voting model.
- Zambyte 3y ago> you can't train a model using human voters to vote during iterations Why not? Just comment multiple times on a post in different ways. Score outputs with more of a desired response higher than outputs with less desired responses. Scale that up site-wide on multiple sites, and it seems like you have a pretty powerful way to get human feedback...
- Jensson 3y agoThese models trains on billions of examples, having bots posting billions of posts on different social media sites every time you train a new model probably isn't a viable strategy. You would get banned real quick since most of those will be really low quality in the early stages.
- vineyardmike 3y agoSo I agree with your overall sentiment, BUT we already have good LLMs, so I think we’ve crossed the bridge from “bad low quality responses in the beginning” and I also suspect there’s more Social Media bots than we’d like to admit.
- Zambyte 3y agoI think we're simply talking about different things. Obviously you wouldn't want to start with a model of pure noise when interacting with real humans like that. I am describing using RLHF to fine tune existing models. That is a way to gamify training.