3 ms·
The question is, where does this "human-ness" lie? In the initial neural network, the training data, or the supervised reinforcement? In theory, a significant
by basch 4y ago
The question is, where does this "human-ness" lie? In the initial neural network, the training data, or the supervised reinforcement?
In theory, a significantly smaller neural network that outputs at nearly as good a quality, should be able to chew through training data, and its re-enforcement process, much faster and cheaper, right? A more generalized, lower parameter model, is almost always preferred, as long as it works?
If the human feeling is all boltonable to the neural net later, there is no reason to discount this component as lacking potential to exceed current models.
- visarga 4y agoI think it's the feedback from RLHF that is mostly responsible, but it only works if the base model is large. Never seen a small model doing good conversation. They can do ok for classification and open book question answering, but generating long form coherent text is hard.