3 ms·

> you can't expect the student model to know better than the teacher model does. Why not? People outgrow their teachers all the time, even in fields where the

> you can't expect the student model to know better than the teacher model does. Why not? People outgrow their teachers all the time, even in fields where there is no new external data, like maths. Often improved understanding comes just from reflecting on an issue from new perspectives, which synthetic data can provide. Perhaps a teacher/student model helps AI develop those new insights, just as it does for us.

michaelt 3y ago
If (when taken in aggregate) the training data says Gerald Ford was the 38th President of the United States, and the model disagrees, is that not a deficiency of the model?
- cameronh90 3y ago
  Probably, and sometimes teachers are wrong. Also, I don't know who Gerald Ford is but I'll take your word that he's the 38th president of the United States as nobody is here to tell me otherwise ;). It's likely though that some otherwise intelligent and sentient people may disagree on who is the current president of the United States! I don't believe anybody is suggesting that we exclusively use synthetic data, but rather that synthetic data can augment other types of training. The other thing to consider is that less sophisticated models can be prone to hallucinating nonsense, but the hallucinations are usually inconsistent, whereas truthful responses tend towards consistency in various directions: between each response, internal consistency, and consistency with reality. It's conceivable that a more sophisticated model would be able to learn a sense of certainty based on the consistency of its training data, much as we do. If you consider your education, I'm guessing you probably had lots of people tell you incompatible nonsense over the years. In my case, I've had teachers give me a ton of inconsistent explanations about how electrons "know" which path to take in a circuit - probably one of my biggest questions since a child. Only one explanation turned out to be internally consistent and demonstrable with experiments I've seen on YouTube. The result is that I now have, I think, a pretty decent understanding of how electric current forms a path within a circuit, or at least one that can be used to make valid predictions, despite being told a vast amount of inconsistent and wrong information over my life.

michaelt 3y ago

If (when taken in aggregate) the training data says Gerald Ford was the 38th President of the United States, and the model disagrees, is that not a deficiency of the model?

cameronh90 3y ago
Probably, and sometimes teachers are wrong. Also, I don't know who Gerald Ford is but I'll take your word that he's the 38th president of the United States as nobody is here to tell me otherwise ;). It's likely though that some otherwise intelligent and sentient people may disagree on who is the current president of the United States! I don't believe anybody is suggesting that we exclusively use synthetic data, but rather that synthetic data can augment other types of training. The other thing to consider is that less sophisticated models can be prone to hallucinating nonsense, but the hallucinations are usually inconsistent, whereas truthful responses tend towards consistency in various directions: between each response, internal consistency, and consistency with reality. It's conceivable that a more sophisticated model would be able to learn a sense of certainty based on the consistency of its training data, much as we do. If you consider your education, I'm guessing you probably had lots of people tell you incompatible nonsense over the years. In my case, I've had teachers give me a ton of inconsistent explanations about how electrons "know" which path to take in a circuit - probably one of my biggest questions since a child. Only one explanation turned out to be internally consistent and demonstrable with experiments I've seen on YouTube. The result is that I now have, I think, a pretty decent understanding of how electric current forms a path within a circuit, or at least one that can be used to make valid predictions, despite being told a vast amount of inconsistent and wrong information over my life.