5 ms·
Fundamentally unsuited because of how they train it using "fill in the blank." Training a large model to guess when it doesn't know the answer results in ficti
by skybrian 4y ago
Fundamentally unsuited because of how they train it using "fill in the blank."
Training a large model to guess when it doesn't know the answer results in fiction. They need to do something else to get nonfiction.
By contrast, for Go the model was trained not to make illegal moves, because checking for that as part of the training is easy and cheap.
- halpmeh 4y agoWe have models that accurate classify things, e.g. whether or not an email is spam. There isn’t a fundamental limitation into building something like a truth classifier into a generative model so that it optimized for outputting “true” statements. The hardest part is probably identifying what is truth and what is falsehood. That’s a fundamental problem with humanity, not neural networks.
- skybrian 4y agoWell, we could quibble about what "fundamental" means but my point is that the way they train large language models doesn't work for this. Something different needs to happen.
- nextaccountic 4y ago> There isn’t a fundamental limitation into building something like a truth classifier into a generative model so that it optimized for outputting “true” statements. Problem is, they didn't do that
- imtringued 4y agoTruth has nothing to do with humanity unless you mean the specific way humans construct belief systems. Anyway I already told you the answer. The AI will need a series of trainable belief systems to verify whether statements are internally consistent. The strange part about this is that the AI would need to have a way to obtain validation and each prompt would have to derive a new belief system which you must use in the next prompt. In other words, the model must be able to learn continuously. That is something that these single shot AI models are not capable of.