4 ms·
I don't know why people act like hallucinations and toxicity are these unsolvable problems. Both of them can be solved using data set curation and annotation,
by CuriouslyC 4y ago
I don't know why people act like hallucinations and toxicity are these unsolvable problems. Both of them can be solved using data set curation and annotation, and model support for "toxicity" and "accuracy" annotations on token sequences ("style" would be a good one too). Then the model would have access to conditional probabilities given a toxicity/accuracy score, and generations could be guided to low toxicity/high accuracy generations.
The cost of annotating data sets for these LLMs will be significant, but necessary.
- amluto 4y agore:hallucinations. There are things that are clearly hallucinations, e.g. made up sources that straight-up don’t exist. But what if you asked an AI early in the pandemic how COVID spreads? re:toxicity. I believe there are things an AI could say that are widely agreed to be toxic. I also think there are things an AI (or a human!) could say that some people think are fine and others think are toxic.
- CuriouslyC 4y agoThis could be handled by treating the annotations for toxicity/accuracy as distributions rather than scalar values (which is probably how we'd get them in the first place, since we'd have redundant evaluators for token sequences) and training the AI to predict both the mean and variance of the distribution. Areas where the AI had limited knowledge and things that people disagree on might have the same mean, but they'd get high variance scores, which would help people differentiate.