3 ms·

Designing trustworthy AI sounds comparably difficult to finding trustworthy people, and finding people who are widely agreed to be trustworthy is incredibly dif

by amluto 4y ago

Designing trustworthy AI sounds comparably difficult to finding trustworthy people, and finding people who are widely agreed to be trustworthy is incredibly difficult. If you want AI to say the right things or have the right opinions, somehow the process needs to find the right things to say or the right opinions to have. This is somewhere between difficult and impossible. If you want AI to give correct facts, somehow you need to determine correct facts. Humans find this extremely difficult — one would need better-than-human AIs! Maybe a middle ground is possible: an AI that acknowledges the existence of multiple perspectives. Sadly a lot of people are forgetting about this lately.

CuriouslyC 4y ago
I don't know why people act like hallucinations and toxicity are these unsolvable problems. Both of them can be solved using data set curation and annotation, and model support for "toxicity" and "accuracy" annotations on token sequences ("style" would be a good one too). Then the model would have access to conditional probabilities given a toxicity/accuracy score, and generations could be guided to low toxicity/high accuracy generations. The cost of annotating data sets for these LLMs will be significant, but necessary.
- amluto 4y ago
  re:hallucinations. There are things that are clearly hallucinations, e.g. made up sources that straight-up don’t exist. But what if you asked an AI early in the pandemic how COVID spreads? re:toxicity. I believe there are things an AI could say that are widely agreed to be toxic. I also think there are things an AI (or a human!) could say that some people think are fine and others think are toxic.
  - CuriouslyC 4y ago
    This could be handled by treating the annotations for toxicity/accuracy as distributions rather than scalar values (which is probably how we'd get them in the first place, since we'd have redundant evaluators for token sequences) and training the AI to predict both the mean and variance of the distribution. Areas where the AI had limited knowledge and things that people disagree on might have the same mean, but they'd get high variance scores, which would help people differentiate.

by amluto 4y ago

CuriouslyC 4y ago

I don't know why people act like hallucinations and toxicity are these unsolvable problems. Both of them can be solved using data set curation and annotation, and model support for "toxicity" and "accuracy" annotations on token sequences ("style" would be a good one too). Then the model would have access to conditional probabilities given a toxicity/accuracy score, and generations could be guided to low toxicity/high accuracy generations. The cost of annotating data sets for these LLMs will be significant, but necessary.

amluto 4y ago
re:hallucinations. There are things that are clearly hallucinations, e.g. made up sources that straight-up don’t exist. But what if you asked an AI early in the pandemic how COVID spreads? re:toxicity. I believe there are things an AI could say that are widely agreed to be toxic. I also think there are things an AI (or a human!) could say that some people think are fine and others think are toxic.
- CuriouslyC 4y ago
  This could be handled by treating the annotations for toxicity/accuracy as distributions rather than scalar values (which is probably how we'd get them in the first place, since we'd have redundant evaluators for token sequences) and training the AI to predict both the mean and variance of the distribution. Areas where the AI had limited knowledge and things that people disagree on might have the same mean, but they'd get high variance scores, which would help people differentiate.