6 ms·

Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.

by gojomo 3y ago

Retric 3y ago
What seems plausible depends on your level of suspect matter expertise. Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?
- gojomo 3y ago
  That's not my experience with the leading models, & I'm seeing others observe even when LLMs are wrong, they often supply interesting potential avenues to consider (especially in the coding-assistant/debugging domains). Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding... • ways to detect likely hallucinations from patterns in internal activations • extra conditioning to reduce hallucinations • success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods • ways to check output against various ground-truths to detect & correct errors To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.
  - Retric 3y ago
    Reducing hallucinations is useful but doesn’t imply that the remaining hallucinations are going to be valuable. In terms of coding the costs of an LLM using some API function that doesn’t exist isn’t that high because we have tools to detect such nonsense and can look at the API in question. In science people don’t get to read realities underlying API’s.
- pixl97 3y ago
  > Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Depends at the cost and rate you can test them. When simulations are fast and cheap you have far more latitude in filtering out 'ridiculously incorrect'.
  - Retric 3y ago
    If you can somehow build an actual physical test that costs 1 cent to both synthesize and validate a chemical you’re still looking at ~1 trillion dollars per candidate. Simulations might be that cheap someday, but that’s still checking plausibility not running an actual experiment.

Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.

by gojomo 3y ago

Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.

What seems plausible depends on your level of suspect matter expertise. Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?

gojomo 3y ago
That's not my experience with the leading models, & I'm seeing others observe even when LLMs are wrong, they often supply interesting potential avenues to consider (especially in the coding-assistant/debugging domains). Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding... • ways to detect likely hallucinations from patterns in internal activations • extra conditioning to reduce hallucinations • success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods • ways to check output against various ground-truths to detect & correct errors To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.
- Retric 3y ago
  Reducing hallucinations is useful but doesn’t imply that the remaining hallucinations are going to be valuable. In terms of coding the costs of an LLM using some API function that doesn’t exist isn’t that high because we have tools to detect such nonsense and can look at the API in question. In science people don’t get to read realities underlying API’s.
pixl97 3y ago
> Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Depends at the cost and rate you can test them. When simulations are fast and cheap you have far more latitude in filtering out 'ridiculously incorrect'.
- Retric 3y ago
  If you can somehow build an actual physical test that costs 1 cent to both synthesize and validate a chemical you’re still looking at ~1 trillion dollars per candidate. Simulations might be that cheap someday, but that’s still checking plausibility not running an actual experiment.