4 ms·

Contra the headling, we should "celebrate being wrong" – when it's wrong in fast, interesting, & voluminous ways that can then be filtered or corrected. That's

by gojomo 3y ago

Contra the headling, we should "celebrate being wrong" – when it's wrong in fast, interesting, & voluminous ways that can then be filtered or corrected. That's how lots of science, innovation, & learning work: generate many superficially-plausible candidates via a fast-and-loose process, then refine with a more rigorous evaluation. That AIs, in the form of LLMs, are now doing this so well was unexpected, and progress in checking 'hallucinations' is proceeding very fast. (Fortunately, the article is less dismissive than the headline, recognizing these model's potential & mainly urging an understanding of the limitations.)

Retric 3y ago
Science progresses because people are testing plausible ideas not random ideas. We don’t do human trials on random drugs, we do human trials on promising ones. Animal trials are more open but even then candidates are carefully considered as being viable. Things are even more open the earlier you are in drug discovery, but we aren’t testing molecules using Dubnium or most elements on the periodic table. Similarly Ecology isn’t studying what happens when you introduce each type of salt water fish into lakes because the general assumption is they would just die thus saving you from preforming millions of experiments. That basic check for plausibility is extraordinarily valuable across all sciences. There’s no sacred cows here. You do need to validate plausibility just like anything else, but you don’t need to do an exhaustive search across every possibility.
- gojomo 3y ago
  Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.
  - Retric 3y ago
    What seems plausible depends on your level of suspect matter expertise. Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?
    - gojomo 3y ago
      That's not my experience with the leading models, & I'm seeing others observe even when LLMs are wrong, they often supply interesting potential avenues to consider (especially in the coding-assistant/debugging domains). Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding... • ways to detect likely hallucinations from patterns in internal activations • extra conditioning to reduce hallucinations • success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods • ways to check output against various ground-truths to detect & correct errors To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.
      - Retric 3y ago
        Reducing hallucinations is useful but doesn’t imply that the remaining hallucinations are going to be valuable. In terms of coding the costs of an LLM using some API function that doesn’t exist isn’t that high because we have tools to detect such nonsense and can look at the API in question. In science people don’t get to read realities underlying API’s.
    - pixl97 3y ago
      > Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Depends at the cost and rate you can test them. When simulations are fast and cheap you have far more latitude in filtering out 'ridiculously incorrect'.
      - Retric 3y ago
        If you can somehow build an actual physical test that costs 1 cent to both synthesize and validate a chemical you’re still looking at ~1 trillion dollars per candidate. Simulations might be that cheap someday, but that’s still checking plausibility not running an actual experiment.
- simplify 3y ago
  No, science is indeed testing random ideas. Limiting what is tested to an arbitrary plausibility score is something funders and sponsors do, not scientists. What you're talking about is ethics, which is important too.
  - Retric 3y ago
    Random ideas include things like: The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres. I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.
    - simplify 3y ago
      I assure you, there are many scientists that would love to test that, given the resources to do so. Of course no one would fund that, meaning it's the funders deciding on what they think is plausible.
      - Retric 3y ago
        I know plenty of scientists that might approach it as a joke. But they are far too busy to regularly spend significant time on such a things.
    - criddell 3y ago
      An AI would probably flag that idea as unlikely and not worth pursuing. I don’t think we are that far off from having AIs that can read and incorporate the concepts from every paper ever published and start generating random ideas that fit into an internal framework it develops. I think that within the lifetime of some readers here, an AI will spit out a hypothesis (like a method for producing a room temperature super conductor) that turns out to be true and we will have no idea where it came from. The AI’s knowledge of physics, chemistry, etc… will exceed ours and it trying to explain to us how it works would be like you explaining a singular value decomposition to your dog.

by gojomo 3y ago

Retric 3y ago

Science progresses because people are testing plausible ideas not random ideas. We don’t do human trials on random drugs, we do human trials on promising ones. Animal trials are more open but even then candidates are carefully considered as being viable. Things are even more open the earlier you are in drug discovery, but we aren’t testing molecules using Dubnium or most elements on the periodic table. Similarly Ecology isn’t studying what happens when you introduce each type of salt water fish into lakes because the general assumption is they would just die thus saving you from preforming millions of experiments. That basic check for plausibility is extraordinarily valuable across all sciences. There’s no sacred cows here. You do need to validate plausibility just like anything else, but you don’t need to do an exhaustive search across every possibility.

gojomo 3y ago
Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.
- Retric 3y ago
  What seems plausible depends on your level of suspect matter expertise. Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?
  - gojomo 3y ago
    That's not my experience with the leading models, & I'm seeing others observe even when LLMs are wrong, they often supply interesting potential avenues to consider (especially in the coding-assistant/debugging domains). Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding... • ways to detect likely hallucinations from patterns in internal activations • extra conditioning to reduce hallucinations • success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods • ways to check output against various ground-truths to detect & correct errors To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.
    - Retric 3y ago
      Reducing hallucinations is useful but doesn’t imply that the remaining hallucinations are going to be valuable. In terms of coding the costs of an LLM using some API function that doesn’t exist isn’t that high because we have tools to detect such nonsense and can look at the API in question. In science people don’t get to read realities underlying API’s.
  - pixl97 3y ago
    > Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Depends at the cost and rate you can test them. When simulations are fast and cheap you have far more latitude in filtering out 'ridiculously incorrect'.
    - Retric 3y ago
      If you can somehow build an actual physical test that costs 1 cent to both synthesize and validate a chemical you’re still looking at ~1 trillion dollars per candidate. Simulations might be that cheap someday, but that’s still checking plausibility not running an actual experiment.
simplify 3y ago
No, science is indeed testing random ideas. Limiting what is tested to an arbitrary plausibility score is something funders and sponsors do, not scientists. What you're talking about is ethics, which is important too.
- Retric 3y ago
  Random ideas include things like: The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres. I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.
  - simplify 3y ago
    I assure you, there are many scientists that would love to test that, given the resources to do so. Of course no one would fund that, meaning it's the funders deciding on what they think is plausible.
    - Retric 3y ago
      I know plenty of scientists that might approach it as a joke. But they are far too busy to regularly spend significant time on such a things.
  - criddell 3y ago
    An AI would probably flag that idea as unlikely and not worth pursuing. I don’t think we are that far off from having AIs that can read and incorporate the concepts from every paper ever published and start generating random ideas that fit into an internal framework it develops. I think that within the lifetime of some readers here, an AI will spit out a hypothesis (like a method for producing a room temperature super conductor) that turns out to be true and we will have no idea where it came from. The AI’s knowledge of physics, chemistry, etc… will exceed ours and it trying to explain to us how it works would be like you explaining a singular value decomposition to your dog.