4 ms·

> Am I completely missing the idea? I don't think trust in LLMs is all that different from trust in code typically is. It's basically the same as trusting a clo

by samtho 3y ago

> Am I completely missing the idea? I don't think trust in LLMs is all that different from trust in code typically is. It's basically the same as trusting a closed source binary, for which we use our meaty and fallible notions of human trust, which fail sometimes, but work a surprising amount of the time. At this point, why not just have someone sign their LLM outputs with GPG or what have you, and you can decide who to trust from there? This has been my problem with LLMs from day one. Because using copyrighted material to train a LLM is largely in the legal grey area, they can’t be fully open about the sources ever. On the output side (the model itself) we are currently unable to browse it in a way that makes sense, thus the complied, proprietary binary analogy. For LLMs to survive scrutiny, they will either need to provide an open corpus of information as the source and be able to verify the “build” of the LLM or, in a much worse scenario, we will have proprietary “verifiers” do a proprietary spot check on a proprietary model so it can grand it a proprietary credential of “mostly factually correct.” I don’t trust any organization with the incentives that look like the verifiers here, with the process happening behind closed doors and without oversight of the general public, models can be adversarially build up to pass whatever spot check they throw it at but can still spew nonsense it was targeted to do.

circuit10 3y ago
> Because using copyrighted material to train a LLM is largely in the legal grey area, they can’t be fully open about the sources ever. I don’t think that’s true, for example some open source LLMs have the training data publicly available, and hiding evidence of something you think could be illegal on purpose sounds too risky for most big companies to do (obviously that happens sometimes but I don’t think it would on that scale)
- samtho 3y ago
  While there may be some, the most notable ones seem to hide behind the veil of “proprietary training data” but assuming the data is open, the method to generate the model must also be reproducible, thus the toolchain need to be open too. I don’t think there is a lot of incentive to do this.
  - ronsor 3y ago
    But GPU-based training of models is inherently non-deterministic
    - Dylan16807 3y ago
      In what way? If you keep your ordering consistent, and seed any random numbers you need, what's left to be a problem?
      - Imnimo 3y ago
        "Inherently" might be too strong of a word, but the default implementations of a lot of key operations are nondeterministic on GPU. With the parallel nature of GPU compute, you can often do things faster if you're willing to be a bit loosey-goosey. PyTorch and TF will typically provide deterministic alternatives, but those come at a cost of efficiency, and might be impractical for LLM training runs that are already massively expensive. https://pytorch.org/docs/stable/notes/randomness.html https://pytorch.org/docs/stable/notes/randomness.html
        Dylan16807 3y ago
        I wonder what the actual speed difference is. I couldn't find any benchmarks.
      - rlupi 3y ago
        The inherent faults or even just speed differences[] of hardware. [] In the real world, a lot of resources are oversubscribed and not deterministic. Just think about how scheduling and power management work in a processor. Large model training happens across thousands to millions of processors (think about all the threads in a GPU * the number of GPUs needed, and add the power throttling that modern computing does to fit their power envelopes at all level... and power is just one dimension, memory and network bandwidth are others sources of randomness too). Making such a training deterministic means going as slow as the slowest link in the chain, or having massive redundancies. I suppose we might be able to solve this eventually, perhaps with innovations in the area of reversible computing (to cancel out undeterminism post-facto), but the current flavor of deep-learning training algorithms can't.

by samtho 3y ago

circuit10 3y ago

> Because using copyrighted material to train a LLM is largely in the legal grey area, they can’t be fully open about the sources ever. I don’t think that’s true, for example some open source LLMs have the training data publicly available, and hiding evidence of something you think could be illegal on purpose sounds too risky for most big companies to do (obviously that happens sometimes but I don’t think it would on that scale)

samtho 3y ago
While there may be some, the most notable ones seem to hide behind the veil of “proprietary training data” but assuming the data is open, the method to generate the model must also be reproducible, thus the toolchain need to be open too. I don’t think there is a lot of incentive to do this.
- ronsor 3y ago
  But GPU-based training of models is inherently non-deterministic
  - Dylan16807 3y ago
    In what way? If you keep your ordering consistent, and seed any random numbers you need, what's left to be a problem?
    - Imnimo 3y ago
      "Inherently" might be too strong of a word, but the default implementations of a lot of key operations are nondeterministic on GPU. With the parallel nature of GPU compute, you can often do things faster if you're willing to be a bit loosey-goosey. PyTorch and TF will typically provide deterministic alternatives, but those come at a cost of efficiency, and might be impractical for LLM training runs that are already massively expensive. https://pytorch.org/docs/stable/notes/randomness.html https://pytorch.org/docs/stable/notes/randomness.html
      - Dylan16807 3y ago
        I wonder what the actual speed difference is. I couldn't find any benchmarks.
    - rlupi 3y ago
      The inherent faults or even just speed differences[] of hardware. [] In the real world, a lot of resources are oversubscribed and not deterministic. Just think about how scheduling and power management work in a processor. Large model training happens across thousands to millions of processors (think about all the threads in a GPU * the number of GPUs needed, and add the power throttling that modern computing does to fit their power envelopes at all level... and power is just one dimension, memory and network bandwidth are others sources of randomness too). Making such a training deterministic means going as slow as the slowest link in the chain, or having massive redundancies. I suppose we might be able to solve this eventually, perhaps with innovations in the area of reversible computing (to cancel out undeterminism post-facto), but the current flavor of deep-learning training algorithms can't.