4 ms·

Except the code it creates is deterministic.

• danparsonson 1y ago
  LLMs almost always operate at least partly stochastically.
  • nojito 1y ago
    Of course but the code they generate only operates in one way.
    • croon 1y ago
      Who determines if that code operates in the intended way?
• close04 1y ago
  I don't know what you mean with "the code it creates is deterministic" but the process an LLM uses to generate code based on an input is definitely not entirely deterministic. To put it simply, the chances that an LLM will output the same result every time given the same input is low. The LLM does not operate deterministically, unlike the manufacturing robot who will output the same door panel every single time. Or as ChatGPT put it: > The likelihood of an LLM like ChatGPT generating the exact same code for the same prompt multiple times is generally low.
  • 33hsiidhkl 1y ago
    For any given seed value, the output of an LLM will be identical- it is deterministic. You can try this at home with Llama.cpp by specifying a seed value when you load a LLM, and then seeing that for a given input the output will always be the same. Of course there may be some exceptions (cosmic ray bit flips). Also, if you are only using online models, you can't set the seed value, plus there are multiple models, so multiple seeds. In summary, LLMs are deterministic.
    • skeeter2020 1y ago
      I feel this is technically correct but intentionally cheating. no one - including the model creators - expects that to be the interface; it undermines they entire value proposition of using an LLM in the first place if I need to engineer the inputs to ensure reproducability. I'd love to hear some real world scenarios that do this where it wouldn't be simpler to NOT use AI.
      • 33hsiidhkl 1y ago
        When should a model's output be deterministic? When should a model's output be non-deterministic? When many humans interact with the same model, then maybe the model should try different seed values, and make measurements. When model interaction is limited to a single human, then maybe the model should try different seed values, and make measurements.
      • deleted 1y ago
        [deleted]
    • close04 1y ago
      > the process an LLM uses to generate code based on an input is definitely not entirely deterministic Technically correct is the least useful kind of correct when it's wrong in practice. And in practice the process AI coding tools use to generate code is not deterministic which is what matters. To make matters worse in the comparison with a manufacturing robot, even the input is never the same. While a robot get the exact command for a specific motion and the exact same piece of sheet metal, in the same position, a coding AI is asked to work with varied inputs and on varied pieces of code. Even stamping metal could be called "non-deterministic" since there are guaranteed variations, just within determined tolerances. Does anyone define tolerances for generated code? That's why the comparison shows a lack of understanding of either of the systems.
      • deleted 1y ago
        [deleted]
      • 33hsiidhkl 1y ago
  • nojito 1y ago
    It's simple. You run the code generated by the llm you will get a deterministic result of said code.