4 ms·

How does an LLM muddy the definition of intelligence any more than a database or search engine does? They are lossy databases with a natural language interface,

• tibbar 2y ago
  Ah, but what is in the database? At this point it's clearly not just facts, but problem-solving strategies and an execution engine. A database of problem-solving strategies which you can query with a natural language description of your problem and it returns an answer to your problem... well... sounds like intelligence to me.
  • uoaei 2y ago
    > problem-solving strategies and an execution engine Extremely unfounded claims. See: the root comment of this tree.
    • travisjungroth 2y ago
      …things that look like problem solving strategies in performance, then.
• madethisnow 2y ago
  datasets and search engines are deterministic. humans, and llms are not.
  • hatefulmoron 2y ago
    The LLM's output is chaotic relative to the input, but it's deterministic right? Same settings, same model, same input, .. same output? Where does the chain get broken here?
    • fc417fc802 2y ago
      Now compare a human to an LSTM with persistent internal state that you can't reset.
      • deleted 2y ago
        [deleted]
    • tsimionescu 2y ago
      Depends on what you mean specifically by the output. The actual neural network will produce deterministic outputs that could be interpreted as probability values for various tokens. But the interface you'll commonly see used in front of these models will then non-deterministiclaly choose a single next token to output based on those probabilities. Then, this single randomly chosen output is fed back into the network to produce another token, and this process repeats. I would ultimately call the result non-deterministic. You could make it deterministic relatively easily by having a deterministic process for choosing a single token from all of the outputs of the NN (say, always pick the one with the highest weight, and if there are multiple with the same weight, pick the first one in token index order), but no one normally does this, because the results aren't that great per my understanding.
      • fc417fc802 2y ago
        You can have the best of both worlds with something like weighted_selection( output, hash( output ) ) using the hash as the PRNG seed. (If you're paranoid about statistical issues due to identical outputs (extremely unlikely) then add a nonce to the hash.)
  • semiquaver 2y ago
    LLMs are completely deterministic. Their fundamental output is a vector representing a probability distribution of the next token given the model weights and context. Given the same inputs an identical output vector will be produced 100% of the time. This fact is relied upon by for example https://bellard.org/ts_zip/ https://bellard.org/ts_zip/ a lossless compression system that would not work if LLMs were nondeterministic. In practice most LLM systems use this distribution (along with a “temperature” multiplier) to make a weighted random choice among the tokens, giving the illusion of nondeterminism. But there’s no fundamental reason you couldn’t for example always choose the most likely token, yielding totally deterministic output. This is an excellent and accessible series going over how transformer systems work if you want to learn more. https://youtu.be/wjZofJX0v4M https://youtu.be/wjZofJX0v4M
    • spunker540 2y ago
      i've heard it actually depends on the model / hosting architecture. some are not deterministic at the numeric level because there is so much floating point math going on in distributed fashion across gpus, with unpredictable rounding/syncing across machines
    • frozenseven 2y ago
      >In practice most LLM systems use this distribution (along with a “temperature” multiplier) to make a weighted random choice among the tokens In other words, LLMs are not deterministic in just about any real setting. What you said there only compounds with MoE architectures, variable test-time compute allocation, and o3-like sampling.
  • daveguy 2y ago
    The only reason LLMs are stochastic instead of deterministic is a random number generator. There is nothing inherently non-deterministic about LLM algorithms unless you turn up the "temperature" of selecting the next word. The fact that determinism can be changed by turning a knob is clear evidence that they are closer to a database or search engine than a human.
    • travisjungroth 2y ago
      You can turn the determinism knob on humans. Psychedelics are one method.
      • mrob 2y ago
        I think that's more adjusting the parameters of the built-in denoising and feature detection circuits of the inherently noisy analog computer that is the brain.