> how is it that an LLM when given the same prompt does not respond in the same deterministic way? In software (not in the model) here's literally a random num

> how is it that an LLM when given the same prompt does not respond in the same deterministic way? In software (not in the model) here's literally a random number generator that picks from a weighted set of "next-token" choices that the model spits out. The selection process can have a series of knobs to manipulate the responses. If you want it to be deterministic (if you have direct access to the software) you can tell it to set "top-k = 1" or "temperature = 0.0" (depending on your software) and it will be deterministic. Usually the default settings are not for determinism, because for whatever reason the quality of the results tends to not be that good when you go fully d.