> I want to ask, for example, how is it that an LLM when given the same prompt does not respond in the same deterministic way? You can control that in most sys

> I want to ask, for example, how is it that an LLM when given the same prompt does not respond in the same deterministic way? You can control that in most systems with an inference-set parameter called "temperature". But setting the temperature as low as possible tends to lead to very low-quality answers - the system can't crawl out of some local optimum and ends up repeating itself over and over. Such answers may be "deterministic" but they're also not good.