4 ms·

Unfortunately not. Various implementation details like attention are usually non-deterministic. This is one of the better blog posts I'm aware of: https://thin

• 8note 8mo ago
  i dont think theres anything that makes it essentiall that llms are non-deterministic though if you rewrote the math to be all fixed point precision on big ints, i think you would still get the useful LLM results? if somebody really wanted to make a compiler in an LLM, i dont think that nondetermism is problem id really imagine an llm compiler being a set of specs, dependency versions, and test definitions to use though, and you'd introduce essential nondetermism by changing a version number, even if the only change was the version name from "experimental" to "lts"
  • AlotOfReading 8mo ago
    They're not inherently non-deterministic, correct. And floating point is deterministic enough, as that blog post is demonstrating.