4 ms·

> However if you ask the same question in the same session, output will be different. When isn't that true? int main() { printf("Continue?\n");

by 9rx 9mo ago

> However if you ask the same question in the same session, output will be different. When isn't that true? int main() { printf("Continue?\n"); } and int main() { printf("Continue?\n"); printf("Continue?\n"); } do not see the compiler produce equivalent outputs and I am not sure how they ever could. They are not equivalent programs. Adding additional instructions to a program is expected to see a change in what the compiler does with the program.

troupo 9mo ago
If you ask the compiler to compile the same input, it will produce the same output. With LLMs the output depends on the phases of the moon.
- 9rx 9mo ago
  > If you ask the compiler to compile the same input, it will produce the same output. As with LLMs, unless you ask for the output to be nondeterministic. But any compiler can be made nondeterministic if you ask for it. That's not something unique to LLMs. > With LLMs the output depends on the phases of the moon. If you are relying on a third-party service to run the LLM, quite possibly. Without control over the hardware, configuration, etc. then there is all kinds of fuckery that they can introduce. A third-party can make any compiler nondeterministic. But that's not a limitation of LLMs. By design, they are deterministic.
  - troupo 9mo ago
    > But any compiler can be made nondeterministic if you ask for it. That's not something unique to LLMs. Not unique as in: no one makes their compilers deterministic, and you have to work to make a non-deterministic one. LLMs are non-deterministic by default, and you have to contort them to the point of uselessness to make them deterministic > If you are relying on a third-party service to run the LLM, quite possibly. Without control over the hardware, configuration, etc. Again. Even if you control everything, the only time they produce deterministic output is when they are completely neutered: - workaround for GPUs with num_thread 1 - temperature set to 0 - top_k to 0 - top_p to 0 - context window to 0 (or always do a single run from a new session)
    - 9rx 9mo ago
      > no one makes their compilers deterministic Go (gc) was designed for reproducible builds by default, so clearly that's not true, but you are right that it isn't the norm. Even the most widely recognized and used compilers, like gcc, clang, even rustc, are non-deterministic by default. Only if you work hard and control all the variables (e.g. -frandom-seed) can you make these compilers deterministic. It's fascinating that anyone on HN thinks that compilers converge on always being deterministic or always being non-deterministic. I thought we were supposed to know things about computers around here?
      - troupo 9mo ago
        > no one makes their compilers deterministic I was typing too fast. "No one makes their compilers nondeterministic. > Even the most widely recognized and used compilers, like gcc, clang, even rustc, are non-deterministic by default. Wat? In which world are these compilers producing non-deterministic output if you run them again and again? > It's fascinating that anyone on HN thinks that compilers converge on always being deterministic It's called reality that is also very trivially verified. > I thought we were supposed to know things about computers around here? That's what I thought, too.
        9rx 9mo ago
        > In which world are these compilers producing non-deterministic output if you run them again and again? The one where deterministic iteration order (think use of unordered maps/sets, parallel execution, etc.) isn't considered a priority or is even seen as something to avoid as determinism here can lead to slower compiler times. Where there is use of heuristics that can have a "tie score" without effort to choose the winner deterministically. Where there is a need to support features like __DATE__ and __TIME__. So on and so on. It is a little out of the way so you've probably never heard of it, but its inhabitants call it "Earth".
        troupo 9mo ago
        And those fictional compilers are? Like what fictional compilers when you run them on source code produce a program that may or may not run, may or may not produce desired result etc.? Strange how literally no one is talking about this fictional non-determinism where the output of the compiler is completely different from run to run, but for the "LLMs are strictly deterministic" non-determinism is the norm and everyone expects it. Edit also note how you went from "any compiler can be made nondeterministic if you ask for it." (compilers are deterministic, and you have to work to make them non-deterministic) to "most widely recognized and used compilers are non-deterministic by default." There are specific things (undefined behavior) and specific things (like floating point precision between implementations) that may be specified as non-deterministic. To pretend that this is somehow equal to LLMs is not even being unrealistic. It's to exist on a plane orthogonal to our reality.

by 9rx 9mo ago

troupo 9mo ago

If you ask the compiler to compile the same input, it will produce the same output. With LLMs the output depends on the phases of the moon.

9rx 9mo ago
> If you ask the compiler to compile the same input, it will produce the same output. As with LLMs, unless you ask for the output to be nondeterministic. But any compiler can be made nondeterministic if you ask for it. That's not something unique to LLMs. > With LLMs the output depends on the phases of the moon. If you are relying on a third-party service to run the LLM, quite possibly. Without control over the hardware, configuration, etc. then there is all kinds of fuckery that they can introduce. A third-party can make any compiler nondeterministic. But that's not a limitation of LLMs. By design, they are deterministic.
- troupo 9mo ago
  > But any compiler can be made nondeterministic if you ask for it. That's not something unique to LLMs. Not unique as in: no one makes their compilers deterministic, and you have to work to make a non-deterministic one. LLMs are non-deterministic by default, and you have to contort them to the point of uselessness to make them deterministic > If you are relying on a third-party service to run the LLM, quite possibly. Without control over the hardware, configuration, etc. Again. Even if you control everything, the only time they produce deterministic output is when they are completely neutered: - workaround for GPUs with num_thread 1 - temperature set to 0 - top_k to 0 - top_p to 0 - context window to 0 (or always do a single run from a new session)
  - 9rx 9mo ago
    > no one makes their compilers deterministic Go (gc) was designed for reproducible builds by default, so clearly that's not true, but you are right that it isn't the norm. Even the most widely recognized and used compilers, like gcc, clang, even rustc, are non-deterministic by default. Only if you work hard and control all the variables (e.g. -frandom-seed) can you make these compilers deterministic. It's fascinating that anyone on HN thinks that compilers converge on always being deterministic or always being non-deterministic. I thought we were supposed to know things about computers around here?
    - troupo 9mo ago
      > no one makes their compilers deterministic I was typing too fast. "No one makes their compilers nondeterministic. > Even the most widely recognized and used compilers, like gcc, clang, even rustc, are non-deterministic by default. Wat? In which world are these compilers producing non-deterministic output if you run them again and again? > It's fascinating that anyone on HN thinks that compilers converge on always being deterministic It's called reality that is also very trivially verified. > I thought we were supposed to know things about computers around here? That's what I thought, too.
      - 9rx 9mo ago
        > In which world are these compilers producing non-deterministic output if you run them again and again? The one where deterministic iteration order (think use of unordered maps/sets, parallel execution, etc.) isn't considered a priority or is even seen as something to avoid as determinism here can lead to slower compiler times. Where there is use of heuristics that can have a "tie score" without effort to choose the winner deterministically. Where there is a need to support features like __DATE__ and __TIME__. So on and so on. It is a little out of the way so you've probably never heard of it, but its inhabitants call it "Earth".
        troupo 9mo ago
        And those fictional compilers are? Like what fictional compilers when you run them on source code produce a program that may or may not run, may or may not produce desired result etc.? Strange how literally no one is talking about this fictional non-determinism where the output of the compiler is completely different from run to run, but for the "LLMs are strictly deterministic" non-determinism is the norm and everyone expects it. Edit also note how you went from "any compiler can be made nondeterministic if you ask for it." (compilers are deterministic, and you have to work to make them non-deterministic) to "most widely recognized and used compilers are non-deterministic by default." There are specific things (undefined behavior) and specific things (like floating point precision between implementations) that may be specified as non-deterministic. To pretend that this is somehow equal to LLMs is not even being unrealistic. It's to exist on a plane orthogonal to our reality.