4 ms·

>compilers behaving in a deterministic and predictable way is an important fundamental of pipelines. LLMs are inherently unpredictable, and so using an LLM for

>compilers behaving in a deterministic and predictable way is an important fundamental of pipelines. LLMs are inherently unpredictable, and so using an LLM for compilation / decompilation -- even an LLM that has 99.99% accuracy You're confusing different concepts here. An llm is technically not unpredictable by itself (at least the ones we are talking about here, there are different problems with beasts like GPT4 [1]). The "randomness" of llms you are probably experiencing stems from the autoregressive completion, which samples from probabilities for a temperature T>0 (which is very common because it makes sense in chat applications). But there is nothing that prevents you from simply choosing greedy sampling, which would make your output 100% deterministic and reproducible. That is particularly useful for disassembling/decompiling and has the chance to vastly improve over existing tools, because it is common knowledge that they are often not the sharpest tools and humans are much better at piecing together working code. The other question here is accuracy for compiling. For that it is important whether the llm can follow a specification correctly. Because once you write unspecified behaviour, your code is fair game for other compilers as well. So the real question is how well does it follow the spec how good is it at dealing with situations where normal compilers will flounder. [1] https://152334h.github.io/blog/non-determinism-in-gpt-4/ https://152334h.github.io/blog/non-determinism-in-gpt-4/

yowlingcat 2y ago
This was a big unlock for me -- I recall antirez saying the same thing in a comment replying to me when I asked a similar question about potential LLM features in Redis [1]. [1] https://news.ycombinator.com/item?id=39617370 https://news.ycombinator.com/item?id=39617370
barrkel 2y ago
Determinism for any given input isn't an interesting metric though. Inputs are always different, or else you could just replace it with a lookup function. What's important is reliability of the output given a distribution of inputs, and that's where LLMs are unreliable. Temperature sampling can be a technique to improve reliability particularly when things get into repetitve loops - though usually it's to increase creativity.
- HanClinto 2y ago
  > What's important is reliability of the output given a distribution of inputs, and that's where LLMs are unreliable. Nailed it! Thank you for stating this better than I did. Maybe I should have used the word "trustworthy" instead of "predictable". If I'm using an LLM as a calculator to solve math problems for my users, I may be able to say with certainty that my model is 99% accurate, but I'm not able to know ahead of time which questions it's going to miss, and which ones it's going to get correct. It's a near-infinite input space, so it's difficult to prove correctness via induction. The attention mechanism is (thus far) fairly inscrutable, and thus proving the correctness via deduction is (currently) not possible either. It would feel weird to use an LLM for a calculator, just as it feels weird to use an LLM as a compiler. I want something that uses an algorithm that is provably and predictably correct -- not one that is "almost always" correct. That unpredictability of accuracy isn't connected to the temperature used when sampling, so much as the inscrutable nature of the attention mechanism when verifying the trustworthiness over the range of all possible inputs.
skissane 2y ago
> The "randomness" of llms you are probably experiencing stems from the autoregressive completion, which samples from probabilities for a temperature T>0 (which is very common because it makes sense in chat applications). Even that “random” sampling is deterministic, in that if you use the same PRNG algorithm with the same random seed, then (all else being equal) you should get the same results every time. To get genuine nondeterminism, you need an external source of randomness, such as thermal noise, keystroke timing, etc. (Even whether that is really non-deterministic depends on a whole lot of contested issues in philosophy and physics, but at least we can say it is non-deterministic for all practical purposes.)

yowlingcat 2y ago

This was a big unlock for me -- I recall antirez saying the same thing in a comment replying to me when I asked a similar question about potential LLM features in Redis [1]. [1] https://news.ycombinator.com/item?id=39617370 https://news.ycombinator.com/item?id=39617370

barrkel 2y ago

Determinism for any given input isn't an interesting metric though. Inputs are always different, or else you could just replace it with a lookup function. What's important is reliability of the output given a distribution of inputs, and that's where LLMs are unreliable. Temperature sampling can be a technique to improve reliability particularly when things get into repetitve loops - though usually it's to increase creativity.

HanClinto 2y ago
> What's important is reliability of the output given a distribution of inputs, and that's where LLMs are unreliable. Nailed it! Thank you for stating this better than I did. Maybe I should have used the word "trustworthy" instead of "predictable". If I'm using an LLM as a calculator to solve math problems for my users, I may be able to say with certainty that my model is 99% accurate, but I'm not able to know ahead of time which questions it's going to miss, and which ones it's going to get correct. It's a near-infinite input space, so it's difficult to prove correctness via induction. The attention mechanism is (thus far) fairly inscrutable, and thus proving the correctness via deduction is (currently) not possible either. It would feel weird to use an LLM for a calculator, just as it feels weird to use an LLM as a compiler. I want something that uses an algorithm that is provably and predictably correct -- not one that is "almost always" correct. That unpredictability of accuracy isn't connected to the temperature used when sampling, so much as the inscrutable nature of the attention mechanism when verifying the trustworthiness over the range of all possible inputs.

skissane 2y ago

> The "randomness" of llms you are probably experiencing stems from the autoregressive completion, which samples from probabilities for a temperature T>0 (which is very common because it makes sense in chat applications). Even that “random” sampling is deterministic, in that if you use the same PRNG algorithm with the same random seed, then (all else being equal) you should get the same results every time. To get genuine nondeterminism, you need an external source of randomness, such as thermal noise, keystroke timing, etc. (Even whether that is really non-deterministic depends on a whole lot of contested issues in philosophy and physics, but at least we can say it is non-deterministic for all practical purposes.)