4 ms·

LLMs are inherently bad at this whole "correctness" thing...

• amelius 3y ago
  Well, not if you force them to use only correctness-preserving transforms. Just reject anything else.
  • dark-star 3y ago
    So how would you prove that the transforms it conjures up are in fact correctness-preserving? Also, if you only allow correctness-preserving transforms, why would you need an LLM in the first place? Just apply those correctness-preserving transforms directly (that's basically how recompilers work, which have been successful in doing that since the late 90s)
    • amelius 3y ago
      1. You write and prove the correctness preserving transforms yourself. They are low-level, so you only need to do a small number of them. 2. The aim is to apply the transforms in such a way that you get different code, which is still efficient, or maybe even more efficient. This is where you rely on the "intuition" of the LLMs. No guarantees there, that's why it was a question.
      • dark-star 3y ago
        ...but that is exactly what optimizing compilers (and re-compilers) do since forever, and they are already quite good at that. I fail to see how an LLM could magically improve that.
        • amelius 3y ago
          Because LLMs have shown that they can reason about code on a higher, more abstract level than a typical compiler.
          • ang_cire 3y ago
            That's just not true. They're not "reasoning" anything, they're assembling code that appears similar to something they were trained on, within whatever additional language training constraints they're given. I've never had an LLM spit out code that didn't need fixing, which defeats the purpose of preferring them over a re-compiler anyways.