4 ms·
> LLM generated code will eventually contain UB. Yes. Even in languages other than C (i.e. you will get behaviour that nothing in the input specified). When
by lelanthran 5mo ago
> LLM generated code will eventually contain UB.
Yes.
Even in languages other than C (i.e. you will get behaviour that nothing in the input specified).
When LLMs generate code, all languages have UB.
- eru 5mo agoThat's a bit silly. UB means literally no restrictions. So if you standard says 'you have to crash with an error message' that's already no longer UB.
- lelanthran 5mo ago> So if you standard says 'you have to crash with an error message' that's already no longer UB. Sure. For crashes. But when you instruct an LLM to do something, the output is probablistic, so you may get behviour that is unexpected and/or unwanted. Like storing security tokens in code. Or nuking the production database.
- eru 5mo agoIf you fix the random seed you use for sampling, your LLM is perfectly deterministic. And there's no requirement for C compilers' UB to be deterministic either.