3 ms·
Well if you want to use it as a high level language where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeas
by sarchertech 9mo ago
Well if you want to use it as a high level language where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible.
You can’t just tell users “sorry there are a million tiny differences all over the app every time we change the slightest thing, that’s just the ambiguous nature of reality”.
- orbital-decay 9mo ago>where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible What, why would you want to write the code anew? Identify the changes in the spec and bring the existing code in line with them.
- sarchertech 9mo agoThat’s the whole thesis of the article. Using an LLM as a high level language. >The codebase should be reconstructable from the documentation
- orbital-decay 9mo agoAnd this kind of reproducibility is insensitive to prompt instability. If your inference is deterministic (trivial if within the lifetime of a snapshot), you give it the same docs and get the same result.
- sarchertech 9mo agoObviously this article is advocating for spec as source code and not checking in code to keep it around between “compilations”. Why would you care about reproducibility at all if you were checking in the code as well. If you check in the code, you’d never need to rerun the same prompt.