13 ms·
I think this might be plausible in the future, but it needs a lot more tooling. For starters you need to be able to run the prompt through the exact same model
by AlexandrB 11mo ago
I think this might be plausible in the future, but it needs a lot more tooling. For starters you need to be able to run the prompt through the exact same model so you can reproduce a "build".
- josephg 11mo agoEven the exact same model isn't enough. There are several sources of nondeterminism in LLMs. These would all need to be squashed or seeded - which as far as I know isn't a feature that openai / anthropic / etc provide.
- BurningFrog 11mo agoOK, then the current models aren't as good as I thought/hoped. I guess one thing it means is that we still need extensive test suites. I suppose an LLM can write those too.