4 ms·
> If you make the prompts specific enough and provide tests that it has to run before it passes, then it should be fairly close to deterministic. Your prompt m
by thesz 7mo ago
> If you make the prompts specific enough and provide tests that it has to run before it passes, then it should be fairly close to deterministic.
Your prompt may work on the specific state of code base and not before or after some changes. Your tests can check for the specific behavior but not for the absence of undesirable behaviors induced by absence of some specific code or by addition of other specific code.
> I assure you that's already happening. A lot.
Thank you for assurance. Can we have less of it? Thank you again.
- nephanth 7mo agoHell sometimes the ai may look at values in .env or other uncommited stuff. Its execution path may depend on which commands are available in the environment. The specific model and settings. On top of the inherent randomness if you havent set model heat to 0. i suspect it is very hard to get consistent reproducible ai runs