3 ms·
2 things. 1. If you turn the temperature down too far, the output is just bad and no amount of running prompts optimization will let you hill climb your way to
by sarchertech 5d ago
2 things.
1. If you turn the temperature down too far, the output is just bad and no amount of running prompts optimization will let you hill climb your way to good performance.
2. It’s not about determinism vs non-determinism. It’s about chaos. A perfectly deterministic model is still chaotic. Meaning that very small changes to the input result in very large changes to the output.
Turning temperature down doesn’t actually get you predictable or reproducible behavior across different inputs.
- roughly 5d agoThank you, I forgot about the chaotic aspect, too - that’s the other part that makes this brutal. The bot performs perfectly on your tests, but your customer abhors the Oxford comma, so you, your marketing team and your test suite can go to hell.