3 ms·
A glaring problem is: Non-determinism of LLMs, creating different answers to the same prompt. I appreciate your blogging and analysis in this space, so I am I a
by dpflan 4y ago
A glaring problem is: Non-determinism of LLMs, creating different answers to the same prompt. I appreciate your blogging and analysis in this space, so I am I am interested in your responses. The non-determinism implies that prompt engineering is brittle, difficult, and prone to no formal evaluation techniques for correctness.
- simonw 4y agoI agree with everything you've said there.
- dpflan 4y agoIt is certainly an interesting phenomenon, and I wonder what techniques from neuroscience for brain mapping could be used for model "brain mapping", which could lend itself more to prompt engineering as a science (latent space mapping).
- jaredsohn 4y agoSet the temperature to zero to make it more deterministic.