5 ms·
One of the great things about LLMs: since they aren’t deterministic prompt engineering will be even more susceptible to magical thinking than it has been with G
by dwighttk 4y ago
One of the great things about LLMs: since they aren’t deterministic prompt engineering will be even more susceptible to magical thinking than it has been with Google
- sebzim4500 4y agoThe big difference is feedback loop. With prompt engineering you can change your prompt slightly and immediately see if benchmarks improve. With SEO you had to wait at least days, and even then it was impossible to rule out external effects like others adding/removing links to you.
- deepGem 4y agoAnyone who had to deal with retraining models understands the pain of retraining. What prompts offer as a feature while at the surface may seem superficial, they are groundbreaking, as they completely remove re-training and the entire ML-Ops deployment lifecycle.
- mirekrusin 4y agoYou can set temperature to 0 to have deterministic output per model.
- ChatGTP 4y agoOnly if the model stays the same? As they’re constantly being tuned this is almost never true.
- mirekrusin 4y agoCan you give an example where it matters?
- ChatGTP 4y agoYes, I’ve seen people at work calling it directly to generate code to talk to APIs with the temp set to zero. They still frequently receive different or broken code.
- mirekrusin 4y agoShouldn't it be persisted/cached once working, possibly with some test/refine/retry? Requerying everything from scratch when using seems like not optimal solution there?
- deleted 4y ago[deleted]