3 ms·
It is 100% possible for performance regressions to occur by changing the model pipeline and not the model itself. A system prompt is a part of said pipeline. P
by minimaxir 3y ago
It is 100% possible for performance regressions to occur by changing the model pipeline and not the model itself. A system prompt is a part of said pipeline.
Prompt engineering is surprisingly fragile.
- mirsadm 3y agoIs that surprising? Seemed like a giant hack to me. Prompt engineering sure sounds better than hack though.
- minimaxir 3y agoIt is a necessary hack, though.
- refulgentis 3y agoAbsolutely! That was covered in the tweet link. If you're suggesting they're lying*, I'm happy to extract it and check. * I don't think you are! I've looked up to you a lot over last year on LLMs btw, just vagaries of online communication, can't tell if you're ignoring the tweet & introducing me to idea of system prompts, or you're suspicious it changed recently. (in which case, I would want to show off my ability to extract system prompt to senpai :)
- minimaxir 3y agoI was agreeing with the tweet and think Anthropic is being honest, my comment was more for posterity since not many people know the difference between models and pipelines. Thanks for liking my work! :)