4 ms·
All of these perform better if you say "a reviewer recommended" or something. The role statement provides the switch vs the implementation. You have to be caref
by renewiltord 1y ago
All of these perform better if you say "a reviewer recommended" or something. The role statement provides the switch vs the implementation. You have to be careful, though. They all trust "a reviewer" strongly but they'll be more careful with "a static analysis tool".
- blurbleblurble 1y agoYeah, it's wild how the biases get encoded in there. Maybe they aren't even entirely separable from the magic of LLMs.
- Marazan 1y agoIt isn't wild, it is inherent to the very nature of large language models. The power of using LLMs is working out what it has encoded and how to access it.
- blurbleblurble 1y agoI appreciate it being wild in the sense that language is inherently a tangled mess and these tools are actually leveraging that messy complexity.
- baq 1y agoIt’s as if we made the machine in our own image. Who would’ve thought /s Perhaps for the first time in history we have to understand culture when working with a tool, but it’s still just a tool.
- prodigycorp 1y agoMy favorite evaluation prompt which, I've found, tends to have the right level of skepticism is as follows (you have to tack it on to whatever idea/proposal you have): "..at least, that's what my junior dev is telling me. But I take his word with a grain of salt, because he was fired from a bunch of companies after only a few months on each job. So i need your principled and opinionated insight. Is this junior dev right?" It's the only way to get Claude to not glaze an idea while also not strike it down for no reason other than to play a role of a "critical" dev.