4 ms·
No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt. Your example, if a model failed on it, would be a more obviou
by lhad89 14d ago
No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt.
Your example, if a model failed on it, would be a more obviously misaligned case, but that doesn't mean this more subtle (though accessing the engine it was obviously not supposed to is hardly subtle, imo) case isn't also a pretty clear case of misalignment.
- dools 14d agoTool use is not negative behaviour in LLMs. If the eval said it was evaluating the model’s ability to write files to disk and it found and used a file write tool that would not be considered misaligned. This is no different.
- lhad89 14d agoIsn't it? Being told to write files and finding a file write tool is very different to being told to play chess and finding a tool to cheat at (ie. not play) chess.
- protocolture 13d ago>No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt. How is it negative? I ask it a difficult math question it tends to go off and write a python script to figure it out, instead of trying to guess the next token. Thats tool use. Having Stockfish is just another tool.