3 ms·
An advanced specialized Model could probably create fleshed out arguments from your thought-bites and subtly manipulate you by weaving other positions you don't
by luckylion 4y ago
An advanced specialized Model could probably create fleshed out arguments from your thought-bites and subtly manipulate you by weaving other positions you don't/didn't have into the argument. "You're right, because xyz is correct and that makes your argument correct", making you more likely to accept xyz as correct, because we like being right and if accepting xyz makes you right in this very moment, that's a small price to pay.
it feels like that would require much more sophistication, but then again who hasn't been amazed how intelligent ChatGPT & friends feel?
- LesZedCB 4y agothe RLHF has indeed made them quite sycophantic. i don't believe in any of the 'latent space wario' stuff, but i really enjoy using it explicitly to get clarification and explanation from opposing viewpoints. i would love to see some experiment where a debate is held/moderated by an LLM which can help find common ground in highly contentious viewpoints. the post-mortem from each side would be a fascinating read.